Creating GCP Roles and Service Accounts (using scripts)
In order for the Velostrata operations to execute properly, there are a number of roles and service accounts that need to be created in GCP first. In GCP, roles are a set of permissions, and service accounts are assigned these roles, in this case within the context of either a project or a service account. For a detailed explanation of the permissions in each role, you can view the YAML files that we'll provide to create them. For more information on what each service account will entail, please read below:
Velostrata Management Service Account (velos-gcp-mgmt-sa):
Enables Velostrata to create objects within the GCP project(s) and then manage those objects based on the Velostrata management tasks that are invoked. For example, this service account will be used to create all the elements that a CloudExtension needs (compute instance creation, cloud storage buckets created, etc.). These are all tasks which are invoked in the GCP project, which velostrata needs permission to perform [via this service account].
Velostrata Cloud Extension Service Account (velos-gcp-ce-sa):
Will leverage a subset of GCP storage permissions, such that it is allowed to read/write storage objects from cloud storage, manage those objects, as well as data requests between the Velostrata cache and cloud storage. All tasks are related to getting things from the source (on-prem or another cloud) and into GCP and would include operations like data reads and data commits, for example.
Velostrata Project Worker Service Account (velos-gcp-worker-sa):
Leverages the same subset of GCP storage permissions as velos-ce-sa, but this service account is only used when Velostrata’s ‘prepare to detach’ operation is invoked. This service account will be used by the Velostrata Worker Auxiliary VM which will prepare the native storage in the cloud and copy data from the cloud storage bucket into the disks that are attached to the Velostrata Worker Auxiliary VM.
How these service accounts are defined will also vary based on your project architecture within GCP. If you are using a standalone project, these roles and service accounts are all created in that standalone project. If you are using an organization with multiple projects, all of the roles will be created within the organization but the service accounts and assignments will take place in various project locations.
The easiest way to create the appropriate service accounts is by using scripts, which we outline below. Using scripts should reduce the time/effort it takes to setup your Velostrata deployment for migration. Though not recommended, you can also accomplish these actions manually if desired.
Before using these scripts, you must ensure your GCP project(s) meet the following requirements:
The following APIs should be enabled by the user or will be enabled by the script:
- Cloud Resource Manager API
- Identity and Access Management (IAM) API
- Compute Engine API
- Google Cloud Storage API
- Google Cloud Deployment Manager API
The user which runs the cloud shell should have the following roles:
- Compute Admin
- Organization Administrator
To see even more information about these scripts, after downloading please unzip the package and refer to the README.TXT file.
Creating a Projects file:
sudo touch projects.csv (can also be a simple txt)
sudo vi projects.csv
Input projects ids
ESC > wq!
Running the scripts
The creation scripts can be found within the cloud shell image under /google/velostrata. You can also run the script with -h or --help to get help at any time. The script has 2 phases: List projects (discovery) and Deploy.
Phase 1: List projects
The list projects phase let's you see all of the available projects. You will have to identify the project(s) that you're looking to migrate to, which is the purpose of these scripts. Here are some examples:
- To list to the console all the projects the user has permissions to:
sudo ./velos_sa_roles.py list-projects
- To list to output file all the projects the user has permissions to:
sudo ./velos_sa_roles.py list-projects --projects-file
- To list to the console all the projects in the specified organization:
sudo ./velos_sa_roles.py list-projects --org-id
- To list to output file all the projects in the specified organization:
sudo ./velos_sa_roles.py list-projects --org-id --projects-file
Phase 2: Deploy phase
The Deploy phase is used to create and assign the service accounts that Velostrata uses in the backend to assist the migration operations. This script has the following arguments:
sudo ./velos_sa_roles.py deploy --host-proj-id> HOST_PROJ_ID --ce-proj-id> CE_PROJ_ID (--org-id ORG_ID or --projects-file PROJECTS_FILENAME) [--audit]
Here is appropriate context for these parameters:
host-proj-id - mandatory. The id of the project which shares one or more VPCs. This will be the Management project.
ce-proj-id - mandatory. The id of the project of the cloud extensions
org-id - When using org-id the tool will create Velostrata Worker Service account in all projects that belong to the org.
projects-file - File which contains the IDs of the projects to be used as projects for workloads and exporters (might be the output file of the list project phase)
Only one of the arguments ‘org-id’ and ‘projects-file’ should be supplied.
audit - enables the user to review the generated script. The generated script name has the form: ‘deployment_<RANDOM>.sh’. The generated script should be ran manually.
Here are some examples of using this command:
- To deploy single project:
Projects file (projects.csv) should contains only the single project id, for example single_proj_id
sudo ./velos_sa_roles.py deploy --host-proj-id single_proj_id --ce-proj-id single_proj_id --projects-file projects.csv
- To deploy multiple project:
Projects file (projects.csv) should contain a list of target projects for workloads migration (one project per line)
sudo ./velos_sa_roles.py deploy --host-proj-id host_proj_id --ce-proj-id ce_proj_id --projects-file projects.csv
- To review the deployments that will be created:
sudo ./velos_sa_roles.py deploy --host-proj-id host_proj_id --ce-proj-id ce_proj_id --projects-file projects.csv --audit
** The generated script should be run manually in order to create the deployments
- To deploy multiple projects and use all projects in the organization as target projects for workloads:
sudo ./velos_sa_roles.py deploy --host-proj-id host_proj_id --ce-proj-id ce_proj_id --org-id
If you need to rollback your service account setup, you can use this script to do that as any time you run the velos_sa_roles.py script, it also generates a rollback script with the name ‘deployment_rollback_<RANDOM>.sh’.
Both generated scripts has the same random suffix.
For the sake of deploying these service accounts to your projects, here is an example of how to use these scripts for both phases that we talked about above:
First, list the project(s) in the organization:
sudo ./velos_sa_roles.py list-projects --org-id 3335 --projects-file projects
Open the file ‘projects’ (vi/vim) and leave the projects ids which should be projects for migrating workloads. Each project ID should be in a separate line.
Then run the deploy phase:
sudo ./velos_sa_roles.py deployment --host-proj-id --ce-proj-id --projects-file
Note: you can run the script with -h or --help to get help at any time.