How To Use gcloud in Third Party CI/CD Services Such As Circle CI and Gitlab CI

This a guide on how to use Google Cloud Platform (GCP) resources in other environments outside the GCP console. This is useful in many instances such as when you want to implement a multi-cloud solution or want to perform builds and deployments in your own server. In this particular guide, I will show you how to do for third-party continuous integration/continuous deployment (CI/CD) services.

25/09/2020

Steps

The main things you need to get are:

  1. Enable the API for the service you need
  2. Create a Service Account for use in the deployment
  3. Give the Service Account a sufficient role related to the API you want to access (You don’t have to do it at this point. This is something you can do later).
  4. Generate and download the Service Account Private Key in JSON format.
  5. Copy the contents of that JSON file into a protected environment variable in your CI/CD service of choice.
  6. n your dev project, add an appropriate CI/CD YAML file depending on the requirements of your third party service.
  7. An example YAML for Gitlab CI is shown below. But the concepts are universal.

Gitlab CI Example

Google provides a docker image already configured with gcloud. The image is google/cloud-sdk.

In this example, we will be build a docker image, save it to Container Registry and then deploy to Cloud Run.

Variables Needed

VariableValue
GCP_SERVICE_KEY{ Service Account Private Key }
GCP_PROJECT_ID{ Project ID }

Resources/ APIs Needed

  1. Cloud Build
  2. Cloud Storage
  3. Container Registry

Permissions/ Roles Needed by Service Account

ResourceRolePath To Role
Cloud BuildCloud Build Service AccountService Agent Roles/ Cloud Build Service Account

Other Steps Needed

  1. Create a storage bucket in Cloud Storage.
  2. Create two folders. Namely:
    • source - to store source files you send to Cloud Build
    • logs - to store logs from build

The Config File

variables:
  APPLICATION_NAME: demo-application

image: google/cloud-sdk

build-image:
  scripts:
    - echo $GCP_SERVICE_KEY > $HOME/gcloud-service-key.json
    - gcloud auth activate-service-account --key-file $HOME/gcloud-service-key.json
    - rm $HOME/gcloud-service-key.json
    - gcloud --quiet config set project $GCP_PROJECT_ID
    - cloud builds submit . --tag gcr.io/$GCP_PROJECT_ID/$APPLICATION_NAME

Reference

  1. Build a CI/CD Pipeline for Google App Engine Site Using CircleCI [Full Walkthrough] - YouTube
  2. https://gist.github.com/troyharvey/bae82c86c27a3aa539dea83857ee9ecd/
  3. password-protected-docs/config.yml at master · sidpalas/password-protected-docs · GitHub
  4. https://circleci.com/docs/2.0/google-auth/
  5. gcloud auth activate-service-account  |  Cloud SDK Documentation