CI/CD for Google Cloud Functions in a mono repo

While preparing for a new enterprise-grade project we had a new challenge. The project will benefit from Google Cloud Platform and will contain 500+ Google Cloud Functions as a part of serverless components.

During early meetings for our source control architecture, we decided to contain all Google Cloud Functions into single repo (Mono-Repo). The challenge was how to implement CI/CD pipeline for the functions.

I decided to take care of the issue and automate the build pipeline, my criteria were as follows:

1. Build/Deploy must be triggered through pushing to GitHub
2. Only changed functions will be rebuilt and deployed to optimize resource allocation and reduce total build time.
3. All functions must remain in a single GitHub repository.
4. The process must be totally transparent, so developers remain in focus.

So, to meet the criteria and accomplish the task, I wrote down the needs:

1. Process start trigger
2. Identify changed functions
3. Loop through changed functions and deploy one-by-one

My solution was to depend on GitHub actions, which is a very good tool for CI/CD automation with a broad spectrum of flexibility. Here are the steps:

Step 1: Structure Mono Repo

1. On my development WSL created a directory, to host all of the project functions and acts as Mono-Repo
2. initiated git repository, created .gitignore file and other required files.
3. created separate directory for each function, the convention used:
   1. each function in a sub-directory
   2. the directory name must start with func, followed by function name.
   3. function entry point (.js file) and exported function must be the same as the folder name

Step 2: Build the trigger

As my repo is ready, I created a new repository in our GitHub organization's account, and added it as remote to the local mono repo, then head to Actions tab to create new CI/CD action. Action triggered for each new push to the main branch, see the code

Step 3: Identify Changed Functions

This is the core of the solution! To identify changed functions, I used git diff command. Git diff is part of Git plumbing tools. It can be used to git changed files or directories between two commits, branches, etc. So, I used git diff and piped results into a series of manipulations to get directories only, filter for directories containing func word (to follow the convention), and convert results into JSON like array of elements. The result will be like this

[“func1”, “func3”, “fun10,”, …]

GitHub actions has a nice feature called matrix that holds elements you can loop on. To this step, I used the matrix to store JSON array.

Step 4: Loop through change functions and deploy on-by-one

Once I have the matrix, it was easy to run google-github-actions/deploy-cloud-functions@v0.1.2 action to deploy changed functions as GitHub actions will loop through each element of the matrix and deploy it one by one.

The full yaml file is here

Thank you for reading and you can read also: 

Business Management Systems

Business Process and Digital Transformation