Setting up GitLab Auto DevOps Lab

17 Nov 2021

GitLab Auto DevOps is a great tool for managing code building and delivery to Kubernetes clusters. It introduces an opinionated way to handle Kubernetes deployments and Environment management. It is used extensively by small and big projects, and the whole concept is designed to be scalable and extensible.

In this tutorial I would like to dive into setting up a working Auto DevOps lab to provide a quick way to get a feeling of the Gitlab Kubernetes CI CD approach.

GitLab Auto DevOps includes many built-in features for all stages of CI/CD process:

Building

Build

Scan dependencies

Testing

Test

Code quality

Performance testing

Container scanning

License compliance

Deploying

Deploy

Review

It even includes other out of the box features to sopport more ops processes, like Monitoring and Security testing.

I believe that the best way to understand a solution is to get the hands dirty and play with it, so lets start with creating a new directory for the lab: mkdir gitlab-test && cd gitlab-test

Dependencies

Make sure to have all dependencies listed below installed:

Components of the lab

The lab requires the following componets to run:

Kubernetes Cluster to run the demo workloads
GitLab Server to host repositories and trigger CICD pipelines
GitLab Runner to run the pipeline jobs

Create a Kubernetes cluster

We will use a Kind cluster as a target for our CICD pipeline. Kind is a great tool to run K8s in Docker.

Create test Kind cluster:

cat << EOF > cluster.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    extraMounts:
      - containerPath: /etc/docker/certs.d
        hostPath: certs.d
    extraPortMappings:
      - containerPort: 80
        hostPort: 80
        protocol: TCP
      - containerPort: 443
        hostPort: 443
        protocol: TCP
    labels:
      ingress-ready: "true"
containerdConfigPatches:
  - |-
    [plugins."io.containerd.grpc.v1.cri".registry.configs."gitlab:5050".tls]
      ca_file = "/etc/docker/certs.d/gitlab:5050/ca.crt"
EOF

kind create cluster --config cluster.yaml --name test

Add Ingress controller to cluster

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml

Set up GitLab

Self-signed certificate for Docker registry

GitLab Auto DevOps is designed for secure docker registry, and if we want to use default actions without modifications, we need to use a registry with TLS. This is just a demo, so certificate details (besides Common Name) are just placeholders.

Generate self-signed certificates for registry:

mkdir -p certs.d/gitlab:5050
openssl req -new \
      -x509 \
      -nodes \
      -days 1000 \
      -subj "/C=US/CN=gitlab" \
            -newkey rsa:2048 \
      -keyout ca.key > certs.d/gitlab:5050/ca.crt

Create a client certificate request:

openssl req -new \
      -nodes \
      -days 365 \
      -subj "/C=US/CN=gitlab" \
            -newkey rsa:2048 \
      -keyout certs.d/gitlab:5050/client.key -out client.csr

Sign a client certificate request:

openssl x509 -req \
       -in client.csr \
       -days 365 \
       -CA certs.d/gitlab:5050/ca.crt \
       -CAkey ca.key \
       -set_serial 01 > certs.d/gitlab:5050/client.cert

Docker Compose and GitLab Server configuration

GitLab Server and Runner with be set up with Docker Compose. This demo is targeted specifically on the Auto DevOps feature, so all other nuances of GitLab setup were intentionally left off. You will also notice some shortcuts which you might not want to do on production.

Create a Gitlab docker-compose.yaml file

# docker-compose.yaml
version: "3.8"
services:
  gitlab:
    image: gitlab/gitlab-ce:latest
    restart: always
    hostname: gitlab
    ports:
      - '8080:80'
      - '2222:22'
    volumes:
      - 'gitlab-config:/etc/gitlab'
      - 'gitlab-logs:/var/log/gitlab'
      - 'gitlab-data:/var/opt/gitlab'
      - './gitlab.rb:/etc/gitlab/gitlab.rb'
      - type: bind
        source: ./certs.d/gitlab:5050/client.cert
        target: /etc/gitlab/ssl/client.cert
      - type: bind
        source: ./certs.d/gitlab:5050/client.key
        target: /etc/gitlab/ssl/client.key
  gitlab-runner:
    image: 'gitlab/gitlab-runner:alpine'
    restart: always
    hostname: 'gitlab-runner'
    volumes:
      - '/var/run/docker.sock:/var/run/docker.sock'
      - 'gitlab-runner-config:/etc/gitlab-runner'
networks:
  default:
    external: true
    name: kind
volumes:
  gitlab-config:
  gitlab-logs:
  gitlab-data:
  gitlab-runner-config:

There are some notable things in this yaml:

Self-signed certificates for Docker registry are bound to GitLab Server with long syntax to handle the colon (‘:’) symbol
Compose is connected to ‘kind’ network which is the default for our Kubernetes cluster
GitLab Runner config is mounted to a docker volume to survive restarts for configuration (will be described below)

Now create a ‘gitlab.rb’ file which will be used by the GitLab image to configure itself:

# gitlab.rb
registry_external_url "https://gitlab:5050"
registry_nginx["ssl_certificate"] = "/etc/gitlab/ssl/client.cert"
registry_nginx["ssl_certificate_key"] = "/etc/gitlab/ssl/client.key"

This configuration defines the certificate for container registry. See official documentation here.

Start GitLab

At this point, we shold be able to start GitLab. In case something goes wrong, omit the ‘-d’ flag and check the logs while it starts:

docker compose up -d gitlab

It takes some time to do initial configuration, so be patient. It is possible to see the logs with docker commands, for example: docker logs -f gitlab-test_gitlab_1.

GitLab initial root password can be retrieved like this:

docker exec -it gitlab-test_gitlab_1 cat /etc/gitlab/initial_root_password

Open GitLab http://localhost:8080 to make sure it works.

GitLab Login

GitLab Runner

We need at least one GitLab runner to pick up and process the jobs from Auto DevOps pipeline.

Open Admin –> Overview –> Runners page in GitLab to find the runner registration token.

docker run --rm -v gitlab-test_gitlab-runner-config:/etc/gitlab-runner --network="kind" gitlab/gitlab-runner register \
  --non-interactive \
  --executor "docker" \
  --docker-image alpine:latest \
  --url "http://gitlab" \
  --registration-token "<runner_registration_token>" \
  --description "docker-runner" \
  --tag-list "docker" \
  --run-untagged="true" \
  --locked="false" \
  --access-level="not_protected" \
  --docker-privileged \
  --docker-network-mode "kind" \
  --docker-volumes "$(pwd)/certs.d:/etc/docker/certs.d"

Here you can find the official documentation on Runner registration.

Some notable parameters here:

--docker-privileged is required to run docker in docker for AutoDevops feature (see here).
--docker-network-mode "kind" is required to connect do a Kubernetes network.
--docker-volumes "$(pwd)/certs.d:/etc/docker/certs.d" is mounting container registry certificates

You should see something like this if the registration was successful:

Registering runner... succeeded                     runner=GUaRkcSe
Runner registered successfully. Feel free to start it, but if it's running already the config should
be automatically reloaded!

Start a new runner:

docker-compose up -d gitlab-runner

Create a Service Account for Kubernetes Cluster connection

Create a ServiceAccount in Kubernetes cluster which will be used by GitLab. Create new manifests with ServiceAccount configuration:

# gitlab-sa.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: gitlab-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: gitlab
    namespace: kube-system

Apply the manifests:

kubectl apply --context kind-test -f gitlab-sa.yaml

Configure Kubernetes Cluster connection

First we need to enable access to local network. Click Menu (on top) –> Admin. Then go to Settings –> Network –> Outbound requests, select the Allow requests to the local network from web hooks and services setting.

Click Menu –> Admin, then Infrastructure –> Kubernetes clusters, after that click Integrate with a cluster certificate

Choose Connect existing cluster and fill in the fields:

Kubernetes cluster name: kind-test
Environment scope: *
API URL: https://test-control-plane:6443

CA Certificate - retrieve certificate contents with the following command:

kind get kubeconfig --name test | grep certificate-authority-data | awk '{print $2}' | base64 -d

Service token - retrieve the token with command:

kubectl --context kind-test -n kube-system describe secret $(kubectl -n kube-system get secret |
grep gitlab | awk '{print $1}') | grep token:

Leave all the other fields with default settings. Click Add Kubernetes cluster.

In kind-test cluster settings, update Base domain to 127.0.0.1.nip.io Click Save changes.

In case you face issues, you can check the instructions from GitLab

Set up a new Project

Go to Projects –> New Project –> Create blank project Create a project called myapp

We need only two files in the repo:

app.js
package.json

Either create them from UI or glone a repo with git clone http://localhost:8080/root/myapp.git

// app.js
const http = require('http');

const hostname = '0.0.0.0';
const port = 5000;

const server = http.createServer((req, res) => {
  res.statusCode = 200;
  res.setHeader('Content-Type', 'text/plain');
  res.end('Hello World!');
});

server.listen(port, hostname, () => {
  console.log(`Server running at http://${hostname}:${port}/`);
});

package.json:

{
  "name": "myapp",
  "description": "an app",
  "version": "1.0.0",
  "engines": {
    "node": "14.x"
  },
  "scripts": {
    "start": "node app.js"
  }
}

Check the Pipelines

Open CI/CD –> Pipelines and check for Auto DevOps pipeline.

GitLab Login

If all done right, Auto DevOps pipeline will run and deploy the application to the Kind cluster. You will then be able to access the app at https://root-myapp.127.0.0.1.nip.io. Ignore the certificate error, since we use a self-signed default certificate from our nginx ingress controller.

GitLab Login

Setting up GitLab Auto DevOps Lab

Dependencies

Components of the lab

Create a Kubernetes cluster

Set up GitLab

Self-signed certificate for Docker registry

Docker Compose and GitLab Server configuration

Start GitLab

GitLab Runner

Create a Service Account for Kubernetes Cluster connection

Configure Kubernetes Cluster connection

Set up a new Project

Check the Pipelines

Related Posts

AWS Reserved Instances 28 Jun 2022

AWS EKS OIDC with Google Workspace 23 Apr 2022

IAM User Traceability in AWS EKS 19 Dec 2021