Tag: Docker

Install and update GitLab Runner

runner at start block

I already wrote a summary on how to Setup GitLab Runner for Docker containers on Synology NAS. As this article has a lot of details for the Synology setup, I decided to write a short summary for this topic on a regular Linux server. So let’s go…

Before we start, we need to have Docker Engine installed on our system. Check out the Docker docs on how to install the Docker Engine on your system.

Install GitLab Runner

To install GitLab Runner, simply pull the container by using:

docker pull gitlab/gitlab-runner:latest

Now you can start the container as described in the section Start GitLab Runner below.

Update GitLab Runner

To update GitLab Runner, you also have to pull the container by using:

docker pull gitlab/gitlab-runner:latest

Then you have to stop and remove the current container image:

docker stop gitlab_runner
docker rm gitlab_runner

Now you can start the new container as described below.

Start GitLab Runner

docker run --detach \
  --name gitlab_runner \
  --restart always \
  --network host \
  -v /run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:latest

And finally you have to register the runner in your GitLab instance. In the following commands, you need to set your own values for <host.example.com> and <registration_token>:

docker exec -it gitlab_runner gitlab-runner register \
  --url https://<host.example.com>/ \
  --registration-token <registration_token>

The registration will ask for some input. In my case, I used a docker executor with the alpine:latest image:

untime platform                                    arch=amd64 os=linux pid=16 revision=e95f89a0 version=13.4.1
Running in system-mode.                            
                                                   
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
https://<host.example.com>/
Please enter the gitlab-ci token for this runner:
<registration_token>
Please enter the gitlab-ci description for this runner:
[synology]: docker_alpine
Please enter the gitlab-ci tags for this runner (comma separated):

Registering runner... succeeded                     runner=sA4DKorC
Please enter the executor: docker-ssh, parallels, docker+machine, kubernetes, docker, shell, ssh, virtualbox, docker-ssh+machine, custom:
docker
Please enter the default Docker image (e.g. ruby:2.6):
alpine:latest
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

Register GitLab Runner in GitLab >= 16.0

In GitLab 16.0 (starting in 15.1 as feature), registering a runner using registration tokens got deprecated. For newer versions, the registration process located in the Admin area can be used. It’s available at “Admin Area > CI/CD > Runners”. Clicking on “New instance runner” shows the following form:

Submitting the form will create a temporary token in the background and show the registration command to be used for a new runner. The command shown is similar to the registration command above but without the need for manually creating the tokens.

 

Docker: how to build and push a Git repository to Docker Hub

To build and push a Git repository to Docker Hub, you can follow these steps:

Step 1 – Create a Dockerfile in the root directory of your Git repository. This file contains instructions for building your Docker image.

Step 2 – Build your Docker image using the docker build command. You will need to provide a name and tag for your image. For example:

docker build -t yourusername/yourimage:tag .

This command will build an image with the tag yourusername/yourimage:tag. The path provided as . is the current path.

Step 3 – Log in to Docker Hub using the docker login command. You will need to provide your Docker Hub username and password.

docker login

Step 4 – Push your Docker image to Docker Hub using the docker push command. For example:

docker push yourusername/yourimage:tag

This command will push your image to Docker Hub with the tag yourusername/yourimage:tag.

Once you have completed these steps, your Docker image will be available on Docker Hub for others to use and download.

 

How to Run and Update Docker Images and Containers

The following text shows the default workflow to run and update a Docker image based on the example of GitLab. But this should also work for other containers.

Docker Installation

First make sure that your local machine has Docker installed. I use Debian and for this, a detailed description is available in Dockers documentation. There are also instructions for CentOS, Ubuntu, Fedora and others.

You can check the installed Docker version:

docker version
Client: Docker Engine - Community
 Version:           20.10.11
 API version:       1.41
 ... 

Run a Docker Container

Let’s start with an environment variable that defines the path where all the Gitlab data is stored. This is outside of the Docker container which allows an easy update without loosing data. To define the variable $GITLAB_HOME, you can use:

export GITLAB_HOME="/srv/gitlab"

To run a Docker container, simply use the run command with your parameters. For GitLab, this might look like:

docker run --detach
  --hostname <host.example.com>
  --publish 443:443 --publish 80:80 --publish 2222:22
  --name gitlab
  --restart always  
  --volume $GITLAB_HOME/config:/etc/gitlab 
  --volume $GITLAB_HOME/logs:/var/log/gitlab
  --volume $GITLAB_HOME/data:/var/opt/gitlab
  gitlab/gitlab-ce:latest

You should use your own settings and replace <host.example.com> with your setup.

For the sake of completeness, you might also need a runner for GitLab. Let’s run this container as well:

docker run --detach 
  --name gitlab_runner
  --restart always 
  --network host 
  -v /run/docker.sock:/var/run/docker.sock 
  gitlab/gitlab-runner:latest

And this one needs to be registered in your GitLab instance:

docker exec -it gitlab_runner gitlab-runner register 
  --url https://<host.example.com>/
  --registration-token <registration_token>

More details about the configuration of the GitLab runner is descriped in this post.

Update a Docker Container to the Latest Version

Let’s check, which container are running by using:

docker ps

The output shows all your containers and their ids:

CONTAINER ID   IMAGE                         COMMAND                  CREATED       STATUS                 NAMES
02fe426970f1   gitlab/gitlab-ce:latest       "/assets/wrapper"        4 hours ago   Up 4 hours (healthy)   gitlab
5742e07f809b   gitlab/gitlab-runner:latest   "/usr/bin/dumb-init …"   4 days ago    Up 5 hours             gitlab_runner

We have to stop the container and remove the image. When you used a run command (for GitLab) as shown above, this should not affect the data as this is stored outside of the container.

docker stop 02fe426970f1
docker rm 02fe426970f1

To remove the associated image, you can use the docker rmi. For this, we need to find the id of this image by listing all images:

docker images
REPOSITORY         TAG      IMAGE ID       CREATED        SIZE
gitlab/gitlab-ce   latest   46cd6954564a   17 hours ago   2.36GB
mysql              5.7      c20987f18b13   2 weeks ago    448MB
cirrusci/flutter   2.8.1    98a87781f179   2 weeks ago    3.49GB

Now we can remove the image (in my case the gitlab-ce image):

docker rmi 46cd6954564a

The update of the image can be done by using pull. This will download the images (in this case the latest version) and stores it on your system:

docker pull gitlab/gitlab-ce:latest

After this, you can use the same run command as before to run a container with the updated image:

docker run --detach
  --hostname <host.example.com>
  --publish 443:443 --publish 80:80 --publish 2222:22
  --name gitlab
  --restart always  
  --volume $GITLAB_HOME/config:/etc/gitlab 
  --volume $GITLAB_HOME/logs:/var/log/gitlab
  --volume $GITLAB_HOME/data:/var/opt/gitlab
  gitlab/gitlab-ce:latest

A final check, shows the container is up and running:

docker ps
CONTAINER ID   IMAGE                         COMMAND                  CREATED        STATUS                 NAMES
f3dab1163c07   gitlab/gitlab-ce:latest       "/assets/wrapper"        2 minutes ago  Up 4 hours (healthy)   gitlab
5742e07f809b   gitlab/gitlab-runner:latest   "/usr/bin/dumb-init …"   4 days ago     Up 5 hours             gitlab_runner

Instead of automatically using the latest version of each image (e.g. gitlab/gitlab-ce:latest) you can also specify a version number like gitlab/gitlab-ce:14.5.2. This is helpful when you need to update a system in steps. Or if you need to follow a specific upgrade path, which is the case for GitLab.

Photo by Ian Taylor on Unsplash.

Synology: How do I update an existing Docker container with a new image?

As always: before you do such an update, make sure to create a backup of all your files. If something goes wrong, this may lead to data loss!

Manual update

To update an existing Docker container manually, the following steps are necessary;

  1. Go to Registry and download new image (mostly the “latest” version)
  2. Go to Container, select the container you need to update and stop it
  3. From Actions menu select “Clear”
    Edit: Under DSM7, the “Clear” command has been renamed “Reset”.
  4. Start the container again

This will clear the complete container and start with the newly downloaded Docker image. Since the data folders are mounted into the container, this will not erase the apllications data. Configurations are also not affected by this.

Side note: when updating a major version of gitlab/gitlab-ce, make sure to follow the update paths! This requires updates in smaller steps (minor versions).

Automatically update Docker images

Updating a Docker image manually might be fine for a small number of images. But there is a more elegant way by using another Docker container called Watchtower. This one can update Docker containers automatically. The image is called containrrr/watchtower. A simple setup can be performed with the following steps:

  1. Load image containrrr/watchtower in the Docker registry
  2. Sign in with root privilege via SSH
  3. Run the following code in the shell:
docker run -d --network host --name watchtower-once -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower:latest --cleanup --include-stopped --run-once

This will start the Watchtower image and update all container once. The container created for this runs once and can then be found switched off in the list of containers. Now you can start it manually again and again as needed or let it run at certain times via Synology Task Scheduler. The command for the task scheduler is then as follows:

docker start watchtower-once -a

Let Watchtower run permanently

Alternatively you can use the scheduler in Watchtower itself. If you want to start it every Monday at 4 a.m., then enter the following command on the shell:

docker run -e "TZ=Europe/Berlin" -d --restart unless-stopped --network host --name watchtower -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --cleanup --include-stopped --schedule "0 0 4 ? * MON"

It is important to set the time zone to your because otherwise you will have an offset to UTC. In addition, the container is not terminated but always restarted. Even if it crashed or the NAS was restarted. The last parameter uses the cron syntax for scheduling the task.

Photo by sergio souza on Unsplash

Setup GitLab Runner for Docker containers on Synology NAS

The setup described in this post has been tested on the following system:

  • DS216+II with 8GB RAM
  • DSM 6.2.3-25426 Update 2

In addition, the following software packages have already been installed on the system using the Synology package manager:

  • Docker 18.09.0-0513

GitLab is installed via the Docker Registry:

  • gitlab/gitlab-ce:latest (in my case GitLab 13.5.1-ce.0)

Install GitLab

You can skip this part, if GitLab is already running on your Synology and continue with the step Install GitLab Runner.

To install GitLab, open the Docker Registry and search for “gitlab”. Double click the entry gitlab/gitlab-ce:latest and select the latest version:

After the image is loaded, it will be listed under image. Launch this image and set the folders to be mounted as shown in the following image. This will simplify the access to the docker files within your Synology.

Mounted folder / volumes
Port settings

The port settings depend on your system. Normally, HTTP is accessible at port 80 or HTTPS on port 443. If your system already uses other apps that are running on those port, you can adjust them in Port Settings.

After completing the setup, it might take some time until the GitLab web surface is available. When accessing GitLab for the first time, you can specify a password for the root environment. The default username for the admin area is root. Now you can create user accounts, projects and perform any adjustments that fit your needs.

Install GitLab Runner

As the Synology DSM uses Docker to run GitLab, we can use Docker as well to install GitLab Runner. For this, connect to the Synology using SSH:

ssh <admin-user>@<synology> -p <port>

Now we can install the Gitlab Runner Docker container that can run other Docker containers to perform the runner tasks:

docker run -d \
--name gitlab_runner_docker \
--restart always \<br>--network host \
-v /run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest

This will install a Docker container with the name gitlab_runner_docker which uses the same network as Docker (‘–network host‘).

As this container is the basis for our Docker-in-Docker setup, we connect the Docker socket of our main container to the new one by using -v /run/docker.sock:/var/run/docker.sock.

You might also use the Docker GUI of DSM to create the container. But the step above can only be done with the help of the console command!

To test the container, you can use the following command to connect to the console:

docker exec -it gitlab_runner_docker /bin/bash

Register the runner

Now it’s time to register the runner:

docker exec -it gitlab_runner_docker gitlab-runner register

This will ask for some input:

Runtime platform                                    arch=amd64 os=linux pid=16 revision=e95f89a0 version=13.4.1
Running in system-mode.                            
                                                   
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
http://10.0.6.102:30000/
Please enter the gitlab-ci token for this runner:
ab1234abcd1234abcd12
Please enter the gitlab-ci description for this runner:
[synology]: docker_alpine
Please enter the gitlab-ci tags for this runner (comma separated):

Registering runner... succeeded                     runner=sA4DKorC
Please enter the executor: docker-ssh, parallels, docker+machine, kubernetes, docker, shell, ssh, virtualbox, docker-ssh+machine, custom:
docker
Please enter the default Docker image (e.g. ruby:2.6):
alpine:latest
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

In this case, the executor docker and the base image alpine:latest is used for this container.

If you use a self-signed certificate for GItLab, then you have to specify the certificate with the option --tls-ca-file during registration:

docker exec -it gitlab_runner_docker gitlab-runner register --tls-ca-file /path/to/fullchain.pem

Setup Docker-in-Docker

Let’s connect to the bash terminal of this container to change some settings:

docker exec -it gitlab_runner_docker /bin/bash

The created Docker container is an alpine base system without any packages installed. To perform the changes, we need a simple terminal text editor like nano or vim. The following commands will install nano for this task:

apt-get update
apt-get install nano

Now you can use nano to edit the GitLab Runner config file:

nano /etc/gitlab-runner/config.toml

And add the following lines as highlighted in the config file below:

clone_url = "http://10.0.6.102:30000/"
privileged = true
pull_policy = "if-not-present"
concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "GitLab Runner Docker"
  url = "http://10.0.6.102:30000/"
  clone_url = "http://10.0.6.102:30000/"
  token = "Examp1eT0ken"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "node:latest"
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
    privileged = true
    pull_policy = "if-not-present"
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

That’s it. Now you can use this GitLab Runner for your repositories and run jobs by using other Docker containers.

Example usage in .gitlab-ci.yml

The following example uses a php container to run tests for a simple PHP application and deploy the code to a server. Some of the settings depend on the setup of your PHP application, so the example will not work out of the box. But it can give you a good idea of what is possible.

# https://hub.docker.com/_/php
image: php:7.4-fpm

services:
  - mysql:5.7

variables:
  # Configure mysql environment variables (https://hub.docker.com/_/mysql/)
  MYSQL_USER: $MYSQL_USER
  MYSQL_PASSWORD: $MYSQL_PASSWORD

before_script:
  # Initialize database, etc

stages:
  - test
  - deploy

test:
  stage: test
  script:
    - composer install
    - composer phpunit

deploy_production:
  stage: deploy
  environment:
    name: production
    url: https://www.example.com
  script:
    # run on server 'git checkout master && git pull origin master && exit'
  only:
    - master

Photo by sergio souza on Unsplash