Category: Software Development

GitLab CI + Flutter: pub: command not found

In one of my projects, I used a GitLab environment to perform Flutter tests. For this, I setup my .gitlab-ci.yaml to use a Flutter docker image of cirrusci/flutter for code quality check or tests. The file looks like this:

code_quality:
  stage: test
  image: "cirrusci/flutter:stable"
  tags:
    - docker
  before_script:
    - pub global activate dart_code_metrics
    - export PATH="$PATH":"$HOME/.pub-cache/bin"
  script:
    - metrics lib -r codeclimate  > gl-code-quality-report.json
      
# [...]

test:
  stage: test
  image: "cirrusci/flutter:stable"
  tags:
    - docker
  before_script:
    - pub global activate junitreport
    - export PATH="$PATH":"$HOME/.pub-cache/bin"
  script:
    - flutter test --machine --coverage | tojunit -o report.xml
    - lcov --summary coverage/lcov.info
    - genhtml coverage/lcov.info --output=coverage

# [...]

Up to version 2.10.* of the Flutter docker image, this worked fine. But starting with version 3.0.0, there seems to be some changes in the binaries or their paths. The scripts failed with an error:

/usr/bin/bash: line 123: pub: command not found

To fix this error, the pub commands need to be adjusted and set to flutter pub:

# [...]

  before_script:
    - flutter pub global activate dart_code_metrics

# [...]

  before_script:
    - flutter pub global activate junitreport

# [...]

This fixed the issue and all tests finished successfully.

 

Testing Googles Indexing of www.frontiersin.org

https://unsplash.com/@mjessier

On 15th of December 2021 a paper was published on www.frontiersin.org: “The Impact of the COVID-19 Pandemic on Avoidance of Health Care, Symptom Severity, and Mental Well-Being in Patients With Coronary Artery Disease”.

Frontiers is a leading Open Access Publisher and Open Science Platform with multiple peer-reviewed and editorial board led journals.

Neither Google, Bing nor DuckDuckGo did index it until today (24th of December). The paper was posted to ResearchGate on 17th of December. The ResearchGate page was indexed within days by Google, Bing and DuckDuckGo.

Now let’s see how long it will take the search engines to index this post (first published on 24th of December). To test this I will search for “Neither Google, Bing nor DuckDuckGo” and the paper title.

Update 26th of December:

This post was indexed by DuckDuckGo. Google and Bing do not yet show it in their results when searching for “Neither Google, Bing nor DuckDuckGo”. None of the search engines finds the paper.

Update 27th of December:

The Frontiers paper and this post was indexed by Bing. DuckDuckGo now also links to Frontiers when searching for the title. No results from Google yet.

Update 29th of December:

Google indexed this post. However, it still doesn’t find the paper on frontiersin.org

Update 4th ofJanuary:

Google finally indexed the frontiersin.org page.

Conclusion: FrontiersIn.org admins need to look into their SEO.

 

How to Run and Update Docker Images and Containers

The following text shows the default workflow to run and update a Docker image based on the example of GitLab. But this should also work for other containers.

Docker Installation

First make sure that your local machine has Docker installed. I use Debian and for this, a detailed description is available in Dockers documentation. There are also instructions for CentOS, Ubuntu, Fedora and others.

You can check the installed Docker version:

docker version
Client: Docker Engine - Community
 Version:           20.10.11
 API version:       1.41
 ... 

Run a Docker Container

Let’s start with an environment variable that defines the path where all the Gitlab data is stored. This is outside of the Docker container which allows an easy update without loosing data. To define the variable $GITLAB_HOME, you can use:

export GITLAB_HOME="/srv/gitlab"

To run a Docker container, simply use the run command with your parameters. For GitLab, this might look like:

docker run --detach
  --hostname <host.example.com>
  --publish 443:443 --publish 80:80 --publish 2222:22
  --name gitlab
  --restart always  
  --volume $GITLAB_HOME/config:/etc/gitlab 
  --volume $GITLAB_HOME/logs:/var/log/gitlab
  --volume $GITLAB_HOME/data:/var/opt/gitlab
  gitlab/gitlab-ce:latest

You should use your own settings and replace <host.example.com> with your setup.

For the sake of completeness, you might also need a runner for GitLab. Let’s run this container as well:

docker run --detach 
  --name gitlab_runner
  --restart always 
  --network host 
  -v /run/docker.sock:/var/run/docker.sock 
  gitlab/gitlab-runner:latest

And this one needs to be registered in your GitLab instance:

docker exec -it gitlab_runner gitlab-runner register 
  --url https://<host.example.com>/
  --registration-token <registration_token>

More details about the configuration of the GitLab runner is descriped in this post.

Update a Docker Container to the Latest Version

Let’s check, which container are running by using:

docker ps

The output shows all your containers and their ids:

CONTAINER ID   IMAGE                         COMMAND                  CREATED       STATUS                 NAMES
02fe426970f1   gitlab/gitlab-ce:latest       "/assets/wrapper"        4 hours ago   Up 4 hours (healthy)   gitlab
5742e07f809b   gitlab/gitlab-runner:latest   "/usr/bin/dumb-init …"   4 days ago    Up 5 hours             gitlab_runner

We have to stop the container and remove the image. When you used a run command (for GitLab) as shown above, this should not affect the data as this is stored outside of the container.

docker stop 02fe426970f1
docker rm 02fe426970f1

To remove the associated image, you can use the docker rmi. For this, we need to find the id of this image by listing all images:

docker images
REPOSITORY         TAG      IMAGE ID       CREATED        SIZE
gitlab/gitlab-ce   latest   46cd6954564a   17 hours ago   2.36GB
mysql              5.7      c20987f18b13   2 weeks ago    448MB
cirrusci/flutter   2.8.1    98a87781f179   2 weeks ago    3.49GB

Now we can remove the image (in my case the gitlab-ce image):

docker rmi 46cd6954564a

The update of the image can be done by using pull. This will download the images (in this case the latest version) and stores it on your system:

docker pull gitlab/gitlab-ce:latest

After this, you can use the same run command as before to run a container with the updated image:

docker run --detach
  --hostname <host.example.com>
  --publish 443:443 --publish 80:80 --publish 2222:22
  --name gitlab
  --restart always  
  --volume $GITLAB_HOME/config:/etc/gitlab 
  --volume $GITLAB_HOME/logs:/var/log/gitlab
  --volume $GITLAB_HOME/data:/var/opt/gitlab
  gitlab/gitlab-ce:latest

A final check, shows the container is up and running:

docker ps
CONTAINER ID   IMAGE                         COMMAND                  CREATED        STATUS                 NAMES
f3dab1163c07   gitlab/gitlab-ce:latest       "/assets/wrapper"        2 minutes ago  Up 4 hours (healthy)   gitlab
5742e07f809b   gitlab/gitlab-runner:latest   "/usr/bin/dumb-init …"   4 days ago     Up 5 hours             gitlab_runner

Instead of automatically using the latest version of each image (e.g. gitlab/gitlab-ce:latest) you can also specify a version number like gitlab/gitlab-ce:14.5.2. This is helpful when you need to update a system in steps. Or if you need to follow a specific upgrade path, which is the case for GitLab.

Photo by Ian Taylor on Unsplash.

How to simply analyze access logs of web servers

There is a tool for that: GoAccess. It’s an easy to use command line tool to analyze any access log and create beautiful insights.

Installation

On macOS you can use HomeBrew to install GoAccess:

brew install goaccess

For other operation systems, please check out the detailed documentation on their website.

Usage

The most elegant way is the terminal output. For this, simply type:

 goaccess access.log --log-format=COMBINED

Which creates such an output (you can navigate down to see much more details):

You can also create an html file that you can view in your browser:

goaccess access.last.log -o report.last.html --log-format=COMBINED

And voilà:

If your terminal is not using the same language as your log files do, then you might need to set LC_TIME for a correct parsing of the log files:

LC_TIME="en_US.UTF-8" goaccess access.last.log -o report.last.html --log-format=COMBINED

To learn more about all options, you can check out the documentation of GoAccess.

Photo by Aryan Dhiman on Unsplash.

GitLab – ERROR: Registering runner… failed, certificate signed by unknown authority

If your self-hosted GitLab server is using a self-signed certificate for https, it might be possible that you get an error during the registration of a GitLab Runner:

ERROR: Registering runner... failed        runner=HbU25D-y status=couldn't execute POST against https://gitlab.example.com/api/v4/runners: Post https://gitlab.example.com/api/v4/runners: x509: certificate signed by unknown authority
PANIC: Failed to register the runner. You may be having network problems.

To solve the problem, you have to provide the full chain certificate *.pem used by your GitLab Server:

gitlab-runner register --tls-ca-file /path/to/fullchain.pem

In my case, the valid certificate could be found on the GitLab server in /etc/gitlab/trusted-certs/fullchain.pem. This one was copied to the GitLab Runner server and used in the command above.

As I did not set up the server on my own, I do not know if this is the default path and filename of a certificate signed by Let’s Encrypt. But in my case, this one worked to register the runner.

Adjust text color to be readable on light and dark backgrounds of user interfaces

Most modern user interfaces are supporting different color schemes for day and night: the so called light and dark modes. Selecting a text color for each of those modes is not a big deal and it’s the way to go when designing the user interface.

In some cases, the text color is driven by the displayed contents. In the example below, the tint color is matched to the color of the drink. The global tint color of this app is totally different, but this color adjustment gives a very nice effect. But as you might already see, there is a small problem when it comes to very light or very dark colors: each color either has a good readability on light or dark backgrounds. Some colors might fit to both, but that’s not always the case. In the example below, the light yellow is still visible, but when it comes to small icons or small text, the details are lost.

To overcome this issue, a simple solution is to select two colors for each recipe so that each mode has a different one. That’s fine, but it might totally change the effect of this colored pages.

Can we calculate a suitable color?

Some time ago, there was an article about Black or white text on a colour background? In this one, I described different algorithms to calculate the best text color (black or white) for a colored background. But now, we need the opposite: a colored text that has a good readability on white (light) or black (dark) backgrounds.

When we look at HSL and HSV/HSB color models, we already have a value for ‘lightness’ or ‘brightness’. The idea is to find a color that matches a given hue and saturation and that has a brightness which is readable on light and dark background. For this, we can use different algorithms. Very good results could be achieved with a ‘Weighted W3C Formula‘. This formula take into consideration that the human eye perceives some of the primary colors darker than others.

f'(x) = r ? 0.299 + g ? 0.587 + b ? 0.11

Each color that is located at the border between the black and white overlay is suitable for light and dark backgrounds.

Step 1: convert the given color to HSV/HSB

Step 2: keep hue and saturation constant and adjust the brightness (make the color lighter or darker)

Step 3: convert the HSV/HSB value back to the required color format

Implementation in PHP

A simple calculation for a given RGB color is shown below. The classes used in this snippet are available on GitHub. The code checks the initial brightness of the color and lightens or darkens the values until the ‘border’ calculated by the ‘Weighted W3C Formula’ is reached. This is the case for the value 127, the total range of the brightness is 0 to 255.

$hsv = Convert::rgb2hsv($rgb);

$step = 0.01;
$brightness = Calculate::weightedW3C($rgb);
if ($brightness < 127) {
    while ($brightness < 127 && $hsv[2] >= 0 && $hsv[2] <= 1) {
        $hsv[2] += $step;
        $brightness = Calculate::weightedW3C(Convert::hsv2rgb($hsv));
    }
} else {
    while ($brightness > 127 && $hsv[2] >= 0 && $hsv[2] <= 1) {
        $hsv[2] -= $step;
        $brightness = Calculate::weightedW3C(Convert::hsv2rgb($hsv));
    }
}

return Convert::hsv2rgb($hsv);

Some examples

But how does this result look for different colors? Let’s start with some dark colors. Those are fine for a light background, but they become unreadable on a dark one. The top colors show the input color (before) and the color below shows the output of the calculation above (after).

Color #632300 adjusted to be readable on light and dark background
Color #454545 adjusted to be readable on light and dark background

And now let’s look at some light colors which are fine for dark backgrounds, but they are totally unreadable on light backgrounds.

Color #73FEFF adjusted to be readable on light and dark background
Color #F0C&96 adjusted to be readable on light and dark background

The last color is similar to the example at the beginning and as you can see, the optimized color has a much better readability. This could be achieved for both light and dark colors. The code example shown above is written in PHP. An adoption should be easily possible for any other coding or scripting language

The algorithm mentioned in this post is also available on GitHub https://github.com/mixable/color-utils. This package is usable with composer:

composer require mixable/color-utils

The optimized color can be calculated with:

use Mixable\Color\Calculate;
// ...

$hex = '#ffcc00';
$optimizedColor = Calculate::readableColorForLightAndDarkBackground($hex);

Xcode: how to disable an extension during app build

Sometimes the development version of an app includes multiple code e.g. an extension that should not be released yet. In this case, it’s possible to exclude the extension when building an app. This keeps all your code, but does not include the extension during the build phase.

To achieve this, simply open the Build Phases of your main app and remove the extension(s) from Dependencies and Embed App Extensions. You can add the extension later when required.

Below is a screenshot of the setting in Xcode.

Photo by Clément Hélardot on Unsplash

Quick Look plugins for software development

Quick Look already supports multiple file types. But there ist more – especially for software development. Here are some plugins that make Quick Look even better.

Note: some of the plugins might not work instantly after brew install ... when you are on macOS Catalina or later. In this case, it is possible to download the plugin manually and copy the .qlgenerator file to ~/Library/QuickLook. This requires to run qlmanage -r (or a system restart) to enable the plugin.

QLMarkdown

QLMarkdown provides QuickLook support for markdown files (*.md). This plugin renders the markdown content and shows the result. To install QLMarkdown, use:

brew cask install qlmarkdown

For manual installation, the plugin is available at https://github.com/toland/qlmarkdown.

QLStephen

This Quick Look plugin provides a file preview for files without extension, e.g. README, INSTALL, Capfile, CHANGELOG, etc. It can be installed using Homebrew:

brew cask install qlstephen

For manual installation, the plugin is available at https://github.com/whomwah/qlstephen.

QLColorCode

This is a Quick Look plug-in that renders source code with syntax highlighting. To install the plugin, use Homebrew:

brew cask install qlcolorcode

For manual installation, the plugin is available at https://github.com/anthonygelibert/QLColorCode. If you want to configure QLColorCode, there are several defaults commands that are described on the download page.

Quick Look Json

This is a Quick Look plug-in that renders json files. To install the plugin, use Homebrew:

brew cask install quicklook-json

For manual installation, the plugin is available at http://www.sagtau.com/quicklookjson.html.

WebP QuickLook

This is an open-source QuickLook plugin to generate thumbnails and previews for WebP images. To install the plugin, use Homebrew:

brew install webpquicklook

For manual installation, the plugin is available at https://github.com/dchest/webp-quicklook.

Something is missing? Please let me know in the comments, if there are any other plugins that might be helpful for software development.

Photo by Shane Aldendorff on Unsplash