Quality assurance (QA) in coding is often associated with writing applications. Coding (or automating common tasks using a structured language) however is by far not longer restricted to writing software applications. With the rise of the tooling for operations and the establishment of the DevOps culture, writing code is being done for e.g. launching your infrastructure on Amazon Web Services (Terraform), building and configuring your run time environment for your application (Docker) or managing your cluster (Kubernetes).

For code used in applications, QA is accepted as being a best practise. We lint our code to coding standards, write unittests, generate documentation and implement integration tests etc. For example Dockerfiles are often being written using “does it build ” Development or DIB. And strangely enough, Docker containers and the originating Dockerfiles are being used in state of the art (automated) Continuous Delivery pipelines without any proper QA. My advice is stop doing DIB Development and start ensuring the quality (and consistency) of your Dockerfiles.

Validating (linting) Dockerfiles

The simplest way of managing the quality of your Dockerfile is linting the Dockerfile. This will take you approximately 10 seconds using hadolint:

docker run --rm -i lukasmartinelli/hadolint < Dockerfile

This will run the docker linting tool with about 50 rules applied. Example output is displayed below:

/dev/stdin DL4001 Either use Wget or Curl but not both
/dev/stdin DL4000 Specify a maintainer of the Dockerfile
/dev/stdin:20 DL3008 Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
/dev/stdin:20 DL3009 Delete the apt-get lists after installing something
/dev/stdin:20 DL3015 Avoid additional packages by specifying `--no-install-recommends`
/dev/stdin:61 DL3020 Use COPY instead of ADD for files and folders

You could choose to ignore specific rules by adding --ignore statements. The full list of rules can be found in the readme of the hadolint repository.

docker run --rm -i lukasmartinelli/hadolint hadolint --ignore DL3008 - < Dockerfile

Testing your Dockerfiles

Ensuring the quality of your code can be taken a step further by using for example rspec and serverspec. Rspec is a BDD tool for Ruby developers which has a strong focus on facilitating test driven development. Serverspec is a suite which runs on top of rspec specifically written for testings server configurations. With using the ruby docker-api library, both are excellent for writing tests cases for your Dockerfile which ensures that your Dockerfiles matches the contract with your application. Using these tools, you can do this fully automated in a CD pipeline. Running these tests can take significant more time as linting, because the actual docker container will be built and started. The total built time depends on many factors, like has the image built before (and thus is likely to be cached), network speed, cpu power (compilation speed) and so forth.

Project structure

$ tree -L 1
.
├── Dockerfile
└── tests

$ tree tests
tests
├── Rakefile
└── spec
    ├── container_spec.rb
    ├── image_spec.rb
    └── spec_helper.rb

The Rakefile is used as a testrunner for your rspec tests.

# tests/Rakefile

require 'rake'
require 'rspec/core/rake_task'

task :spec    => 'spec:all'
task :default => :spec

namespace :spec do

  targets = []
  targets << '.'

  task :all     => targets
  task :default => :all

  targets.each do |target|
    desc "Run rspec tests"
    RSpec::Core::RakeTask.new('.') do |t|
      ENV['TARGET_HOST'] = 'localhost'
      t.pattern = "spec/*_spec.rb"
    end
  end

end

It will execute all files matching *_spec.rb in the tests/spec directory. The basic configuration options for rspec can be set in a helper file, spec_helper.rb:

 # spec/spec_helper.rb
 
 require 'docker'
 require 'serverspec'
 
 RSpec.configure do |config|
   config.color = true
   config.tty = true
   config.formatter = :documentation # :progress, :html, :textmate
 end

This file will be included by using the required keyword in top of the files containing your actual tests. For those tests, it is advisable to split the tests in those which needs only the docker image to be built and in those which needs a running container. Like many testsuites, rspec allows you to wrap your tests in a setup and teardown structure. Splitting the tests like this, will keep this setup and teardown structure simple.

For a simple docker container running Ubuntu with Apache installed, the following Dockerfile is created:

FROM ubuntu:xenial
MAINTAINER Merlijn Tishauser <[email protected]>

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update -qq -y && \
    apt-get install -yq --no-install-recommends \
	bash \
	apache2 && \
	apt-get clean &&\
	rm -rf /var/lib/apt/lists/*

VOLUME ["/etc/apache2"]

EXPOSE 80 443

CMD ["apachectl", "-D", "FOREGROUND"]

On the dockerfile itself we could test if ports 80 and 443 are exposed and that /etc/apache2 is declared as volume to allow easy override of the apache configuration . To test the Dockerfile an image is built using the ruby docker api.

# spec/image_spec.rb
require 'spec_helper'

describe "Image" do

  before(:all) do
    @image = Docker::Image.build_from_dir('../')

    set :os, family: :debian
    set :backend, :docker
    set :docker_image, @image.id
  end
  
    describe 'Dockerfile#config' do
      it 'should expose the http port' do
        expect(@image.json['ContainerConfig']['ExposedPorts']).to include("#{HTTP_PORT}/tcp")
      end
      
      it 'should expose the http port' do
        expect(@image.json['ContainerConfig']['ExposedPorts']).to include("#{HTTPS_PORT}/tcp")
      end
      
      it 'should have /etc/apache2 as volume' do
        expect(@image.json['ContainerConfig']['Volumes']).to include('/etc/apache2')
      end
    end

  after(:all) do
    # optional, remove the built image before every test
    # @image.remove(:force => true)
  end

end

On the actual running container we could check if we run the expected version of Ubuntu, that apache is properly installed and we are able to connect to port 80. Port 443 should not be enabled by default, so this is checked as well.

# spec/container_spec.rb
require 'spec_helper'
require 'socket'
require 'timeout'


HTTP_PORT = 80
HTTPS_PORT = 443

describe "Container" do

  def is_port_open(ip, port)
    begin
      Timeout::timeout(1) do
        begin
          s = TCPSocket.new(ip, port)
          s.close
          return true
        rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH
          return false
        end
      end
    rescue Timeout::Error
    end

    return false
  end

  before(:all) do
    @image = Docker::Image.build_from_dir('../')

    set :os, family: :debian
    set :backend, :docker
    set :docker_image, @image.id
  end

  describe 'when running' do
    before(:all) do
      @container = Docker::Container.create(
          'Image' => @image.id,
          'HostConfig' => {
            'NetworkMode' => "container:" + ENV['CONTNAME'],
            'Hostname' => 'localhost'
          }
      )
      @container.start
    end

    it "should be the 16.04 version of Ubuntu" do
      os_version = command('lsb_release -a').stdout
      expect(os_version).to include('16.04')
      expect(os_version).to include('Ubuntu')
    end

    describe package('apache2') do
      it { should be_installed }
    end

    it "allow connections to port #{HTTP_PORT}" do
      expect(is_port_open(Socket.ip_address_list[0].ip_address, "#{HTTP_PORT}")).to be true
    end

    it "should not allow connections to port #{HTTPS_PORT} by default" do
      expect(is_port_open('127.0.0.1', "#{HTTPS_PORT}")).to be false
    end

    after(:all) do
      @container.kill
      @container.delete(:force => true)
    end

  end

end

Running the tests is done by calling rake:

$ docker run -it -e "CONTNAME=testdocker" --name testdocker -v "/var/run/docker.sock:/var/run/docker.sock" -v "$(PWD):/projectfiles" fourstacks/serverspec

Exceuting this command will give the following output:

Container
  when running
    should be the 16.04 version of Ubuntu
    allow connections to port 80
    should not allow connections to port 443 by default
    Package "apache2"
      should be installed

Image
  Dockerfile#config
    should expose the http port
    should expose the https port
    should have /etc/apache2 as volume

Finished in 2.98 seconds (files took 1.33 seconds to load)
7 examples, 0 failures

Reminder, all files described above can be found in this gitrepository. Have fun experimenting and implementing your own tests.

References used: