Lab 9 - Containers and Configuration Management

Overview

This lab is designed to give you some hands-on experience with Docker and Puppet! By the end of this assignment, you should be able to:

To submit the lab, answer the questions in this Google form

Installing Docker

One of the annoying things about Docker is it’s usually not in apt and, if it is, it’s always under a different name. Here’s a link to the .deb for Docker Community Edition you can download and install on your VM: https://download.docker.com/linux/debian/dists/stretch/pool/stable/amd64/docker-ce_18.03.0~ce-0~debian_amd64.deb.

I trust by this point in the course installing this on your Linux machine should be a breeze :).

Hint: use wget and dpkg.

After installing, I recommend running sudo usermod -aG docker $USER. This adds your user to the docker group so you can run docker as a non-root user. This means you won’t have to type sudo docker all the time. This is optional but for the rest of the lab I’m going to assume that you did this. You might see output like:

sent invalidate(passwd) request, exiting
sent invalidate(group) request, exiting

This is normal when adding users to groups.

Creating your first Docker container

To verify that you installed things correctly, try running:

docker run hello-world

You should see some friendly output like so:

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
ca4f61b1923c: Pull complete
Digest: sha256:97ce6fa4b6cdc0790cda65fe7290b74cfebd9fa0c9b8c38e979330d547d22ce1
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.
[...]

This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps:

  1. The Docker client contacted the Docker daemon.
  2. The Docker daemon pulled the “hello-world” image from the Docker Hub.
  3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading.
  4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal.

Some quick definitions from Docker’s website:

An image is a lightweight, stand-alone, executable package that includes everything needed to run a piece of software, including the code, a runtime, libraries, environment variables, and config files. Images are useful primarily for their speed, but images can also be used as a base to be built on top of in future images, as you’ll see later with Dockerfiles. In the last example hello-world was the image used to test our docker installation.

A container is a runtime instance of an image—what the image becomes in memory when actually executed. It runs completely isolated from the host environment by default, only accessing host files and ports if configured to do so. A container gets created upon executing docker run on an image.

Be sure to read through the output from running the hello-world image to get an understanding of what the Docker daemon was doing.

Running an interactive container

We’re now going to walk you through running a container interactively. This is useful if you ever need to play around and install stuff on a bare system without messing up your current system. Try running the following command:

docker run -it ubuntu:latest

The -i flag tells docker to keep STDIN open to your container, and the -t flag allocates a pseudo-TTY for you. Basically you need both for you to have a shell into your newly started container. Try installing some packages from apt or just play around. It should look like a bare Linux system.

You can exit the container with CTRL+D.

Questions
  1. What user are you logged in as by default?
  2. If you start and then exit an interactive container, and then use the docker run -it ubuntu:latest command again; is it the same container? How can you tell?

Dockerfiles

A more powerful way to interface with Docker is by using a Dockerfile. A Dockerfile allows you to define an image by specifying all of the commands you would type manually to create an image. Docker can then build images from a specified Dockerfile. These Dockerfiles can be put into version control and the images distributed as a binary to keep track of both how the image is constructed and also to keep pre-built images around.

Dockerfiles are very powerful and have many different commands and features. We’ll go over a basic example, but you should check out the reference page if you are trying to do anything more complex.

Here is an example Dockerfile that will build an image based on Fedora Linux that has python3.6 installed by default. Note that docker will automatically fetch fedora:latest when building the Dockerfile. The code for this exercise can be found in b9 directory of the decal labs repo ~/decal-labs/b9/missile (don’t forget to git pull if it’s not there!).

FROM fedora:latest

RUN yum update -y && yum install python3-pip && \
    pip3 install --upgrade pip && \
    pip3 install pyfiglet termcolor

ADD missile.py /
CMD ["/usr/bin/python3.6", "missile.py"]

Note: there are some “best practices” for writing Dockerfiles that the above example doesn’t use, because it’s a basic example. If you’re interested in this stuff, check out this article.

What is this doing? We first specify a base image fedora:latest (Fedora 27). We then specify that we should run (RUN) the command yum update (yum is Fedora’s package manager), and then yum install pip3 installs pip for python3. Then we install the pyfiglet and termcolor packages via pip3 since our python script depends on them. We put the missile.py script in the container, and lastly we run the script with python3.6. Take a moment to appreciate how cool this is. We have a completely different Linux distribution running on our system that ships with a version of python that is not in apt on your student VMs!

Build the Dockerfile with the following:

docker build -t missile:latest .

This tells Docker to look in the current directory for a Dockerfile to build, and builds it. The -t flag tells Docker to tag this build with the name missile:latest. Note that building the missile image will take a couple of minutes to complete.

You can see all of the images you’ve built on your machine with the docker images command.

Questions
  1. Run the image you just built with no flags. What do you observe?
  2. Write and build a Dockerfile based on ubuntu:xenial that installs the packages fortune and fortunes-min and runs the fortune executable (located in /usr/games/fortune after you install it). Note that you won’t need to use the -it flags when you run the container as fortune doesn’t need STDIN. Submit your Dockerfile with this lab. Hint: if you’re having trouble writing your Dockerfile, try booting an interactive container and installing both packages. How can you translate what you did interactively to a Dockerfile?
  3. Paste the output of running docker images command after questions 1 and 2.

Dockerizing a Web Server

For our last trick, we’re going to use Docker to run multiple Apache web servers inside containers.

For simplicity, you will not have to write this Dockerfile. Go ahead and pull the httpd image from Docker Hub. Now, it’s your job to figure out how to run three instances of the Apache containers on your machine.

Docker creates a separate network for containers, so you will need to forward your host port to your container’s port (this is called port forwarding, or port mapping). The container is listening on port 80 by default. It is your job to run each instance on ports 4000, 4001, and 4002. I recommend running the containers in detached mode with the -d flag. Detached mode will run a container in the background and print its new container ID. You can view running containers with docker ps.

Hints (if you want them):

Questions
  1. While your three containerized Apache web servers are running in detached mode, paste the output of docker ps.
  2. Observe that in the output of docker ps each container has an associated container ID. Explain why containers have IDs rather than being named after the image, for example httpd.
  3. Now go ahead and stop your containers (see slide 30). Paste the command you used to stop one of the containers.

Congratulations! You’ve successfully Dockerized and ran a web server without affecting your setup on your machine :) There’s a lot more about Docker and containers to learn about, but I hope this was enough to wrap your head around the basic concepts and get some experience working with it.

For further reading, I recommend just reading the official documentation so you can see what is possible with the Docker container format.

Puppet

To get some hands on experience with Puppet, we are going to write a very basic manifest that pulls quotes from the internet and appends them to a file every two minutes. Although you were taught that in production environments we have the puppet master running on its own server, for simplicity you will apply the manifest locally, making you both the puppet master and the puppet agent. If you are confused about the Puppet vocabulary, check out slide 21.

First things first. We plan to pull quotes from the web, so we need to ensure the wget package is installed on our system. Next we need to create a user, ‘quotes’, that runs this command for us. We create a separate user for this task because Puppet runs everything as root by default, and pulling anything from the web poses a security risk. For example, a malicious actor could perform a domain hijacking attack to get remote code execution as the root user; the severity of this threat is significantly reduced if the attack is performed on a user with less privileges. Also take note of the dependency between the ‘quotes’ user and the ‘quotegather’ group. The require line says the ‘quotegather’ group must exist for the user ‘quotes’ to be created.

Because we have full control over our user, let’s make its home directory /tmp. Since this user has one purpose, to pull quotes from the web, they don’t need a login shell. Go ahead and set it to /bin/false.

Now we need a cron resource that grabs the quotes for us. I went ahead and filled in the command, so all you need to do is specify which user is to run the cron job and the interval at which it runs (every 2 minutes).

Below I’ve provided a skeleton manifest quotes.pp. This can also be found in ~/decal-labs/b9/puppet/quotes.pp, but comments are included here for your convenience.

package { '': } # What package is required for this script to run?

group { 'quotegather':
  ensure  => '', # Do we want the group on this system?
}

user { 'quotes':
  ensure  => '', # Do we want the user on this system?
  gid     => , # Note that we can pass a group name to gid
  home    => '',
  shell   => '',
  # Do not create the user quotes unless the quotegather group exists
  require => Group['quotegather'],
}

cron { 'getquote':
  user     => '', # Which user do we want to run this cron job?
  # You can ignore this, it pulls the quotes from an API and appends
  # them to a file.
  command  => "curl -X POST http://api.forismatic.com/api/1.0/ -d 'method=getQuote' -d 'format=text' -d 'lang=en' >> quotes && echo >> quotes",
  minute   => '', # Hint: passing * means this runs every minute
}

Once you have completed your manifest, you can apply the changes to your system with the command sudo puppet apply quotes.pp.

Some tips for writing this manifest:

Wait 10 minutes or so and running cat /tmp/quotes should yield a list of quotes.

Questions
  1. Submit your completed quotes.pp file.

Congratulations! You have successfully written your own puppet manifest. This could easily be added to a puppet master and deployed on thousands of systems with ease. Take a minute to consider how powerful this technology is and curious students are encouraged to take a look at the OCF’s extensive puppet configuration to get an idea for how manifests and modules are grouped.