Lab 10 - Containers and Configuration Management
Overview
This lab is designed to give you some hands-on experience with Docker and Puppet! By the end of this assignment, you should be able to:
- Create and use a Docker container interactively and create a Dockerfile, which allows you to declaratively define your containers.
- Write a basic Puppet manifest and apply the configuration to your VM.
Keep track of your answers to the questions, as you’ll need to submit them to Gradescope. Also make sure your decal-labs repository is up to date (see lab b9)
Getting started with Docker
You have a couple of options for installing Docker, but for your convenience, here’s are links to the packages you will need to install the latest version of Docker Community Edition:
At this point, you should be able to install these packages in a breeze!
Hint: use either apt or dpkg with the appropriate commands. You might notice that these packages have dependencies, which means either you’ll need to install them in order (containerd.io -> docker-ce-cli -> docker-ce) or all at once, which you can do by just adding them in the command you run.
After installing, I recommend running sudo usermod -aG docker $USER, then logout and login again. This adds your user to the docker group so you can run docker as a non-root user. This means you won’t have to type sudo docker all the time. This is optional but for the rest of the lab I’m going to assume that you did this.
Creating your first Docker container
To verify that you installed things correctly, try running:
docker run hello-world
You should see some friendly output like so:
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete
Digest: sha256:c3b4ada4687bbaa170745b3e4dd8ac3f194ca95b2d0518b417fb47e5879d9b5f
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
This message shows that your installation appears to be working correctly.
...
This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps:
- The Docker client contacted the Docker daemon.
- The Docker daemon pulled the “hello-world” image from the Docker Hub.
- The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading.
- The Docker daemon streamed that output to the Docker client, which sent it to your terminal.
Some quick definitions from Docker’s website:
An image is a lightweight, stand-alone, executable package that includes everything needed to run a piece of software, including the code, a runtime, libraries, environment variables, and config files. Images are useful primarily for their speed, but images can also be used as a base to be built on top of in future images, as you’ll see later with Dockerfiles. In the last example hello-world was the image used to test our docker installation.
A container is a runtime instance of an image—what the image becomes in memory when actually executed. It runs completely isolated from the host environment by default, only accessing host files and ports if configured to do so. A container gets created upon executing docker run on an image.</p>
Be sure to read through the output from running the hello-world image to get an understanding of what the Docker daemon was doing.
Running an interactive container
Now, let’s try to run a container interactively. This is useful if you ever need to play around and install stuff on a bare system without messing up your current system. Try running the following command:
docker run -it ubuntu:latest
The -i flag tells docker to keep STDIN open to your container, and the -t flag allocates a pseudo TTY for you. Basically you need both for you to have a shell into your newly started container. Try installing some packages from apt or just play around. It should look like a bare Linux system.
You can exit the container with CTRL+D.
Questions
- What user are you logged in as by default in the container?
- If you start and then exit an interactive container, and then use the
docker run -it ubuntu:latest command again; is it the same container? How can you tell?
Dockerfiles
The natural question is, how are Docker images built? A Dockerfile is like the source code of an image. Rather, a Dockerfile allows you to define an image by specifying all of the commands you would type manually to create an image. Docker can then build images from a specified Dockerfile. These Dockerfiles can be put into version control and the images uploaded to online repositories. Can you see how this can be useful for deploying your application?
Dockerfiles are very powerful and have many different commands and features. We’ll go over a basic example, but you should check out the reference page if you are trying to do anything more complex.
Let’s jump in. We’re going to create an image that deploys your new startup’s app, Missile! Unfortunately, so far you only have the opening animation complete, and the source code is in decal-labs/b10/missile.py.
Your program has a couple of dependencies. Namely, it requires Python and the python packages termcolor and pyfiglet to be installed. Here is a Dockerfile that puts those requirements into code, by installing Python 3 and the packages onto a base Fedora Linux image.
# Specify Fedora Linux as base image
FROM fedora:latest
# Install Python with yum (Fedora's Package Manager)
# Install required Python packages
RUN yum update -y && yum install -y python3 python3-pip && \
python3 -m pip install pyfiglet termcolor
# Add the missile.py file to the final image
ADD missile.py /
# Specify the command to be run on container creation
CMD ["/usr/bin/python3", "missile.py"]
Note: there are some “best practices” for writing Dockerfiles that the above example doesn’t use, because it’s a basic example. If you’re interested in this stuff, check out this article.
Take a moment to appreciate how cool this is. We have a completely different Linux distribution with an application running on our system that can all be spun up with a single command. Now, when (if?) your startup finally takes off, scaling up will be a breeze!
Make sure you have both files named missile.py and Dockerfile respectively then build the image with the following command:
docker build -t missile:latest .
This tells Docker to look in the current directory for a Dockerfile to build, and builds it. The -t flag tells Docker to tag this build with the name missile:latest. Note that building the missile image will take a couple of minutes to complete.
You can see all of the images you’ve built on your machine with the docker images command.
Questions
- Run the image you just built with no flags. What do you observe?
- Write and build a
Dockerfile based on ubuntu:bionic that installs the packages fortune and fortunes-min and runs the fortune executable (located in /usr/games/fortune after you install it). Note that you won’t need to use the -it flags when you run the container as fortune doesn’t need STDIN. Submit your Dockerfile with this lab. Hint: if you’re having trouble writing your Dockerfile, try booting an interactive container and installing both packages. How can you translate what you did interactively to a Dockerfile?
- Paste the output of running
docker images command after completing questions 1 and 2.
Dockerizing a Web Server
For our last trick, we’re going to use Docker to run multiple Apache web servers inside containers.
For simplicity, you will not have to write this Dockerfile. Go ahead and pull the httpd image from Docker Hub. Now, it’s your job to figure out how to run three instances of the Apache containers on your machine.
Docker creates a separate network for containers, so you will need to forward your host port to your container’s port (this is called port forwarding, or port mapping). The container is listening on port 80 by default. It is your job to run each instance on ports 4000, 4001, and 4002. I recommend running the containers in detached mode with the -d flag. Detached mode will run a container in the background and print its new container ID. You can view running containers with docker ps.
Hints:
- The
-p flag takes in a colon separated pair of HOST_PORT:CONTAINER_PORT (it can actually accept a ton of more options, but don’t worry about that for now).
- You can see if you were successful by executing
curl localhost:4000 on your student VM. Check that you;’ve also done it correctly for ports 4001 and 4002.
- Refer to the Docker commands slide if you’re stuck!
Questions
- While your three containerized Apache web servers are running in detached mode, paste the output of
docker ps.
- Observe that in the output of
docker ps, each container has an associated container ID. Explain why containers have IDs/Names rather than being named after the image, for example httpd.
- Now go ahead and stop your containers. Paste the command you used to stop one of the containers.
Congratulations! You’ve successfully Dockerized and ran a web server without affecting your setup on your machine :) There’s a lot more about Docker and containers to learn about, but I hope this was enough to wrap your head around the basic concepts and get some experience working with it.
For further reading, I recommend just reading the official documentation so you can see what is possible with the Docker container format.
Getting started with Puppet
First, we’re going to install Puppet. Feel free to simply copy the commands below to set up Puppet. Make sure to copy the whole thing!
wget https://apt.puppetlabs.com/puppet6-release-bionic.deb && \
sudo dpkg -i puppet6-release-bionic.deb && \
sudo apt-get update && \
sudo apt-get -y install puppet
To get some hands on experience with Puppet, we are going to write a basic manifest that pulls Kanye West quotes from a service called https://kanye.rest and appends them to a file every two minutes. Although you were taught that in production environments we have the puppet master running on its own server, for simplicity you will apply the manifest locally, making you both the puppet master and the puppet agent. If you are confused about the Puppet vocabulary, review the slides!
The skeleton file quotes.pp is located inside decal-labs\b10 folder, which you will be filling in. The next few paragraphs describe what your manifest should contain.
We plan to pull quotes from the web, so we need to ensure the curl package is installed on our system. Next we need to create a user, quotes, that runs this command for us. We create a separate user for this task because Puppet runs everything as root by default, and pulling anything from the web poses a security risk. For example, a malicious actor could perform a domain hijacking attack to get remote code execution as the root user; the severity of this threat is significantly reduced if the attack is performed on a user with less privileges. Also take note of the dependency between the quotes user and the quotegather group. The require line says the quotegather group must exist for the user quotes to be created.
Because we have full control over our user, let’s make its home directory /tmp. Since this user has one purpose, to pull quotes from the web, they don’t need a login shell. Go ahead and set it to /bin/false.
Now we need a cron resource that grabs the quotes for us. I went ahead and filled in the command, so all you need to do is specify which user is to run the cron job and the interval at which it runs (every 2 minutes).
Once you have completed your manifest, you can apply the changes to your system with the command sudo puppet apply quotes.pp.
Some tips for writing this manifest:
Wait 10 minutes or so and running cat /tmp/quotes should yield a list of enlightening quotes.
Questions
- Submit your completed
quotes.pp file.
Congratulations! You have successfully written your own puppet manifest. This could easily be added to a puppet master and deployed on thousands of systems with ease. Take a minute to consider how powerful this technology is. The OCF uses Puppet extensively, and you can take a look at how we group our manifests and modules here: https://github.com/ocf/puppet/tree/master/modules