Lab 3 - Packages and Packaging

Facilitator: Ben Plate, Trinity Chung

Table of contents

1. About This Lab
   1. Grading note
   2. Workflow
2. Debian: An introduction to apt and dpkg
   1. apt
   2. dpkg
3. Getting Started
4. Exercise 1: Compiling and Packaging
   1. Writing and Compiling the Program
   2. Packaging the executable
5. Exercise 2: Troubleshooting
6. Exercise 3: Spelunking
7. For Hotshots
8. Resources

About This Lab

Grading note

Labs are graded on completion. Treat this lab as seeds of exploration instead of just a grade.

Workflow

This lab should be completed on your Linux VM, and not on tsunami. You may need root access for part of the lab.

Debian: An introduction to apt and dpkg

In this class, we will be focused on using Debian. As noted within this week’s lecture, Debian uses apt/dpkg as its package manager. Other distributions use different package managers.

apt

The frontend package manager for Debian is apt. For the majority of times when you need to deal with a package manager, apt is usually the way to go. Before doing anything with apt, it is typically a good habit to update the package list so that the package manager can find and fetch the most updated versions of various packages. To do that, you can run:

apt update

To find a package to install:

apt search [package|description]

To install a package:

apt install [package]

To remove a package:

apt remove [package]

Once you have been using the packages that you installed for a while, you may notice that they don’t automatically update themselves, a feature that may be present on programs written for other operating systems. To update the packages that you have installed, run:

apt upgrade or sometimes apt dist-upgrade

It is more commonplace to use apt upgrade to update your packages, but there are times when you need to use apt dist-upgrade. You can read up more about the differences between the two here.

In some circumstances, you want to be absolutely sure of the version of the package that you want to install. To list the potential versions that you can install, you can run:

apt policy [package]

This lists the candidate version to install, according to its pin priority, along with other versions that are compatible with the system. To install a a version for a specific target release, you can run:

apt -t [targetrelease] install [package]

There are also other commands that can remove unneeded dependencies and purge packages, but that is what the man pages are for. Please note that you are going to have to use sudo for the above commands since you are actually modifying the system itself.

dpkg

The backend package manager is dpkg. Traditionally, dpkg is used to install local packages. Using dpkg, you also can inspect packages and fix broken installs. To install local packages, run:

dpkg -i [packagefilename]

To remove a system package:

dpkg --remove [package]

To inspect a package for more information about the package:

dpkg -I [packagefilename]

To fix/configure all unpacked but unfinished installs:

dpkg --configure -a

Getting Started

We are going to use gcc to compile source code and a simple utility called fpm to create packages in this lab.

Using the commands above, install gcc, make, ruby-dev, and ruby-ffi.

Now check if GCC is installed by typing the followng:

gcc --version

Now install fpm using gem, Ruby’s own package manager:

sudo gem install fpm

Now check if fpm is installed:

fpm

Now clone the decal-web repository:

git clone https://github.com/0xcf/decal-labs.git

Exercise 1: Compiling and Packaging

Packaging manually for Debian can be very hard and frustrating, especially for first timers. That’s why for this class, we’ll be using a really cool Ruby package called fpm which simplifies the task of packaging a lot.

Note: This method is a great way to backport or package your own applications extremely quickly, but is not up to the more formal standards set by the Debian New Maintainers’ Guide. If you’re up for a challenge, feel free to try following the lab instructions, but using the guidelines here for dpkg-buildpackage instead of using fpm.

Now we will create a simplistic package using the hellopenguin executable that you will make in the coming steps. First, move into the lab 3 folder in the repository that you cloned in the Getting Started section:

cd decal-labs/3

Now we are going to create a folder to work in for this exercise:

mkdir ex1

And now move into the folder:

cd ex1

Writing and Compiling the Program

Now, we will make a very simple application in C that prints “Hello Penguin!” named hellopenguin. Invoke:

touch hellopenguin.c

This will create an empty file named hellopenguin.c. Now, using the a preferred text editor of your choice, such as vim, emacs, or nano, insert the following code into hellopenguin.c

#include <stdio.h>

int main()

{

   printf("Hello Penguin!\n");

   return 0;

}

We will now compile the source file that you have just written:

gcc hellopenguin.c -o hellopenguin

What this does is to take in a source file hellopenguin.c and compile it to an executable named hellopenguin with the -o output flag.

Packaging the executable

Now, we will create the folder structure of where the executable shall reside in. In Debian, user-level packages usually reside in the folder /usr/bin/:

mkdir -p packpenguin/usr/bin

Now move your compiled hellopenguin exectuable into the packpenguin/usr/bin/ folder.

mv hellopenguin packpenguin/usr/bin/

Now we will create a package called hellopenguin. Move into the parent directory of the packpenguin folder and invoke the following:

fpm -s dir -t deb -n hellopenguin -v 1.0~ocf1 -C packpenguin

This specifies that you want to take in a directory, using the -s flag, and to output a .deb package using the -t flag. It takes in a directory called packpenguin, using the -C flag, and output a .deb file named hellopenguin, using the -n, with a version number of 1.0~ocf1, using the -v flag.

Now test it by invoking apt and installing it:

sudo dpkg -i ./hellopenguin_1.0~ocf1_amd64.deb

Note: For m1 users, the package might be hellopenguin_1.0~ocf1_arm64.deb

Now you should be able to run hellopenguin by doing the following:

hellopenguin

Exercise 2: Troubleshooting

Now we are going to try and troubleshoot a package. Move to the other folder, ex2.

Try installing the ocfspy package using dpkg. It should error. Take note what it is erroring on! Now try and fix it.

Hint: Inspect the package for more details. The file to create that application is in the folder. Try compiling and packaging it. Exercise 1 may be a useful reference if you are stuck.

After you’re done, complete the following questions and made a submission to Gradescope.

Compiling and packaging

1. Will we still be able to run “hellopenguin” from any directory if we packaged it into “/usr/share” instead of “/usr/bin”?
2. What is your rationale for the previous answer?

Debugging

1. What package was missing after trying to install ocfspy?
2. What is the password that ocfspy outputs after fixing the dependency problem?

Note that you may want to clean up your VM by removing hellopenguin, ocfdocs, and ocfspy from your system.

Exercise 3: Spelunking

Let’s shift gears a bit and take a look at a popular package to learn more about how it’s structured! If you recall from lecture, we took at look at the contents of htop. For this next section, choose another package from the Debian repository to download and extract. You can choose any package that you’ve used/installed before (such as tmux, sl, or tree), or one from this list.

Note that this exercise is mainly for exploration and learning purposes- you wouldn’t actually install a package using this method.

Once you’ve extracted the files (using the method shown in lecture), answer the following questions on Gradescope:

1. What package did you choose?
2. What are the package’s dependencies? What file can you find them in?
3. Extract data.tar.gz and view its contents. If there exists a folder(s) other than usr/bin/ and usr/share/, pick one and briefly describe its purpose (both generally and in the context of this package). If not, explain why additional folders are not needed for this package.
4. What’s one other interesting thing you learned about this package? (Binaries you never knew existed, easter eggs in documentation, a cool pre-install script…)

Hints:

• The command to download a package is apt download <packagename>.
• To use aunpack, you might need to sudo apt install atool.
• Try to choose a package with a smaller filesize, so you won’t have to wait long for it to download and extract.
• The lecture demo will be quite helpful! You may want to watch it again for reference.

For Hotshots

In the past examples, we have always precompiled a given program before packaging it. One upside to this, is that the package will always work for systems similar to the one that you run. However, once we start introducing other machines with potentially different architectures, we suddenly need to create duplicate packages compiled specifically for those systems. Create a new package that unpacks the source code for a file, compiles it, moves all of the relevant files to their respective locations, before deleting the irrelevant files.

Resources

Below are some resources that I found helpful in the creation of this lab. If you are feeling adventurous, you may want to poke around these documents as well.

fpm

TLDR pages, a more readable man page

dpkg, alternatively man dpkg

apt, alternatively man apt

Debian New Maintainers’ Guide