Advanced Lab 2 - Packages and Packaging and Troubleshooting

Lecture: Thursday 2019-09-19

Lab Released: Monday 2019-09-23

Lab Due: Monday 2019-09-30

Grading note

Labs are graded on completion. Treat this lab as seeds of exploration instead of just a grade.

Workflow

This lab should be completed on your student VM. You should have received an email with instructions and credentials for connecting your VM. Before starting the lab, ssh into your student VM.

Debian: An introduction to apt and dpkg

In this class, we will be focused on using Debian. As noted within this week’s lecture, Debian uses apt/dpkg as its package manager. Other distributions use different package managers.

apt

The frontend package manager for Debian is apt. For the majority of times when you need to deal with a package manager, apt is usually the way to go. Before doing anything with apt, it is typically a good habit to update the package list so that the package manager can find and fetch the most updated versions of various packages. To do that, you can run:

apt update

To find a package to install:

apt search [package|description]

To install a package:

apt install [package]

To remove a package:

apt remove [package]

Once you have been using the packages that you installed for a while, you may notice that they don’t automatically update themselves, a feature that may be present on programs written for other operating systems. To update the packages that you have installed, run:

apt upgrade or sometimes apt dist-upgrade

It is more commonplace to use apt upgrade to update your packages, but there are times when you need to use apt dist-upgrade. You can read up more about the differences between the two here.

In some circumstances, you want to be absolutely sure of the version of the package that you want to install. To list the potential versions that you can install, you can run:

apt policy [package]

This lists the candidate version to install, according to its pin priority, along with other versions that are compatible with the system. To install a a version for a specific target release, you can run:

apt -t [targetrelease] install [package]

There are also other commands that can remove unneeded dependencies and purge packages, but that is what the man pages are for. Please note that you are going to have to use sudo for the above commands since you are actually modifying the system itself.

dpkg

The backend package manager is dpkg. Traditionally, dpkg is used to install local packages. Using dpkg, you also can inspect packages and fix broken installs. To install local packages, run:

dpkg -i [packagefilename]

To remove a system package:

dpkg --remove [package]

To inspect a package for more information about the package:

dpkg -I [packagefilename]

To fix/configure all unpacked but unfinished installs:

dpkg --configure -a

Getting Started

We are going to use gcc to compile source code and a simple utility called fpm to create packages in this lab.

Using the commands above, install gcc, make, ruby-dev, and ruby-ffi.

Now check if GCC is installed by typing the followng:

gcc --version

Now install fpm using gem, Ruby’s own package manager:

sudo gem install fpm

Now check if fpm is installed:

fpm

Now clone the decal-web repository:

git clone https://github.com/0xcf/decal-labs.git

Exercise 1: Compiling and Packaging

Packaging manually for Debian can be very hard and frustrating, especially for first timers. That’s why for this class, we’ll be using a really cool Ruby package called fpm which simplifies the task of packaging a lot. Please note that if you were going to package for Debian’s own repositories, this is NOT the correct/formal way to go about packaging. It is merely a great way to backport or package your own applications extremely quickly.

Now we will create a simplistic package using the hellopenguin executable that you will make in the coming steps. First, move into the a2 folder in the repository that you cloned in the Getting Started section:

cd decal-labs/a2

Now we are going to create a folder to work in for this exercise:

mkdir ex1

And now move into the folder:

cd ex1

Writing and Compiling the Program

Now, we will make a very simple application in C that prints “Hello Penguin!” named hellopenguin. Invoke:

touch hellopenguin.c

This will create an empty file named hellopenguin.c. Now, using the a preferred text editor of your choice, such as vim, emacs, or nano, insert the following code into hellopenguin.c

#include <stdio.h>

int main()

{

   printf("Hello Penguin!\n");

   return 0;

}

We will now compile the source file that you have just written:

gcc hellopenguin.c -o hellopenguin

What this does is to take in a source file hellopenguin.c and compile it to an executable named hellopenguin with the -o output flag.

Packaging the executable

Now, we will create the folder structure of where the executable shall reside in. In Debian, user-level packages usually reside in the folder /usr/bin/:

mkdir -p packpenguin/usr/bin

Now move your compiled hellopenguin exectuable into the packpenguin/usr/bin/ folder.

mv hellopenguin packpenguin/usr/bin/

Now we will create a package called hellopenguin. Move into the parent directory of the hellopenguin folder and invoke the following:

fpm -s dir -t deb -n hellopenguin -v 1.0~ocf1 -C packpenguin

This specifies that you want to take in a directory, using the -s flag, and to output a .deb package using the -t flag. It takes in a directory called packpenguin, using the -C flag, and output a .deb file named hellopenguin, using the -n, with a version number of 1.0~ocf1, using the -v flag.

Now test it by invoking apt and installing it:

sudo dpkg -i ./hellopenguin_1.0~ocf1_amd64.deb

Now you should be able to run hellopenguin by doing the following:

hellopenguin

Exercise 2: Troubleshooting

Now we are going to try and troubleshoot a package. Move to the other folder, ex2.

Try installing the ocfspy package using dpkg. It should error. Take note what it is erroring on! Now try and fix it.

Hint: Inspect the package for more details. The file to create that application is in the folder. Try compiling and packaging it. Exercise 1 may be a useful reference if you are stuck.

Conclusion

After you’re done, complete the following questions and made a submission to gradescope.

Compiling and packaging

  1. Will we still be able to run “hellopenguin” from any directory if we packaged it into “/usr/share” instead of “/usr/bin”?
  2. What is your rationale for the previous answer?

Debugging

  1. What package was missing after trying to install ocfspy?
  2. What is the password that ocfspy outputs after fixing the dependency problem?

Note that you may want to clean up your VM by removing hellopenguin, ocfdocs, and ocfspy from your system.

For Hotshots

In the past examples, we have always precompiled a given program before packaging it. One upside to this, is that the package will always work for systems similar to the one that you run. However, once we start introducing other machines with potentially different architectures, we suddenly need to create duplicate packages compiled specifically for those systems. Create a new package that unpacks the source code for a file, compiles it, moves all of the relevant files to their respective locations, before deleting the irrelevant files.

Resources

Below are some resources that I found helpful in the creation of this lab. If you are feeling adventurous, you may want to poke around these documents as well.

fpm

TLDR pages, a more readable man page

dpkg, alternatively man dpkg

apt, alternatively man apt