Lab 3 - Packages and Packaging
Facilitator: Oliver Ni
9 min readTable of contents
- About This Lab
- Debian: An introduction to
apt
anddpkg
- Getting Started
- Exercise 1: Compiling and Packaging
- Exercise 2: Troubleshooting
- Exercise 3: Spelunking
- For Hotshots
- Resources
About This Lab
Grading note
Labs are graded on completion. Treat this lab as seeds of exploration instead of just a grade.
Workflow
This lab should be completed on your Linux VM, and not on tsunami. You may need root access for part of the lab.
Debian: An introduction to apt
and dpkg
In this class, we will be focused on using Debian. As noted within this week’s lecture, Debian uses apt/dpkg as its package manager. Other distributions use different package managers.
apt
The frontend package manager for Debian is apt
. For the majority of times when you need to deal with a package manager, apt
is usually the way to go. Before doing anything with apt
, it is typically a good habit to update the package list so that the package manager can find and fetch the most updated versions of various packages. To do that, you can run:
apt update
To find a package to install:
apt search [package|description]
To install a package:
apt install [package]
To remove a package:
apt remove [package]
Once you have been using the packages that you installed for a while, you may notice that they don’t automatically update themselves, a feature that may be present on programs written for other operating systems. To update the packages that you have installed, run:
apt upgrade
or sometimes apt dist-upgrade
It is more commonplace to use apt upgrade
to update your packages, but there are times when you need to use apt dist-upgrade
. You can read up more about the differences between the two here.
In some circumstances, you want to be absolutely sure of the version of the package that you want to install. To list the potential versions that you can install, you can run:
apt policy [package]
This lists the candidate version to install, according to its pin priority, along with other versions that are compatible with the system. To install a a version for a specific target release, you can run:
apt -t [targetrelease] install [package]
There are also other commands that can remove unneeded dependencies and purge packages, but that is what the man
pages are for. Please note that you are going to have to use sudo
for the above commands since you are actually modifying the system itself.
dpkg
The backend package manager is dpkg
. Traditionally, dpkg
is used to install local packages. Using dpkg
, you also can inspect packages and fix broken installs. To install local packages, run:
dpkg -i [packagefilename]
To remove a system package:
dpkg --remove [package]
To inspect a package for more information about the package:
dpkg -I [packagefilename]
To fix/configure all unpacked but unfinished installs:
dpkg --configure -a
Getting Started
We are going to use gcc
to compile source code and a simple utility called fpm
to create packages in this lab.
Using the commands above, install gcc
, make
, ruby-dev
, and ruby-ffi
.
Now check if GCC and Ruby are installed by typing the followng:
gcc --version
ruby --version
Now install fpm
using gem
, Ruby’s own package manager:
sudo gem install fpm
Now check if fpm
is installed:
fpm
Now clone the decal-labs
repository:
git clone https://github.com/0xcf/decal-labs.git
Exercise 1: Compiling and Packaging
Packaging manually for Debian can be very hard and frustrating, especially for first timers. That’s why for this class, we’ll be using a really cool Ruby package called fpm which simplifies the task of packaging a lot.
Note: This method is a great way to backport or package your own applications extremely quickly, but is not up to the more formal standards set by the Debian New Maintainers’ Guide. If you’re up for a challenge, feel free to try following the lab instructions, but using the guidelines here for dpkg-buildpackage
instead of using fpm
.
Now we will create a simplistic package using the hellopenguin executable that you will make in the coming steps. First, move into the lab 3 folder in the repository that you cloned in the Getting Started section:
cd decal-labs/3
Now we are going to create a folder to work in for this exercise:
mkdir ex1
And now move into the folder:
cd ex1
Writing and Compiling the Program
Now, we will make a very simple application in C that prints “Hello Penguin!” named hellopenguin. Invoke:
touch hellopenguin.c
This will create an empty file named hellopenguin.c
. Now, using the a preferred text editor of your choice, such as vim
, emacs
, or nano
, insert the following code into hellopenguin.c
#include <stdio.h>
int main()
{
printf("Hello Penguin!\n");
return 0;
}
We will now compile the source file that you have just written:
gcc hellopenguin.c -o hellopenguin
What this does is to take in a source file hellopenguin.c
and compile it to an executable named hellopenguin
with the -o
output flag.
Packaging the executable
Now, we will create the folder structure of where the executable shall reside in. In Debian, user-level packages usually reside in the folder /usr/bin/
:
mkdir -p packpenguin/usr/bin
Now move your compiled hellopenguin
exectuable into the packpenguin/usr/bin/
folder.
mv hellopenguin packpenguin/usr/bin/
Now we will create a package called hellopenguin
. Move into the parent directory of the packpenguin
folder and invoke the following:
fpm -s dir -t deb -n hellopenguin -v 1.0~ocf1 -C packpenguin
This specifies that you want to take in a directory, using the -s
flag, and to output a .deb
package using the -t
flag. It takes in a directory called packpenguin
, using the -C
flag, and output a .deb
file named hellopenguin
, using the -n
, with a version number of 1.0~ocf1
, using the -v
flag.
Now test it by invoking apt and installing it:
sudo dpkg -i ./hellopenguin_1.0~ocf1_amd64.deb
Note: For m1 users, the package might be hellopenguin_1.0~ocf1_arm64.deb
Now you should be able to run hellopenguin
by doing the following:
hellopenguin
Exercise 2: Troubleshooting
Now we are going to try and troubleshoot a package. Move to the other folder, ex2
.
Try installing the ocfspy
package using dpkg
. It should error. Take note what it is erroring on! Now try and fix it.
Hint: Inspect the package for more details. The file to create that application is in the folder. Try compiling and packaging it. Exercise 1 may be a useful reference if you are stuck.
After you’re done, complete the following questions and made a submission to Gradescope.
Compiling and packaging
- Will we still be able to run “hellopenguin” from any directory if we packaged it into “/usr/share” instead of “/usr/bin”?
- What is your rationale for the previous answer?
Debugging
- What package was missing after trying to install ocfspy?
- What is the password that ocfspy outputs after fixing the dependency problem?
Note that you may want to clean up your VM by removing hellopenguin
, ocfdocs
, and ocfspy
from your system.
Exercise 3: Spelunking
Let’s shift gears a bit and take a look at a popular package to learn more about how it’s structured! If you recall from lecture, we took at look at the contents of htop
. For this next section, choose another package from the Debian repository to download and extract. You can choose any package that you’ve used/installed before (such as tmux
, sl
, or tree
), or one from this list.
Note that this exercise is mainly for exploration and learning purposes- you wouldn’t actually install a package using this method.
Once you’ve extracted the files (using the method shown in lecture), answer the following questions on Gradescope:
- What package did you choose?
- What are the package’s dependencies? What file can you find them in?
- Extract
data.tar.gz
and view its contents. If there exists a folder(s) other thanusr/bin/
andusr/share/
, pick one and briefly describe its purpose (both generally and in the context of this package). If not, explain why additional folders are not needed for this package. - What’s one other interesting thing you learned about this package? (Binaries you never knew existed, easter eggs in documentation, a cool pre-install script…)
Hints:
- The command to download a package is
apt download <packagename>
. - To use
aunpack
, you might need tosudo apt install atool
. - Try to choose a package with a smaller filesize, so you won’t have to wait long for it to download and extract.
- The lecture demo will be quite helpful! You may want to watch it again for reference.
For Hotshots
In the past examples, we have always precompiled a given program before packaging it. One upside to this, is that the package will always work for systems similar to the one that you run. However, once we start introducing other machines with potentially different architectures, we suddenly need to create duplicate packages compiled specifically for those systems. Create a new package that unpacks the source code for a file, compiles it, moves all of the relevant files to their respective locations, before deleting the irrelevant files.
Resources
Below are some resources that I found helpful in the creation of this lab. If you are feeling adventurous, you may want to poke around these documents as well.
TLDR pages, a more readable man page
dpkg, alternatively man dpkg
apt, alternatively man apt