@import "./support/support"; @import "./color_schemes/"; @import "./modules"; @import "./custom/custom";
Link
Sysadmin DeCal

Advanced Lab 5 - Processes and Services

6 min read

Table of contents

  1. Overview
  2. Using systemd
    1. What services are running right now?
    2. Controlling Services
    3. Creating a service
    4. Debugging
    5. Crash the service!
  3. Exploration
  4. Submission

Overview

For this lab, we are going to deep dive into the components and systemd. We will do this by writing our own systemd service from scratch, while showing the benefits of running a service with systemd. This lab should be completed on your decal VM.

Using systemd

What services are running right now?

Run systemctl. You’ll see a long table of every unit known to systemd. Let’s narrow it down to services for now. Run systemctl --type=service. Now you can see a list of all services running on your computer. Each of these services is a daemon running in the background. Do you see any familiar services running?

Question 1: What is the name of a systemd service running on your system? What does it do?

Controlling Services

Now let’s use systemd to control a an nginx web server. If you don’t have it already, install nginx by issuing sudo apt install nginx. Once that is done we can tell systemd to start the service with the following: sudo systemctl start nginx. Run systemctl status nginx to ensure it is running and navigate to http://yourvm.decal.xcf.sh/ – you should be greeted by the nginx default landing page. (You may need to update the firewall you set up in lab A4 to allow connections to port 80.)

Note: If you have a webserver running from lab A4, you may need to shut it down, so that port 80 is available for nginx to use.

Now let’s make nginx listen for connections on the nonstandard port 420. In /etc/nginx/sites-available/default change the following lines:

listen 80 default_server;
listen [::]:80 default_server;

to:

listen 420 default_server;
listen [::]:420 default_server;

TIP: The first line configures the server to listen on IPv4, and the second line configures IPv6.

Tell systemd that nginx has changed configuration and needs reloading with: sudo systemctl reload nginx. (Once again, you may need to allow port 420 through your firewall.) Now, accessing http://yourvm.decal.xcf.sh/ should now give you a connection refused error and your webserver will only be accessible via http://yourvm.decal.xcf.sh:420/.

Note that not all services can be reloaded; systemd will notify you if this is the case and such services will have to be restarted instead with: sudo systemctl restart yourservice.

Finally go ahead and stop the nginx service with sudo systemctl stop nginx.

Question 2: What is the difference between systemctl reload yourservice and systemctl restart yourservice?

Creating a service

Let’s set up a web server and create a systemd unit for it. Make sure git is installed; if it’s not, install it using apt.

To get the code, run:

$ wget https://decal.ocf.berkeley.edu/static/a5/a5.tar.gz
$ tar xvf a5.tar.gz

The materials for this part of the lab will be in the a5 directory. We will also need to install some dependencies. Go ahead and execute the following commands:

# apt update
# apt install build-essential make python3-virtualenv

Now run ./run. This should start up a simple web server at http://yourvm.decal.xcf.sh:5000. (Again… might need to open the port in your firewall.)

Your mission, should you choose to accept it, is to write a systemd service that manages this web server. To do this, make a new unit file in /etc/systemd/system/toy.service. Refer to the slides for an example; DigitalOcean also has a good guide on how to write systemd units. Here is a skeleton; all you need to do is fill in the values for each field.

[Unit]
Description=
Requires=
After=

[Install]
WantedBy=multi-user.target

[Service]
ExecStart=
User=

Some questions worth considering while writing this unit file are:

  • What units needs to be started before a webserver starts? (Hint: you can get a list of special “target” units using systemctl --type=target.)
  • What script should systemd run to start the webserver?
  • Units run by root as default. Is that a safe practice for web servers?

You are encouraged to experiment with other fields as suits your liking. Once you have finished creating toy.service, let’s start the service and have the it start whenever our machine is booted.

# systemctl start toy.service
# systemctl enable toy.service

Debugging

You can check if the unit file succeeded by running systemctl status toy.service. If you are having issues with the unit file or the web server, check the logs for this unit by running journalctl -u toy.service. If you run into errors don’t get demoralized (it is, after all, only a decal); as a sysadmin you’ll have to become comfortable making sense of arcane error messages.

TIP: You can omit the .service in systemctl command for speed. If the unit is another type (e.g. target, socket, or timer), you must include the type. We include the .service for clarity.

Crash the service!

One of the great benefits of using systemd to manage your services is that you don’t have to worry unnecessarily about bringing a process back up if it crashes. So let’s crash the service! You can do this by either sending a POST request with the json payload {"crash":"true"} to http://yourvm.decal.xcf.sh:5000/crash (Hint: use cURL) or by killing the webserver manually by sending a signal – both will cause the unit to crash. You can verify if you succeeded by running systemctl status toy.service, and the unit should either be in an inactive or failed state, depending on how you killed it.

Question 3: What command did you run to crash the service?

Now add the following the /etc/systemd/system/toy.service under the Service directive:

Restart=always
RestartSec=10

To tell systemd that the unit file has changed run sudo systemctl daemon-reload. Now start your webserver and crash it again in any way you please, and you should see that it come back online after 10 seconds! Note that you can also run daemon-reload and change a unit file while a service is running.

Question 4: Upload your fully featured toy.service file to Gradescope.

Exploration

Congratulations, you have completed the lab! This is just the tip of the iceberg when it comes to processes and services. If you want to learn more, here are some related topics you can look into.

Submission

Go to Gradescope to submit your answers!