Lab 6 - Networking 102

Overview

This lab will go over some concepts of networking and how certain parts of a network stack are implemented and configured in linux systems. It is assumed that you are familiar with basic networking concepts such as those presented in Lab b5.

Additional information about certain files discussed here can be found in their corresponding man pages by typing man <filename>.

As with the previous labs, there will be questions sprinkled throughout- head over to Gradescope to submit your answers to them!


Network Interfaces

Network interfaces represent a point of connection between a computer and a network. Typically network interfaces are associated with a physical piece of hardware, like a network interface card. However, interfaces can also be entirely implemented in software and have no physical counterpart – take the loopback interface lo for example. lo is a virtual interface; it simulates a network interface with only software.

/etc/network/interfaces

Network interface configurations are stored under the /etc/network/interfaces file on your system. Here, there is plenty of room for complexity. For example, you can have certain interfaces automatically brought up by hooking them to system boot scripts or specify some interfaces to only be available under certain circumstances, with some of the provided control flow options.

This lab will go over some common configuration keywords, but there is much more to the file. For a detailed page of the features and syntax of the file simply type man interfaces to pull up the man page for the file.

Firstly, configurations are logically divided into units known as stanzas. The /etc/network/interfaces file is comprised of zero or more stanzas which begin with iface, mapping, auto, allow-, source, or source-directory. For brevity, we will go over the two most commonly used stanzas auto and iface.

The auto stanza is fairly simple, its syntax is auto <iface>. The auto stanza flags an interface to be brought up whenever ifup is run with the -a option (More on ifup below). Since system boot scripts use ifup with the -a option, these interfaces are brought up during boot. Multiple auto stanzas will be executed in the same order as they are written in the file.

The iface stanzas lets you express complex configurations for individual interfaces by leveraging its features. Its syntax is iface <iface> <address-family> <method>. Let’s go over some of the arguments the stanza takes.

<address-family> identifies the addressing that the interface will be using. The most common address families that you’re probably familiar with are:

Address families can be configured via different methods expressed by the <method> option. Some common methods you should be familiar with are:

Methods also have options that let you supply certain configuration parameters. For example, for the static method you can additionally use the address <ip-address> and netmask <mask> options to specify the static IP address and netwask you want the interface to use.

Moreover, the iface stanza additionally has its own options compatible with all families and methods. To present just a few, we have:

As a final note, any changes to the configurations done in this file during runtime are not applied automatically. Changes have to be reloaded via calls to ifupdown, the de facto command suite for interacting with /etc/network/interfaces.

Checkpoint

Question 1a: Is the result of running ping enough to determine whether or not you can reach a server? Why or why not?

Question 1b: Here’s a quick check for your understanding – below is a very common default configuration for /etc/network/interfaces:

auto lo
iface lo inet loopback

In your own words, explain what this configuration does. What would happen if you deleted these lines and rebooted?

Question 1c: Write a few stanzas that configure an interface called test that is brought up on boot and given the following address: 192.168.13.37/16.

/proc filesystem

proc is a virtual filesystem that presents runtime system information in a file-like structure. This file-like interface provides a standardized method for querying and interacting with the kernel, which dumps metrics in the read-only files located in this directory. Using a tool like cat, you can dynamically read those files at runtime. But keep in mind, there are no ‘real’ files within proc.

/proc/net/

We will be focusing on certain portions of proc, the first of which being /proc/net/. This subdirectory in proc contains information about various parts of the network stack in the form of virtual files. Many commands, such as netstat, use these files when you run them.

/proc/net/dev

This file contains information and statistics on network devices. ifconfig is an example command that reads from this file. Take a look below and notice how the information presented in the ifconfig output corresponds to data dumped in dev on how many bytes and packets have been received or transmitted by an interface. alt text

/proc/net/[tcp|udp|raw]

Thetcp, raw, and udp files each contain metrics on open system sockets for their respective protocols, i.e. reading tcp displays info on TCP sockets. As a side note, raw sockets are network sockets that offer a finer degree of control over the header and payload of packets at each network layer as opposed to leaving that responsibility to the kernel. They are ideal for uses cases that send or receive packets of a type not explicitly supported by a kernel, think ICMP. For additional information check this article out. These files are read by ss, netstat, etc. Check out the example for tcp below. alt text

/proc/net/route

This file contains information about the kernel routing table. Some commands that use this file include ip and netstat. Take a look at how the file is parsed and processed by the netstat command.

alt text

/proc/net/arp

This file contains a dump of the system’s ARP cache. The arp command reads from this file. For example, look at how closely the output of the arp command resembles the raw text dumped by the kernel into the file.

alt text

/proc/net/snmp

This file contains statistics intended to be used by SNMP agents, which are a part of the Simple Network Management Protocol (SNMP). Regardless of whether or not your system is running SNMP, the data in this file is useful for investigating the network stack. Take the screenshot below for example, examining the fields we see InDiscards which according to RFC 1213 indicates packets that are discarded since problems were encountered that prevented their continued processing. Lack of buffer space is a possible cause of having a high number of discards. Having a statistic like this one, amonst others, can help pinpoint a network issue. For additional information on fields please refer to the header file here. The image is a bit small so feel free to right click -> “Open image in a new tab” to magnify the output.

alt text

/proc/sys/

Whereas the files and subdirectories mentioned above are read-only that isn’t true about the /proc/sys subdirectory which contains virtual files that also allow writes. You can not only query for system runtime parameters but also write new parameters into these files. This means you have the power to adjust kernel behavior without the need for a reboot or recompilation.

Mind-blowing.

While the /proc/sys directory contains a variety of subdirectories corresponding to aspects of the machine, the one we will be focusing on is /proc/sys/net which concerns various networking topics. Depending on configurations at the time of kernel compilation, different subdirectories are made available in /proc/sys/net, such as ethernet/, ipx/, ipv4/, and etc. Given the sheer variety of possible configurations, we will confine the scope of this discussion to the most common directories.

/proc/sys/net/core/

core/ is the first subdirectory that we’ll cover. As its name implies, it deals with core settings that direct how the kernel interacts with various networking layers.

Now we can go over some specific files in this directory, their functionality, and motivations behind adjusting them.

As a sysadmin many of the configuration decisions you make will be balancing between two extremes and the optimal point isn’t hard and fast. Many times you’ll have to adjust system parameters on a case-by-case basis and, after empirical testing, come to a good point.

/proc/sys/net/ipv4/

ipv4/ is another common subdirectory that contains setting relevant to IPv4. Often the settings used in this subdirectory are used, in conjunction with other tools, as a security measure to mitigate network attacks or to customize behavior when the system acts as a router.

One argument against disabling ICMP is that it makes obtaining diagnostic information about servers much harder. The output of tools that rely on ICMP, i.e. ping, are no longer as useful. On the other hand, allowing ICMP might be a bad idea if your goal is to hide certain machines. Additionally, ICMP has been used in the past in DOS attacks.

Additional information on configurable system parameters can be found either at this tutorial or in documentation via kernel.org or bootlin.

Checkpoint

Question 2a: Describe the tcp_syncookies sysctl option. How can we toggle this value on, and when would we want this on?

ARP configuration

The entries in the kernel’s arp cache can be read during system runtime via /proc/net/arp as mentioned above.

Additionally, ARP can be configured with persistent static entries. This typically done via a file. Batches of static entries can be included in such a file. The line-by-line format should be <mac-address> <ip-address>. To load the file’s entries into the system’s ARP cache one can run arp -f <file>. Typically the file that holds these entries has the path /etc/ethers.

Static ARP entries are cleared from the system ARP cache on reboot, meaning one would have to run the above command on each boot if we wanted the mappings to ‘persist’. To automate the procedure of running the command we can leverage the interface configuration workflow. Recall that /etc/network/interfaces provides the auto stanza to identify interfaces to be automatically configured on boot. Used in conjunction with the iface stanza and its post-up <command> option, we can execute the arp -f /etc/ethers command. This effectively has static entries ‘persist’ by having them added alongside interface configuration during boot.

DNS configuration

Some of the DNS configuration files that we will be going over are /etc/hosts, /etc/resolv.conf, /etc/nsswitch.conf.

/etc/hosts

This is simple text file that stores static mappings from IP addresses to hostnames. The format for each line is <ip-address> <cannonical-hostname> [aliases]. An example line would be 31.13.70.36 www.facebook.com fb ZuccBook

Thanks to this entry we have mapped www.facebook.com and any aliases we listed to 31.13.70.36. A very common example entry is localhost which also has the aliases ip6-localhost,ip6-loopback which explains why running something like ping localhost or ping ip6-loopback works. This file is one way to manually define translations for certain hostnames.

/etc/resolv.conf

Whereas /etc/hosts is for static translations of specific hostnames, many times we want to dynamically resolve names by issuing a query to a name server. There are usually many nameservers, public or private, available to fufill such a query and deciding which ones to query is the job of /etc/resolv.conf amongst other configuration options.

/etc/resolv.conf is the configuration file for the system resolver which is the entity that communciates with DNS name servers on your machine’s behalf. If this file does not exist, queries will default to the name server on your local machine. This file consists of one domain or search lines up to three nameserver lines and any number of options. Let’s dive into the details behind these configuration options.

One thing to note is that search and domain are mutually exclusive keywords and having both defined causes the last instance to take precedence and override earlier entries.

Following are additional useful configurable options in this file. Options are defined in this format options <option1> [additional-options]. Some example options follow below:

/etc/nsswitch.conf

With multiple sources of information for resolving hostnames, one can’t help but wonder how the system decides which sources to query and in what order. This is answered with the /etc/nsswitch.conf file. It is this file’s responsibility to list sources of information and configure prioritization between sources. Similar information sources can be grouped into categories that are referred to as ‘databases’ within the context of the file. The format of the file is as follows: database [sources]. While this file provides configuration for a wide array of name-service databases, we will focus on an example relevant to the topic at hand.

The hosts database configures the behavior of system name resolution. So far we have introduced two ways to resolve names:

  1. Using entries in /etc/hosts
  2. Using a resolver to issue DNS queries to DNS name servers

To let the system know about the above two sources of information there are corresponding keywords, files and dns, respectively.

We can then configure name resolution by writing the line hosts: files dns The example syntax above tells the system to first prioritize files before issuing DNS queries. Naturally, this can be customized to best fit your use case.

DHCP client configuration

The Internet Systems Consortium DHCP client, known as dhclient, ships with Debian and can be configured via /etc/dhcp/dhclient.conf. Lines in this file are terminated with a semicolon unless contained within brackets, like in the C programming language. Some potentially interesting parameters to configure include:

Timing

A client with multiple network interfaces may require different behaviour depending on the interface being configured. Timing parameters and certain declarations can be enclosed in an interface declaration, and those parameters will then be used only for the interface that matches the specified name.

The syntax for an example interface snippet is:

interface <iface-name> {
    send host-name "death.ocf.berkeley.edu";
    request subnet-mask, broadcast-address, time-offset, routers,
       domain-search, domain-name, domain-name-servers, host-name;
    [additional-declarations];
}

As mentioned above this file also supports defining static leases via a lease declaration. Defining such leases may be useful as a fallback in the event that a DHCP server cannot be contacted.

The syntax for a example static lease is:

lease {
#  interface "eth0";
#  fixed-address 192.33.137.200;
#  option host-name "death.ocf.berkeley.edu";
#  option subnet-mask 255.255.255.0;
#  option broadcast-address 192.33.137.255;
#  option routers 192.33.137.250;
#  option domain-name-servers 127.0.0.1;
#  renew 2 2000/1/12 00:00:01;
#  rebind 2 2000/1/12 00:00:01;
#  expire 2 2000/1/12 00:00:01;
#}

While the function of most keywords in the above snippet can be inferred from their syntax, more information can be found by simply reading the man page for this file (man dhclient.conf).

Sysadmin commands

ifupdown

ifupdowm is a simple suite of commands for interacting with network interfaces. The two commands you’ll be using most are ifup and ifdown which are relatively self-explanatory. ifup brings and interface up and vice versa for ifdown. These two commands should be your de facto commands for bringing interfaces up or down since using these commands loads configurations defined in /etc/network/interfaces.

mtr

mtr is a command that combines the functionality of traceroute with that of ping. Take a look at this article for a good primer for using mtr and interpreting its output.

iptables

In favor of not reinventing the wheel please check out these excellent and pretty short articles by DigitalOcean, who sponsored this semester’s offering of the decal by supplying us with VMs.

  1. An Introduction to iptables
  2. Adding rules
  3. Deleting rules
  4. Common rules and tips

Checkpoint

Question 3a: If we preferred name resolution be done dynamically rather than using static entries in /etc/hosts what file do we need to edit and what is the line we should add?

Question 3b: Assume the following information:

Exercises

Now, let’s break things do some experimentation! Remember to submit your answers to Gradescope when you’re done. (Also, don’t forget to submit the checkpoint questions from the sections above!)

The files for these exercises can be found in the decal-labs repository. Clone it now: git clone https://github.com/0xcf/decal-labs

đŸ”„ This is fine đŸ”„

This section will have you thinking like a sysadmin.

IMPORTANT NOTE

Do not run the scripts directly in your student VM! These scripts are dangerous and will brick your VM so please follow the provided setup instructions.

Each script might make changes to your network stack with the intent of damaging your machine’s connectivity. To confine the scope of the ‘attacks’, scripts will specifically try to alter your connectivity to google.com and ocf.berkeley.edu.

Setup

  1. If you haven’t already, ssh into your student VM (username@username.decal.xcf.sh) and clone decal-labs.
  2. Go into the vm directory: cd decal-labs/a6/vm
  3. Get Virtualbox: sudo apt install virtualbox
  4. Get Vagrant: curl -O https://releases.hashicorp.com/vagrant/2.2.6/vagrant_2.2.6_x86_64.deb then sudo apt install ./vagrant_2.2.6_x86_64.deb
  5. Start a Vagrant instance: vagrant up
  6. Enter your Vagrant instance: ssh vagrant@192.168.42.42. The default password is vagrant.
  7. The decal-labs repo should be available in the Vagrant instance. If it isn’t there, you can sudo apt install git then clone it again.
  8. cd decal-labs/a6/scenario.

Problem Instructions

There should be 6 scripts, named 1.py to 6.py. Your task is to choose at least 3 of these to run and attempt to fix the problem that they cause (if any).

Launch each script with sudo, i.e. sudo python3 <script.py>.

For each script, follow this two step process. Only move onto another script once you have finished resolving your current one.

  1. Analyze whether or not your connectivity has been damaged. If your stack has been damaged identify the issue or which part of your network is no longer functioning as intended.

  2. If you concluded there was a problem, resolve the issue. What commands did you use and how did you conclude things were fully functional again?

    Additionally, for each step you must explain the tools you used and how you came to your conclusions i.e.

    I ran example --pls --fix computer and I noticed that line 3: computer-is-broken meant my machine was f*****.

    This script damaged my ability to connect to google.com by poisoning my arp cache with bogus entries.

Common Issues

I can’t ssh into my Vagrant instance!

I accidentally bricked something, how do I reset my Vagrant instance?

Net Ninjas

Run this in your Vagrant instance. The file should be found in decal-labs/a6/.

  1. The ninja has spent a few years training in a dojo and has mastered fireball (火球) jutsu. He can use his new skills to tamper with your network stack, incinerating your attempts to catch him. Run sudo python3 advanced_ninja_port.py. Fix the damage he has done and then successfully send him a found you message!