Lab 5 - Networking 102

Overview

This lab will go over some concepts of networking and how certain parts of a network stack are implemented and configured in linux systems. It is assumed that you are familiar with concepts presented in the basic lab found here.

Additionally more information about certain files discussed here can be found in their corresponding man pages by typing man <filename>.


Network Interfaces

Network interfaces represent a point of connection between a computer and a network. Typically network interfaces are associated with a physical piece of hardware e.g. a network interface card. However interfaces can be entirely implemented in software and have no physical counterpart – take the loopback interface lo for example. lo is a virtual interface, it simulates a network interface with only software.

/etc/network/interfaces

Network interface configuration is stored under the /etc/network/interfaces file on your system. This file defines how interfaces should be configured, and there is plenty of room for complexity. For example, you can have certain interfaces automatically brought up by hooking them to system boot scripts or specify some interfaces to only be available under certain circumstances, with some of the provided control flow options.

This lab will go over some common configuration keywords but keep in mind there is much more to the file. For a detailed page of the features and syntax of the file simply type man interfaces to pull up the man page for the file.

Firstly, configurations are logically divided into units known as stanzas. The /etc/network/interfaces file is comprised of zero or more stanzas which begin with iface, mapping, auto, allow-, source, or source-directory. For brevity, we will go over the two most commonly used stanzas auto and iface.

The auto stanza is fairly simple, its syntax is auto <iface>. The auto stanza flags an interface to be brought up whenever ifup is run with the -a option (More on ifup below). Additionally, since system boot scripts use ifup with the -a option, this means that these interfaces are brought up during boot. Multiple auto stanzas will be executed in the same order as they are written in the file.

The iface stanzas has much more syntax and lets you express complex configurations for individual interfaces by leveraging its features. It’s syntax is iface <iface> <address-family> <method>. Let’s go over some of the arguments the stanza takes.

<address-family> identifies the addressing that the interface will be using. The most common address families that you’re probably familiar with are:

Address families can be configured via different methods expressed by the <method> option. Some common methods you should be familiar with are:

Methods also have options that let you supply certain configuration parameters. For example, for the static method you can additionally use the address <ip-address> and netmask <mask> options to specify the static IP address and netwask you want the interface to use.

Moreover, the iface stanza additionally has its own options compatible with all families and methods. To present just a few, we have:

As a final note, any changes to the configurations done in this file during runtime are not applied automatically. Changes have to be reloaded via calls to ifupdown, the de facto command suite for interacting with /etc/network/interfaces.

/proc filesystem

proc is special since it is a virtual filesystem. proc presents runtime system information in a file-like structure. This file-like interface provides a standardized method for querying and interacting with the kernel. The kernel dumps metrics in the read-only files located in this directory and using a tool like cat lets a user dynamically read those files at runtime. But keep in mind, there are no ‘real’ files within proc.

/proc/net/

We will be focusing on certain portions of proc, the first of which being /proc/net/. This subdirectory in proc contains information about various parts of the network stack in the form of virtual files. Many commands, such as netstat, use these files when you run them.

/proc/net/dev

This file contains information and statistics on network devices. ifconfig is an example command that reads from this file. Take a look below and notice how the information presented in the ifconfig output corresponds to data dumped in dev on how many bytes and packets have been received or transmitted by an interface. alt text

/proc/net/[tcp|udp|raw]

Thetcp, raw, and udp files each contain metrics on open system sockets for their respective protocols, i.e. reading tcp displays info on TCP sockets. As a side note, raw sockets are network sockets that offer a finer degree of control over the header and payload of packets at each network layer as opposed to leaving that responsibility to the kernel. They are ideal for uses cases that send or receive packets of a type not explicitly supported by a kernel, think ICMP. For additional information check this article out. These files are read by ss, netstat, etc. Check out the example for tcp below. alt text

/proc/net/route

This file contains information about the kernel routing table. Some commands that use this file include ip and netstat. Take a look at how the file is parsed and processed by the netstat command.

alt text

/proc/net/arp

This file contains a dump of the system’s ARP cache. The arp command reads from this file. For example, look at how closely the output of the arp command resembles the raw text dumped by the kernel into the file.

alt text

/proc/net/snmp

This file contains statistics intended to be used by SNMP agents, which are a part of the Simple Network Management Protocol (SNMP). Regardless of whether or not your system is running SNMP, the data in this file is useful for investigating the network stack. Take the screenshot below for example, examining the fields we see InDiscards which according to RFC 1213 indicates packets that are discarded since problems were encountered that prevented their continued processing. Lack of buffer space is a possible cause of having a high number of discards. Having a statistic like this one, amonst others, can help pinpoint a network issue. For additional information on fields please refer to the header file here. The image is a bit small so feel free to right click -> “Open image in a new tab” to magnify the output.

alt text

/proc/sys/

Whereas the files and subdirectories mentioned above are read-only that isn’t true about the /proc/sys subdirectory which contains virtual files that also allow writes. You can not only query for system runtime parameters but also write new parameters into these files. This means you have the power to adjust kernel behavior without the need for a reboot or recompilation.

Mind-blowing.

While the /proc/sys directory contains a variety of subdirectories corresponding to aspects of the machine, the one we will be focusing on is /proc/sys/net which concerns various networking topics. Depending on configurations at the time of kernel compilation, different subdirectories are made available in /proc/sys/net, such as ethernet/, ipx/, ipv4/, and etc. Given the sheer variety of possible configurations, we will confine the scope of this discussion to the most common directories.

/proc/sys/net/core/

core/ is the first subdirectory that we’ll cover. As its name implies, it deals with core settings that direct how the kernel interacts with various networking layers.

Now we can go over some specific files in this directory, their functionality, and motivations behind adjusting them.

As a sysadmin many of the configuration decisions you make will be balancing between two extremes and the optimal point isn’t hard and fast. Many times you’ll have to adjust system parameters on a case-by-case basis and, after empirical testing, come to a good point.

/proc/sys/net/ipv4/

ipv4/ is another common subdirectory that contains setting relevant to IPv4. Often the settings used in this subdirectory are used, in conjunction with other tools, as a security measure to mitigate network attacks or to customize behavior when the system acts as a router.

One argument against disabling ICMP is that it makes obtaining diagnostic information about servers much harder. The output of tools that rely on ICMP, i.e. ping, are no longer as useful. On the other hand, allowing ICMP might be a bad idea if your goal is to hide certain machines. Additionally, ICMP has been used in the past in DOS attacks.

Additional information on configurable system parameters can be found either at this tutorial or in documentation via kernel.org or bootlin.

ARP configuration

The entries in the kernel’s arp cache can be read during system runtime via /proc/net/arp as mentioned above.

Additionally, ARP can be configured with persistent static entries. This typically done via a file. Batches of static entries can be included in such a file. The line-by-line format should be <mac-address> <ip-address>. To load the file’s entries into the system’s ARP cache one can run arp -f <file>. Typically the file that holds these entries has the path /etc/ethers.

Static ARP entries are cleared from the system ARP cache on reboot, meaning one would have to run the above command on each boot if we wanted the mappings to ‘persist’. To automate the procedure of running the command we can leverage the interface configuration workflow. Recall that /etc/network/interfaces provides the auto stanza to identify interfaces to be automatically configured on boot. Used in conjunction with the iface stanza and its post-up <command> option, we can execute the arp -f /etc/ethers command. This effectively has static entries ‘persist’ by having them added alongside interface configuration during boot.

DNS configuration

Some of the DNS configuration files that we will be going over are /etc/hosts, /etc/resolv.conf, /etc/nsswitch.conf.

/etc/hosts

This is simple text file that stores static mappings from IP addresses to hostnames. The format for each line is <ip-address> <cannonical-hostname> [aliases]. An example line would be 31.13.70.36 www.facebook.com fb ZuccBook

Thanks to this entry we have mapped www.facebook.com and any aliases we listed to 31.13.70.36. A very common example entry is localhost which also has the aliases ip6-localhost,ip6-loopback which explains why running something like ping localhost or ping ip6-loopback works. This file is one way to manually define translations for certain hostnames.

/etc/resolv.conf

Whereas /etc/hosts is for static translations of specific hostnames, many times we want to dynamically resolve names by issuing a query to a name server. There are usually many nameservers, public or private, available to fufill such a query and deciding which ones to query is the job of /etc/resolv.conf amongst other configuration options.

/etc/resolv.conf is the configuration file for the system resolver which is the entity that communciates with DNS name servers on your machine’s behalf. If this file does not exist, queries will default to the name server on your local machine. This file consists of one domain or search lines up to three nameserver lines and any number of options. Let’s dive into the details behind these configuration options.

One thing to note is that search and domain are mutually exclusive keywords and having both defined causes the last instance to take precedence and override earlier entries.

Following are additional useful configurable options in this file. Options are defined in this format options <option1> [additional-options]. Some example options follow below:

/etc/nsswitch.conf

With multiple sources of information for resolving hostnames, one can’t help but wonder how the system decides which sources to query and in what order. This is answered with the /etc/nsswitch.conf file. It is this file’s responsibility to list sources of information and configure prioritization between sources. Similar information sources can be grouped into categories that are referred to as ‘databases’ within the context of the file. The format of the file is as follows: database [sources]. While this file provides configuration for a wide array of name-service databases, we will focus on an example relevant to the topic at hand.

The hosts database configures the behavior of system name resolution. So far we have introduced two ways to resolve names:

  1. Using entries in /etc/hosts
  2. Using a resolver to issue DNS queries to DNS name servers

To let the system know about the above two sources of information there are corresponding keywords, files and dns, respectively.

We can then configure name resolution by writing the line hosts: files dns The example syntax above tells the system to first prioritize files before issuing DNS queries. Naturally, this can be customized to best fit your use case.

DHCP client configuration

The Internet Systems Consortium DHCP client, known as dhclient, ships with Debian and can be configured via /etc/dhcp/dhclient.conf. Lines in this file are terminated with a semicolon unless contained within brackets, like in the C programming language. Some potentially interesting parameters to configure include:

Timing

A client with multiple network interfaces may require different behaviour depending on the interface being configured. Timing parameters and certain declarations can be enclosed in an interface declaration, and those parameters will then be used only for the interface that matches the specified name.

The syntax for an example interface snippet is:

interface <iface-name> {
    send host-name "death.ocf.berkeley.edu";
    request subnet-mask, broadcast-address, time-offset, routers,
       domain-search, domain-name, domain-name-servers, host-name;
    [additional-declarations];
}

As mentioned above this file also supports defining static leases via a lease declaration. Defining such leases may be useful as a fallback in the event that a DHCP server cannot be contacted.

The syntax for a example static lease is:

lease {
#  interface "eth0";
#  fixed-address 192.33.137.200;
#  option host-name "death.ocf.berkeley.edu";
#  option subnet-mask 255.255.255.0;
#  option broadcast-address 192.33.137.255;
#  option routers 192.33.137.250;
#  option domain-name-servers 127.0.0.1;
#  renew 2 2000/1/12 00:00:01;
#  rebind 2 2000/1/12 00:00:01;
#  expire 2 2000/1/12 00:00:01;
#}

While the function of most keywords in the above snippet can be inferred from their syntax, more information can be found by simply reading the man page for this file (man dhclient.conf).

Sysadmin commands

ifupdown

ifupdowm is a simple suite of commands for interacting with network interfaces. The two commands you’ll be using most are ifup and ifdown which are relatively self-explanatory. ifup brings and interface up and vice versa for ifdown. These two commands should be your de facto commands for bringing interfaces up or down since using these commands loads configurations defined in /etc/network/interfaces.

mtr

mtr is a command that combines the functionality of traceroute with that of ping. Take a look at this article for a good primer for using mtr and interpreting its output.

iptables

In favor of not reinventing the wheel please check out these excellent and pretty short articles by DigitalOcean, who sponsored this semester’s offering of the decal by supplying us with VMs.

  1. An Introduction to iptables
  2. Adding rules
  3. Deleting rules
  4. Common rules and tips

Exercises

Checkoff form can be found here. Please read the sections below as well, there are instructions for setting up your environment.

Riddle Me This

  1. Is the result of running ping enough to determine whether or not you can reach a server, i.e. if you can ping a server you can reach a server and vice versa. Justify.
  2. Describe the tcp_syncookies sysctl option. How can we toggle this value on, and when would we want this on?
  3. Please write a few stanzas that configures an interface called test and configures it so that it is brought up on boot and given the following address: 192.168.13.37/16. Where should this snippet be placed?
  4. If we preferred name resolution be done dynamically rather than using static entries in /etc/hosts what file do we need to edit and what is the line we should add?
  5. Assume the following information:
    • /etc/resolv.conf file has 3 nameserver entries and a options timeout:1 entry.
    • A successful DNS response takes 20 ms.

    You need to add the retry:n option so that you retry a query as many times as possible but the total time to resolve a name, irrelevant of success or failure, takes less than 5 seconds. What should the value of n be?

:fire: This is fine :fire:

This section will have you thinking like a sysadmin.

Each script might make changes to your network stack with the intent of damaging your machine’s connectivity. To confine the scope of the ‘attacks’, scripts will specifically try to alter your connectivity to google.com and ocf.berkeley.edu. These scripts are dangerous so we are having you run them inside a disposable vm which you should install using the following steps.

  1. Install vagrant by either downloading it here or using the appropriate package manager for your system.
  2. Install virtualbox by either downloading it here or using the appropriate package manager for your system.

    As a side note the installation and configuration scripts have been tested with vagrant 2.0.2 and virtualbox 5.2.8. They have additionally been confirmed to work with vagrant v1.9.x and virtualbox 5.1.34. Compatibility issues may arise with other versions, please try shifting to the above listed versions or come to office hours.

  3. On your own local machine i.e. laptop or desktop, clone the repo found here.
  4. Go into the vm directory cd decal-labs/a5/vm.
  5. Run vagrant up. Vagrant will automatically bring up a virtual machine on your local machine according to the files in the vm directory.
  6. ssh into your machine by running ssh vagrant@192.168.42.42, if this prompts you for a password it should default to vagrant.
  7. Your vm should be configured and also have the decal-labs repo cloned to your home directory. Scripts are numbered and located under decal-labs/a5/scenarios.
  8. Launch scripts, one at a time, with sudo, i.e. sudo python3 <script.py>.
  9. For each script, follow this two step process. Only move onto another script, once you have finished resolving your current one.
    1. Analyze whether or not your connectivity has been damaged. If your stack has been damaged identify the issue or which part of your network is no longer functioning as intended.
    2. If you concluded there was a problem, resolve the issue. What commands did you use and how did you conclude things were fully functional again?

    Additionally, for each step you must explain the tools you used and how you came to your conclusions i.e.

    I ran example --pls --fix computer and I noticed that line 3: computer-is-broken meant my machine was f*****.

    This script damaged my ability to connect to google.com by poisoning my arp cache with bogus entries.

Net Ninjas

Files for this part can also be found in the repo on your vagrant vm.

  1. The ninja has spent a few years training in a dojo and has mastered fireball (火球) jutsu. He can use his new skills to tamper with your network stack, incinerating your attempts to catch him. Run sudo python3 advanced_ninja_port.py. Fix the damage he has done and then successfully send him a found you message!