Lab 5 - Introduction to Networking
Facilitator: Mark Zhang17 min read
- Sysadmin Commands
It is undeniable that the internet is an important system that has redefined our world. The ability to develop networks and allow devices to communicate is critical to modern day computer systems. This lab will take a look into the basics of computer networking and then examine networks through the perspective of a sysadmin.
We will be using web browsing as an analogy to understand the basics of networking. What exactly happens when I go web browsing for cat pictures?
But first let’s take a short dive into the details of networking.
Media access control (MAC) addresses are identifiers uniquely assigned to network interfaces.
Since the MAC address is unique this is often referred to as the physical address. The octets are often written in hexadecimal and delimited by colons. An example MAC address is
00:14:22:01:23:45. Note that the first 3 octets refer to the Organizationally Unique Identifier (OUI) which can help identify manufacturers. Fun fact – the
00:14:22 above is an OUI for Dell.
IP addresses are means of identifying devices connected to a network under Internet Protocol. There are two versions of the internet protocol, IPv4 and IPv6, that which differ on the size of their addresses. An example IPv6 address is
2001:0db8:85a3:0000:0000:8a2e:0370:7334 which is considerably longer than an IPv4 address like
127.0.0.1. For the sake of time we will only go over IPv4, but IPv6 is certainly gaining ground and worth checking out!
IPv4 addresses are 32 bits, i.e. 4 bytes, long and are delimited by a dot (.) every byte. An example IPv4 address is
127.0.0.1. Coincidentally this address is known as the loopback address which maps to the loopback interface on your own machine. This allows network applications to communicate with one another if they are running on the same machine, in this case your machine. But why
127.0.0.1 and not
The answer is that
127.0.0.1 is simply convention, but technically any address in the network block
127.0.0.0/8 is a valid loopback address. But what exactly is a network block?
In IPv4 we can partition a block of addresses into a subnet. This is written in a format known as CIDR. Let’s take the subnet above as an example
127.0.0.0/8. The number that comes after the slash (
/), in this case 8, is the subnet mask. This represents how many bits are in the network address, the remaining bits identify a host within the network. In this case the network address is
127.0.0.0 and the Mask is
127.0.0.1 would be the first host in the
127.0.0.0/8 network and so on and so forth.
This diagram provides a visual breakdown of CIDR addressing
Address Resolution Protocol (ARP) is a protocol used to resolve IP addresses to MAC addresses. In order to understand ARP, we first discuss two ways to send a frame, unicast and broadcast. In the context of Layer 2, unicasting a frame means to send that frame to exactly one MAC address. On the other hand, broadcasting a frame by sending it to the broadcast address means the frame should be sent to every device on the network, effectively “flooding” the local network.
For example let’s imagine a sender, A, who has MAC
00:DE:AD:BE:EF:00, broadcasting a message that essentially asks “Who has IP address
220.127.116.11 please tell A at 00:DE:AD:BE:EF:00”.
If a machine, B, with MAC
12:34:56:78:9a:bc has the IP address
18.104.22.168 they send a unicast reply back to the sender with the info “
22.214.171.124”. The sender stores this information in an arp table so whenever it receives packets meant for machine B i.e. a packet with an destination IP address of
126.96.36.199 it sends the packet to MAC it received from B.
In order to route IP packets, devices have what is known as a routing table. Routing entries are stored in the routing table and they are essentially rules that tell the device how packets should be forwarded based on IP. A routing entry specifies a subnet and the interface that corresponds to that entry. The device chooses an entry with a subnet that is most specific to a given packet and forwards it out the interface on that entry.
Routing tables are usually to also have a default gateway. This serves as the default catch all for packets in the absence of a more specific matching entry.
Take this routing table for example.
default via 10.0.2.2 dev eth0 10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.15 10.0.2.128/25 dev eth0 proto kernel scope link src 10.0.2.15 192.168.162.0/24 dev eth1 proto kernel scope link src 192.168.162.162
A packet destined for
188.8.131.52 would be forwarded out eth0, the default gateway.
A packet destined for
10.0.2.1 would be forwarded according to the second entry, out of eth0.
A packet destined for
10.0.2.254 would be forwarded according to the third entry, out of eth0.
A packet destined for
192.168.162.254 would be forwarded according to the fourth entry, out of eth1.
We’ve gone over IP addresses and how they are means of communicating with a host over IP, but while IP addresses are machine friendly (computers love numbers) they aren’t exactly human friendly. It’s hard enough trying to remember phone numbers, memorizing 32 bit IP addresses isn’t going to be any easier.
But it’s much easier for us to remember names like www.google.com, www.facebook.com, or coolmath-games.com. So out of this conflict the Domain Name System (DNS) was born as a compromise between machine friendly IP addresses and human friendly domain names.
DNS is a system that maps a domain name like google.com to
184.108.40.206. When you query for google.com your computer sends out a DNS query for google.com to a DNS server. Assuming things are properly configured and google.com has a valid corresponding address you will receive a response from an authoritative server that essentially says “google.com has IP address
Now let’s flush out this black magic a bit…
DNS servers store data in the form of Resource Records (RR). Resource records are essentially a tuple of (name, value, type, TTL). While there are a wide variety of types of DNS Records the ones we are most concerned with are
A records name = hostname value = IP address
This record is very simply the record that has the IP address for a given hostname, essentially the information we want to end up with.
NS records name = domain value = name of dns server for domain
This record points to another dns server that can provide an authoritative answer for the domain. Think of this as redirecting you to another nameserver.
CNAME records name = alias value = canonical name
These records point to the canonical name for a given alias for example docs.google.com would be an alias which simply points to documents.google.com try www.facebook.com
MX records The record used by mail service.
Now we will transition into a discussion on the protocols at the transport layer. The two most well known protocols at this layer are Transmission Control Protocol (TCP) and User Datagram Protocol (UDP).
TCP is a stateful stream oriented protocol that ensures reliable transport. Reliable transport essentially guarantees that information arrives wholly intact and in order at the destination.
TCP is a connection oriented protocol which means it must first establish a connection before sending any data. This connection exchanges information that is the mechanisms TCP uses to provide reliable transport amongst other features. A TCP connection begins with something known as the TCP handshake.
The TCP handshake consists of setting certain flags in the TCP header of packets exchanged between sender and receiver. The sender initiating a TCP connection by first sending a SYN, a packet with the SYN flag set. The server acknowledges this connection request by sending back a SYN-ACK, a packet with both the SYN and ACK flags set. The client acknowledges this by sending one final ACK back to the server, and the connection is then established.
TCP then begins transmitting data and if it successfully arrives on the other end of the connection then an ACK is issued. Therefore if data is lost, reordered, or corrupted, TCP is capable of recognizing this and sends a request for retransmission of any lost data.
TCP also has a procedure to close connections. We only consider a graceful termination here, abrupt terminations have a different procedure we will not go over. If you’re interested, CS168 has some great material here. Let’s assume machine A wants to close its connection to machine B.
A begins by sending a FIN. B must respond by sending a FIN and an ACK. If B only sends a ACK the connection persist and additional data can be sent until an FIN is sent. On the other hand B can also send just one packet with both FIN and ACK flags set, i.e. FIN+ACK if B is ready to close the connection and doesn’t need to send additional data Once A has received a FIN and an ACK it sends one last ACK to signal the connection termination.
UDP is stateless connectionless protocol. UDP focuses on sending messages in datagrams. Being connectionless UDP also doesn’t incur the overhead of the TCP handshake and termination. UDP also makes no guarantees about reliable transport so messages may be corrupted, arrive out of order, or not arrive at all. For this reason UDP is sometimes called Unreliable Datagram Protocol.
While UDP makes no guarantees about reliable transport it doesn’t suffer from the overhead of establishing and closing connections like in TCP. UDP is therefore ideal for usage cases where we just want to send packets quickly and losing a few of those isn’t disastrous.
Moreover, compared to TCP each UDP datagram sent needs to be individually received. While for TCP you pass a stream of data that is transparently split into some number of sends and the data stream is transparently reconstructed as a whole on the other end.
Ports define a service endpoint, broadly speaking – ports mark a point of traffic ingress and egress. Whereas IP addresses connect hosts, ports connect process that run on such hosts. Only one process can be bound to a port at a time. Ports are represented by a 16 bit number meaning thus ranging from 0 to 65535. Ports from 0 to 1023 are well known ports, i.e. system ports. Using these ports usually has a stricter requirement. 1024 to 49151 are registered ports. IANA maintains the official list of well-known and registered ranges. The remaining ports from 49152 to 65535 are ephemeral ports which can be dynamically allocated for communication sessions on a per request basis.
Some port numbers for well known services are as follows:
As a sysadmin, trying to diagnose network issues can often be pretty challenging. Given the scale and complexity of networks, it’s tough trying to narrow down the scope of a problem to a point of failure. What follows is a list of commands/tools that can help with triaging problems. There are a lot of tools and we don’t expect you to memorize every single detail. However, it is important to know what tools exist and when to use them when problems inevitably arise. If you ever need more details the
man pages for these commands are a great place to turn to for reference.
Tools also tend to overlap in functionality – for example there are multiple tools that can display interface information or test connectivity. When possible, it is a good idea to use multiple tools to cross-check one another.
Note that when it comes to real world networks there are even more factors to consider that we haven’t touched on like network security. For example, two machines can have a fully functioning connection but if one machine has been configured to drop all packets then it might seem as if they aren’t connected.
So take the output of these tools with a grain of salt, they a means of narrowing down issues. It is important not to misinterpret outputs or jump to conclusions too quickly.
A simple and straightforward command that can display information about a host, IP addresses, FQDN, and etc. Make sure to also check out
host, which is a similar command that provides more detailed information by doing a lookup on a given name.
Another simple command, most of the time you’ll be using ping as a first step towards testing connectivity. If ping can’t reach a host then there is likely an issue with connectivity. The ping tool does this by sending out ICMP messages to the host expecting a response. (More on the protocol here)
Moreover ping also provides metrics for Round Trip Time (RTT) and packet loss. Round trip time is defined as the time it takes for a response to arrive after sending the ping packet. These can prove to be very useful statistics.
Traceroute sends packets Time to Live (TTL) equal to the number of hops. Routers decrease the value of TTL for incoming packets. If a packet’s TTL = 0 then the router drops it and may send back diagnostic information to the source about the router’s identity. Otherwise the router continues forwarding the packet.
Traceroute provides a detailed view of the routers that a packet traverses while on its way to a destination.
If router does not respond within a timeout then traceroute prints an asterisk.
Provides info on and the ability to manipulate the ARP cache of the system.
With arp you can display the system arp table. Add, remove, or modify arp entries and much more.
Utility for doing dns query and triaging DNS issues.
Dig by default performs queries to nameservers in
/etc/resolv.confbut some options allow you to: specify name server, choose query type (iterative vs recursive), and much more – making
diga very flexible DNS tool.
ipis a command with many subcommands offering a lot of functionality – so much it can be overwhelming at first. You will most commonly be using ip to display/modify routing, IP addresses, or network interfaces.
It will take time to get use to how much functionality is included in this command but for reference here is a pretty compact cheatsheet. A few common use cases include:
ip addrwhich displays information on your IP addresses,
ip routewhich displays information on your routing table, and
ip linkwhich displays information about your network interfaces.
cURL does as its name suggests, and allows you to see the contents at certain URLs. Beyond this it’s also an extremely powerful program that lets you interact with and inspect servers over several different protocols certain protocols such as HTTP, FTP, etc …
Be sure to check out its documentation for specific use cases.
wgetis quite similar to
curlin the sense that they are both command line tools designed to transfer data from or to servers with certain protocols and both come with a bunch of features.
There are differences between the commands, two notable examples being that wget is command line only meaning there no library or API. However, wget has a big advantage of being able to download recursively. You can read a bit more on the two tools here.
netstat(Optional) This tool is good for printing network connections, routing tables, and probing sockets, amongst other functions.
netstatalso has functionality to probe sockets for activity and displays information such protocol (UDP/TCP)
If you are investigating sockets
lsofare also options you may want to consider
tcpdump(Optional) Perfect for monitoring incoming or outgoing traffic on a machine.
tcpdumpoffers countless options when it comes to analyzing traffic: it can capture packets, log traffic, compute metrics, filter traffic, monitor specific interfaces, etc. As a primer you can check out these examples.
nc(Optional) Netcat is a very powerful tool that can be used for just about anything involving TCP or UDP. It can open TCP connections, send UDP packets, listen on arbitrary TCP and UDP ports, do port scanning, and deal with both IPv4 and IPv6.
- Does HTTP use TCP or UDP and why? How about Discord and Skype, why?
- Who manufactured the NIC with mac address 52:54:00:d7:ce:cc?
- How many distinct hosts can 127.0.0.0/8 contain?
- What types of records do you get when you perform a DNS lookup of facebook.com?
- Write a shell script
is_on.sh hostshows whether
hostis online. If it is, show “OK”. If it’s not, show “Host is not reachable”. Don’t show anything else. Some clarifications:
- A host is online here means the
pingto the host is successful
pingthe server once (we assume the internet connection is reliable and the packet will not be dropped)
- You can use
man pingto see how to make the
pingonly ping the server once, and what the return value of
pingcommand means. Use
ifto decide what to print.
- A host is online here means the
- Write a shell script
mac.shwhich processes the output of
ipcommand and displays the MAC address of the network interface
eth0of your VM.
- First figure out how to use
ipcommand to get an output which contains the information we want
- Then use
tailcommand and pipes to tailor
ip’s output to one line
cutcommand (Examples) to get the MAC address. Since we know that the MAC address has fixed length, feel free to count the indices.
- The final shell script only has to have one line, although a answer with multiple lines are also acceptable.
- First figure out how to use
Go to Gradescope for submission.