@import "./support/support"; @import "./color_schemes/"; @import "./modules"; @import "./custom/custom";
Link
Sysadmin DeCal

Lab a1 - Shell Scripting

Facilitator: Ben Cuan

7 min read

Table of contents

  1. A Note on Labs
    1. Workflow
  2. Question 1
  3. Question 2
  4. Question 3
  5. Question 4 (Extra)
  6. Submission
  7. Footnotes

A Note on Labs

Labs are graded on completion. As long as you give your best effort and submit an answer to each question, you will receive full credit. Treat them as seeds of exploration instead of just a grade.

As this is the first lab, we have done our best to make it relatively straightforward. (If something seems overly difficult, there is probably a simpler way to do it!) Bash scripting isn’t the main goal of the DeCal but this lab should introduce you to some fun bash features you may not have encountered before, such as loops and shell expansions, that you’ll probably find useful in the future.

If you ever find yourself confused, stuck, and/or curious to learn more, talk to us about it! The best way to connect with us (and your peers) is through Piazza. You can also join our Discord channel or Slack channel.

Workflow

This lab can be done on your own UNIX-like machine, or you can ssh into tsunami.ocf.berkeley.edu using your OCF account to finish the lab there. As always, man and Google will be your friends.

If you’d like to test your scripts for correctness, feel free to run the provided examples or make some of your own! Since labs are graded on completion, there are no autograder tests or anything of the sort to worry about (which will also be true for the following labs). We will release sample solutions after the lab is due, but keep in mind that there are many ways to solve these problems.

Question 1

At some point, everyone has looked at a problem and thought to themselves: “Hey, I can do this in one line!”

Lets find out if you can. I need to sort out some of my most listened to albums by making directories for each of them, specifically for my favorite artist Future.

I have hosted a list of my favorite albums followed by their respective artist at https://raw.githubusercontent.com/0xcf/decal-labs/master/a1/albums.txt, in a comma delimited format like Die Lit, Playboi Carti. For the GOAT artist Future, I want to create a folder for each of his albums. For example, for an entry like SUPER SLIMEY, Future I would expect a directory called SUPER SLIMEY to be created.

TLDR: You need to fetch the list from the web, filter out the albums we want, trim out the album name, and then make a directory for each one, all in one line.

$ cat albums.txt
...
Drip or Drown 2, Gunna
Playboi Carti, Playboi Carti
DS2 (Deluxe), Future  <-- GOAT album detected!
Drip Harder, Lil Baby
The WIZRD, Future  <-- GOAT album detected!
What a Time To Be Alive, Drake
...

# After our magic one liner...

$ ls
'DS2 (Deluxe)'   'The WIZRD'   ...
# We got our new directories!

Hints:

  • What common text manipulation commands can help you solve this?
  • As always, be aware that there isn’t one unique solution to this problem!
  • Also be aware that xargs behaves differently on different platforms.

Submit your one line solution on Gradescope!

Question 2

I like Lisp and Scheme, and miss car and cdr in my usual programming tasks.1

In bash, implement car and cdr (aka head and tail) such that they operate on file paths.

e.g.

$ ./car /home/a/ab/abizer/some/path
home
$ ./cdr /home/a/ab/abizer/some/path
a/ab/abizer/some/path

You may assume that only absolute paths2 will be given.

Hint: There’s no need to use complicated string manipulations such as regex’s for this task. The easiest way to do this is with one very short command.

As an optional bonus challenge: generalize this solution to work for cadr, caddr, etc.

$ ./cadr /home/a/ab/abizer/some/path
a
$ ./cddr /home/a/ab/abizer/some/path
ab/abizer/some/path

Question 3

With the invention of the .norm file format, file extension innovation is at its peak!3

However, your computer is old and doesn’t support it, so we’ll need to convert all of the files ending in .norm into .docx files.

Using Bash functions and shell wildcard expansion, write a shell script rename.sh to batch rename file extensions in a particular directory.

Here is some more specific info about this function:

  • It should take in 3 arguments: the directory, the original extension, and the new extension.
  • It should print the line renaming <old file> to <new file> for each renamed file.
  • It should not modify any files in the directory that do not have the specified extension.

Example:

$ ls Documents/
cats.norm data.norm dogs.norm ...
$ ./rename Documents norm docx # Run your script!
renaming Documents/data.norm to Documents/data.docx
renaming Documents/cats.norm to Documents/cats.docx
renaming Documents/dogs.norm to Documents/dogs.docx
...
$ ls Documents/
cats.docx data.docx dogs.docx ...

Your script should be able to convert between any arbitrary file formats, not just .norm and .docx! For example:

$ ls
# Creates a new directory tmp and adds 26 new files a.dat, b.dat ... to z.dat into it
$ mkdir tmp && touch tmp/{a..z}.dat 
$ ./rename.sh tmp dat txt
renaming tmp/a.dat to tmp/a.txt
... # 24 more lines
renaming tmp/z.dat to tmp/z.txt
$ ls -lAh tmp | grep .txt | wc -l # Gets the number of lines in ls which contain .txt
26

for bonus points, instead of using something like sed to affect the rename, use shell parameter expansion.

Question 4 (Extra)

This question is optional but it’s quite fun and you should do it if you have the time!

Using Bash functions, write a script mkrandom.sh that generates a user-specified number of files of user-specified size filled with random content.

e.g.

$ ./mkrandom.sh 10 100  # create 10 100 byte random files
$ ls -lAh
total 44K
-rw-r--r-- 1 abizer ocf  100 Sep 16 21:57 1
-rw-r--r-- 1 abizer ocf  100 Sep 16 21:57 10
-rw-r--r-- 1 abizer ocf  100 Sep 16 21:57 2
-rw-r--r-- 1 abizer ocf  100 Sep 16 21:57 3
-rw-r--r-- 1 abizer ocf  100 Sep 16 21:57 4
-rw-r--r-- 1 abizer ocf  100 Sep 16 21:57 5
-rw-r--r-- 1 abizer ocf  100 Sep 16 21:57 6
-rw-r--r-- 1 abizer ocf  100 Sep 16 21:57 7
-rw-r--r-- 1 abizer ocf  100 Sep 16 21:57 8
-rw-r--r-- 1 abizer ocf  100 Sep 16 21:57 9
-rwxr-xr-x 1 abizer ocf  147 Sep 16 21:56 mkrandom

Submission

Submit your solutions on Gradescope! There’ll be some extra feedback questions as well that we would appreciate you filling out.

Footnotes

You may want to look into dd4 and the iflag=fullblock argument, seq, and /dev/random5.

  1. Aren’t too familiar with car and cdr? Here’s a brief article about it.. If you take CS61A, you’ll see it there as well! ↩

  2. As a quick reminder, absolute paths always start from the root directory (/), whereas relative paths start from the current directory. ↩

  3. Relevant xkcd: https://xkcd.com/2116/ ↩

  4. dd is a command used to copy files.6 It’s most commonly used to clone data from one device to another, such as when you want to generate a bootable Linux USB drive. ↩

  5. A curious individual might find the device file /dev/urandom as well. What’s the difference? True randomness is a rather difficult problem for computers, as they’re expected to do the same thing given the same state, so they pull in random data from metrics like internal temperature and mouse movement. Unfortunately, such entropy may not exist in certain machines and gathering entropy may be prohibitively long. Thus, /dev/urandom, or “unlimited random”, is a useful source when such randomness is not cryptographically critical. ↩

  6. “But wait,” a nearby straw-man asks, “isn’t that what cp does?”7 ↩

  7. They are indeed right, but dd has some useful features such as partial writing and reading that make it handy in weirder scenarios, such as devices. StackOverflow has a good explainer and the ArchWiki has some common examples. ↩