Regex Solution

Description

Objectives

• Use Regex with common UNIX/Linux commands

• Practice using useful UNIX/Linux commands (remember man can help you understand how each command operates!)

• 15. diff o grep o cut o uniq o sed o awk

• Practice creating and running bash shell scripts

• Practice using pipes

Lab 2 Exercise

For each step, record the commands (and options) that you used to complete the task in a file called Lab2_Solutions.txt. At the end you will receive credit for the lab by showing your TA these commands.

Download Practice Files From Moodle

For Today’s lab we will be using the following data files:

• scene1_v1.txt

• scene2_v2.txt

• password_demo.txt

• grades.txt

• cryptic.txt

• regex_practice_data.txt

Download Lab2 from Moodle and copy the provided zip file to your home directory and decompress it using the unzip command:

unzip Lab2.zip -d Lab2

cd Lab2

Check to make sure each of the above files was correctly unzipped into the Lab2 directory.

Part 1: Use Unix Commands to compare Monty Python Scripts!

Step 1: – Use the diff command

1. Use diff to display all the lines that have changed from scene1_v1.txt to get to scene2_v2.txt.

2. For the original diff output, what do the ‘>’ or ‘<’ character mean at the beginning of each line?

3. Try using the –c option, what does that do?

For the Steps 2 – 4, we will have 2 problems to solve. The first will demonstrate a standard use case for a specific unix command (grep, cut, uniq). The second problem utilizes piping to build a complex unix statement that will eventually use all of the unix commands at once!

Step 2: – Use the grep command

1. Use grep to display each line that contains the word “pigeon”, as well as its line number, in scene1_v1.txt

2. Use grep to display the lines that were modified in scene1_v1.txt?

(Hint: Pipe the output from your first diff command into the grep command)

Step 3: – Use the cut command

1. Using the delimiter ‘:’, display the name of the characters who are speaking in scene1_v1.txt (make sure to ignore any lines that do not include the delimiter).

2. Now use cut to only display the name of the characters that have had their lines altered from scene1_v1.txt to scene1_v2.txt.

Step 4: – Use the uniq command

1. Use the uniq command to list only the duplicate lines in scene1_v1.txt.

2. Use uniq to show how many times each character has had their lines altered from scene1_v1.txt to scene1_v2.txt.

**As a note on uniq, only compares adjacent lines. To find all repeated lines, the text must be sorted**

Part 2: Working with Regular Expressions & AWK

Step 5: – Use the sed command

1. Using sed and regular expressions try playing around with cryptic.txt file.

1. 1. Remove all the letters

1. 2. Replace all numbers with an ‘_’

2. Using pipes, create a script that pipes together multiple sed commands to replace each number with its matching character. How can this be done without piping? LEET Alphabet Used:

Step 6: – More practice with regular expressions

For the following problems use grep or egrep with the regex_practice_data.txt file.

1. How many phone numbers are in the dataset?

2. How many city of Boulder phone numbers (e.g. starting with 303-441-…)?

Step 7: – Use the awk command

pizzaOrders.txt Column Descriptions:

ID – Order IDentification Number

TP – Total Number of Pizzas Ordered

NP – Number of Pepperoni Pizzas Ordered ($5.50) NS – Number of Sausage Pizzas Ordered ($5.75) NC – Number of Cheese Pizzas Ordered ($5.00) TC – Total Cost

1. Using pizzaOrders.txt, print out the average cost per pizza for each order.

2. Using pizzaOrders.txt, calculate and print the percent of all pizzas sold that were cheese.

Credit: To get credit for this lab exercise, get your Lab2_Solutions.txt file checked with your TA and submit the file on Moodle. All partners should submit copies of the same file.