Lab 10 – Looking Down from Space

Description

Welcome!

In today’s lab, you will:

1. 1. Get first hand experience implementing threads and concurrency

2. Learn how to use Locks to ensure Mutual Exclusion when accessing resources shared across threads.

2. Write code implementing the producer and consumer pattern

Grading

This lab will be marked out of 10.

For full marks this week, you must:

1. (2 point) Commit and push to GitHub after each non-trivial change to your code

2. (4 points) Successfully implement the requirements as described in this document

3. (2 points) Implement the Producer-Consumer pattern correctly

4. (2 points) Write code that is consistently commented and formatted correctly using good variable names, efficient design choices, atomic functions, constants instead of magic numbers, etc.

Requirements

Clone‌‌your‌‌repo‌‌using‌‌github‌‌classroom:‌htps://classroom.github.com/a/kgo0u4mR

In today’s lab we are going to tackle the famous producer-consumer problem. In this problem, one or more threads are known as producers, that is they provide data to a common buffer (in our case, a queue)

And as you may expect there are one or more threads knowns as consumers, who read and remove data from this common buffer and do something with it

This is an extremely simple problem and a great way to get your hands dirty with writing multithreaded code.

In today’s lab we will be getting data from a singleton called ISSDataRequest. Calling this singleton simulates retrieving data from an external source that introduces some time delay.

In this lab, you will be loading data about city locations (latitude and longitude) from locations.txt.

Step 1:

The first thing, clone the repo. I have provided several data structures.

• Location

This data structure represents a single location. It stores the latitude and longitude of a city.

• CityInfo

This is a data structure that represents all the information about a city

• ISSDataRequest

This singleton accepts a latitude and a longitude as its parameters and returns city information (CityInfo)

Write some code to test out and ensure that your code works. Use the locations_test.txt file for test data. Extract location objects from the test data, and pass them into ISSDataRequest to get the corresponding CityInfo objects for each city.

You are not allowed to modify any of the provided code in the ISSDataRequest class

Step 2: Creating our Buffer

The first step in our producer-consumer problem is to create a buffer that will hold the extracted CityInfo objects. We will use a Queue for this. Examine the CityInfoQueue class.

• dataQueue

Shared vector of CityInfos

• accessQueueMutex

Mutex to protect dataQueue access

• dataIncoming

Boolean to indicate if more data is being added to the data_queue. This attribute should change to False after the producer threads have joined the main thread and finished processing all the cities.

• void put(CityInfo) -> None:

This method is responsible for adding to the queue. Accept a CityInfo parameter and append it to the dataQueue list.

• CityInfo get() -> CityInfo:

This method is responsible for removing an element from a Queue. Remember a queue is a FIFO data structure, that is it is First In First Out. Each call to this method should return the element at index 0 and delete it from the vector.

Step 3: Creating a Producer and a Consumer Thread

We now need 2 classes that our future threads will call. Examine the following classes:

1. Producer

The Producer class has the following methods:

1. • Producer(vector<Location>locations, CityInfoQueue &queue)

queue is shared between the Producer and Consumer

1. • void operator()():

This method is called automatically when we attach this object to a thread. It should loop over each Location and pass it to the ISSDataRequest.get_city_info() method. It then proceeds to add the CityInfo to the queue. After reading in 5 cities, the thread should sleep for 1 second.

At this point, write some code in the main method to create threads with the Producer objects and join them to ensure this thread works as intended.

2. Consumer

The Consumer is responsible for consuming data from the queue and printing it out to the console. It has the following methods:

• Consumer(CityInfoQueue &q):

Initializes the ConsumerThread with the same queue as the one the Producer has.

• Void operator()():

While the queue’s dataIncoming is true OR the queue is not empty, this method should get an item from the queue and print it to the console and then sleep for 0.5 seconds. If the queue is empty while processing, put the thread to sleep for 0.75 seconds.

Now write some more code in the main method to create a consumer thread and make sure it works as intended

Step 4: Adding locks

threads. Since the operating system can interrupt a thread and switch between them randomly, the queue may not be in sync. To ensure that we are accessing shared variables and resources safely we need to implement Locks.

Locks allow us to define areas of code for Mutual Exclusion. This means that only one thread can acquire the lock and access that block of code at a time. Mutual exclusion ensures that only one thread is modifying the queue at any given time. This avoids Race Conditions.

Race conditions are what happens when Locks are not implemented. These are situations where the order in which instructions are executed gets mixed up due to multiple switching threads and this causes errors.

Examine the CityInfoQueue class and find the accessQueueLock. Use this lock to control access to the code in the put and get methods. Is there anywhere else the dataQueue is accessed and needs protection? Test and run the code again to ensure it works as expected

You may want to use locations_test.txt for testing so you don’t spend a lot of time waiting to process 100+ cities. You can add more cities to this txt file to gradually increase the amount of cities as well.

Step 5: Adding more producers

The final step is to create 3 or more producer threads and at least 2 consumers. Split the locations across the producer threads and start them. This should speed up your code and requests significantly.

• Ensure you push your work to github classroom. I’d like to see sensible git commits comments, and commits must take place at logical points in development.

That’s it for this lab!

Sample output when running the completed program with the included locations_test.txt file. There are 3 producer and 2 consumer threads:

Consumer 1 is sleeping since queue is empty

Consumer 2 is sleeping since queue is empty

Producer 2 is adding to the queue

Producer 3 is adding to the queue

Element added to queue! Queue has 1 elements Producer 1 is adding to the queue

Element added to queue! Queue has 2 elements

Element added to queue! Queue has 3 elements

Producer 1 is adding to the queue

Element added to queue! Queue has 4 elements

Producer 3 is adding to the queue

Producer 2 is adding to the queue

Element added to queue! Queue has 5 elements

Element added to queue! Queue has 6 elements

Producer 3 is adding to the queue

Element added to queue! Queue has 7 elements

Consumer 2 is consuming from the queue

Consumer 1 is consuming from the queue

Element removed from queue! Queue has 6 elements left

latitude: 45.500000

longitude: -73.583300

timezoneId: America/Toronto

offset: -4

Element removed from queue! Queue has 5 elements left

countryCode: CA

latitude: 54.130100

longitude: -108.434700

timezoneId: America/Swift_Current

offset: -6

countryCode: CA

mapUrl: https://maps.google.com/maps?q=54.1301,-108.4347&z=4

mapUrl: https://maps.google.com/maps?q=45.5,-73.5833&z=4

Consumer Consumer 1 is consuming from the queue 2 is consuming from the queue

Element removed from queue! Queue has 4 elements left

latitude: 50.017100

longitude: -125.250000

timezoneId: America/Vancouver

offset: -7

countryCode: CA

mapUrl: https://maps.google.com/maps?q=50.0171,-125.25&z=4

Element removed from queue! Queue has 3 elements left

latitude: 62.442000

longitude: -114.397000

timezoneId: America/Yellowknife

offset: -6

countryCode: CA

mapUrl: https://maps.google.com/maps?q=62.442,-114.397&z=4

Consumer 1 is consuming from the queue

Element removed from queue! Queue has 2 elements left latitude: 49.273400

longitude: -123.121600

timezoneId: America/Vancouver

offset: -7

countryCode: CA

mapUrl: https://maps.google.com/maps?q=49.2734,-123.1216&z=4

Consumer 2 is consuming from the queue

Element removed from queue! Queue has 1 elements left

latitude: 53.016700

longitude: -112.816600

timezoneId: America/Edmonton

offset: -6

countryCode: CA

mapUrl: https://maps.google.com/maps?q=53.0167,-112.8166&z=4

Consumer 1 is consuming from the queue

Element removed from queue! Queue has 0 elements left

latitude: 51.000500

longitude: -118.183300

timezoneId: America/Vancouver

offset: -7

countryCode: CA

mapUrl: https://maps.google.com/maps?q=51.0005,-118.1833&z=4