Description
-
Introduction
As part of this assignment, you will be implementing system calls in a teaching OS (gemOS), some of which you became familiar during Assignment-1. We will be using a minimal OS called gemOS to implement these system calls and provide system call APIs to the user space. The gemOS source can be found in the src directory. This source provides the OS source code and the user space process (i.e., init process) code (in src/user directory).
gemOS is a teaching operating system. Typically OS boots on a hardware (bare metal or virtual). We will be using gem5 (http://gem5.org/Main Page), an open source architectural simulator to boot gemOS. An architectural simulator simulates the hardware in software. In other words, all the hardware functionalities are implemented in software using some programming language. For example, Gem5 simulator implements architectural elements for di erent architectures like ARM, X86, MIPS etc. Advantages of using software simulators for OS development are,
Internal hardware state and operations at di erent components (e.g., decoder, cache etc.) can be pro led to better understand the hardware-software interfacing.
Hardware can be modi ed for several purposes|make it simpler, understand implica-tions of hardware changes on software design etc.
Bugs during OS development can be better understood by debugging both hardware code and the OS code.
The getting started guide helps you to setup Gem5 and execute gemOS on it. You can refer to this online tutorial to know more about gem5.
You need to setup gem5 by following the instructions (Step-0) mentioned in Section 2. We suggest to complete Step-0 immediately to have a working gem5 simulator to start with the assignment. Step-0 also helps you to understand how to boot gemOS binary in gem5. You can build the gemOS source provided in the src directory to test the working.
The assignment is divided into ve tasks. The implementation of system calls for each task should be POSIX compliant. Which means, you need to read the man pages of each of corresponding system calls to know about their exact behavior. Testing procedure and submission guidelines are mentioned at the end of the document.
1
This section explains the setup procedure of gem5. Further, it explains the process to build and execute gemOS on gem5 platform and access the terminal to see the messages printed by the gemOS.
2.1 Preparing Gem5 simulator
System pre-requisites
Git
gcc 4.8+
python 2.7+ SCons
protobuf 2.1+
On Ubuntu install the packages by executing the following command.
-
sudo apt-get install build-essential git m4 scons zlib1g zlib1g-dev
libprotobuf-dev protobuf-compiler libprotoc-dev libgoogle-perftools-dev python-dev python automake
Gem5 installation
Clone the gem5 repository from https://gem5.googlesource.com/public/gem5.
$ git clone https://gem5.googlesource.com/public/gem5
Change the current directory to gem5 and build it with scons:
$ cd gem5
$ scons build/X86/gem5.opt -j9
In place of 9 in [-j9], you can use a number equal to available cores in your system plus one. For example, if your system have 4 cores, then substitute -j9 with -j5. For rst time it will take around 10 to 30 minutes depending on your system con guration. After a successful Gem5 build, test it using the following command,
-
build/X86/gem5.opt configs/example/se.py –cmd=tests/test-progs/hello/bin/x86/linux/hello
The output should be as follows,
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the –copyright option for details.
gem5 compiled Aug 4 2018 11:00:44
gem5 started Aug 4 2018 17:15:06
2
gem5 executing on BM1AF-BP1AF-BM6AF, pid 8965
command line: build/X86/gem5.opt configs/example/se.py \
–cmd=tests/test-progs/hello/bin/x86/linux/hello
/home/user/workspace/gem5/configs/common/CacheConfig.py:50: SyntaxWarning: import * only \ allowed at module level def config_cache(options, system):
Global frequency set at 1000000000000 ticks per second
warn: DRAM device capacity (8192 Mbytes) does not match the address range assigned (512 Mbytes)
-
system.remote_gdb: listening for remote gdb on port 7000
**** REAL SIMULATION ****
info: Entering event queue @ 0. Starting simulation…..
Hello world!
Exiting @ tick 5941500 because exiting with last active thread context
2.2 Booting gemOS using Gem5
Gem5 execute in two modes|system call emulation (SE) mode and full system (FS) sim-ulation mode. The example shown in the previous section, was a SE mode simulation of Gem5 to execute an application. As we want to execute an OS, Gem5 should be executed in FS mode. There are some initial setup to do before we can execute gemOS using Gem5 FS mode. To run OS in full-system mode, where we are required to simulate the hardware in detail, we need to provide the following les,
gemOS.kernel: OS binary built from the gemOS source.
gemOS.img: root disk image
swap.img: swap disk image
Gem5 is required to be properly con gured to execute the gemOS kernel. The con guration requires changing some existing con guration les (in gem5 directory) as follows,
Edit the configs/common/FSConfig.py le to modify the makeX86System function where the value of disk2.childImage is modi ed to (disk(’swap.img’)).
Edit the configs/common/Benchmarks.py le to update it as follows,
elif buildEnv[’TARGET_ISA’] == ’x86’:
return env.get(’LINUX_IMAGE’, disk(’gemOS.img’))
Create a directory named gemos in gem5 directory and populate it as follows,
/home/user/gem5$ mkdir gemos
/home/user/gem5$ cd gemos
/home/user/gem5/gemos$ mkdir disks; mkdir binaries
/home/user/gem5/gemos$ dd if=/dev/zero of=disks/gemOS.img bs=1M count=128 /home/user/gem5/gemos$ dd if=/dev/zero of=disks/swap.img bs=1M count=32
3
For the time being, you can use gemOS.kernel provided with the assignment (can be found in src directory). Copy the gemOS.kernel to gemos/binaries directory.
We need to set the M5 PATH environment variable to the gemos directory path as follows,
/home/user/gem5$ export M5 PATH=/home/user/gem5/gemos
Now, we are ready to boot GemOS.
gem5$ build/X86/gem5.opt configs/example/fs.py
–kernel=/home/user/gem5/gemos/binaries/gemOS.kernel –mem-size=2048MB gem5 output will look as follows,
gem5 Simulator System. http://gem5.org
gem5 is copyrighted software; use the –copyright option for details.
gem5 compiled Aug 21 2019 23:45:13
gem5 started Aug 22 2019 10:45:01
gem5 executing on kparun-BM1AF-BP1AF-BM6AF, pid 28942
command line: build/X86/gem5.opt configs/example/fs.py –kernel=/home/kparun/gem5/gemos/binaries/gemOS.kernel –mem-size=2048MB
Global frequency set at 1000000000000 ticks per second
warn: DRAM device capacity (8192 Mbytes) does not match the address range assigned (512 Mbytes)
info: kernel located at: /home/kparun/gem5/gemos/binaries/gemOS.kernel
system.pc.com_1.device: Listening for connections on port 3456
-
-
rtc: Real-time clock set to Sun Jan 1 00:00:00 2012
-
-
system.remote_gdb: listening for remote gdb on port 7000 warn: Reading current count from inactive timer.
**** REAL SIMULATION ****
info: Entering event queue @ 0. Starting simulation…
warn: Don’t know what interrupt to clear for console.
Execute the following command in another terminal window to access the gemOS console
/home/user$ telnet localhost 3456
At this point, you should be able to see the gemOS shell.
2.3 How to build gemOS
To build gemOS.kernel, you need to run make inside src folder. After that you need to copy gemOS.kernel binary to gemos/binaries directory. This step is necessary every time you build the gemOS and want to test it. After copying gemOS.kernel, run
gem5$ build/X86/gem5.opt configs/example/fs.py
–kernel=/home/user/gem5/gemos/binaries/gemOS.kernel –mem-size=2048MB
Open a terminal window as before and access the console using the following command,
/home/user$ telnet localhost 3456.
2.4 How to test your Implementation
In the GemOS# terminal (accessed using the telnet command as shown above), you can type init to execute the user space process i.e, init. The user space code is available in
4
src/user/init.c. Three user space les are used to implement the user space logic. They are
init.c: Implements the rst user space process which can invoke fork() to create more pro-cesses. Note that, there is no exec system call yet in the version provided to you. For changing the user space logic, you are required to modify only init.c.
lib.c: Implements system call wrappers and provide di erent user space libraries (e.g., printf).
Note that you do not modify this le.
lib.c: Provides declarations of macros and functions. Note that you do not modify this le.
You need to write your test cases in init.c to validate your implementation. The sample test-cases (in src/user/test cases/testcase*.c) can be copied into init.c to make use of them. If your implementation is correct, the output of executing test cases should match the expected output provided in src/user/test cases/testcase*.output. The user and kernel code are compiled into a single binary le, i.e., gemOS.kernel when built using make from the src directory.
The real assignment
The process control block (PCB) is implemented using a structure named exec context de ned in src/include/context.h. One of the important member of exec context for this assignment is an array of struct file (declared in include/file.h) named as files. This is the le descriptor table where the index of the array (in files) corresponds to the le descriptor. For example, files[0] represents the le descriptor 0 and points to the le object for the standard input (STDIN). You are required to manipulate the files structure and provide appropriate logic for the le object to implements the assignment. The template code provides detailed documentation for understanding the tasks further. Note that, there are several function pointers in struct file which can be implemented to provide le system functionalities as we discuss further.
-
Task-1: Basic le operations (20 Marks)
List of Syscalls to Implement
int open(const char *pathname, int flags,int mode) int read(int fd, void *buf, int count)
int write(int fd, const void *buf,int count)
3.1 open
To implement open system call, you are required to provide implementation for the template function do regular file open (in file.c) which takes the current context, lename, ags
5
and mode as arguments. Open call can be used to open an existing le or create a new one by passing the O CREAT ag as per the POSIX semantics. For regular les, an underlying inode is provided through the FS APIs which you are required to invoke.
While creating a le, the rst step is to get an inode from the underlying FS (File System) layer by invoking create inode (implemented in fs.c). The signature of create inode is as follows,
struct inode *create inode (char *filename, u64 mode)
where filename and mode should be same as it is passed to the do regular file open function. The mode can take O READ, O WRITE, O EXEC values which corresponds to Read, Writ and Execute permissions (passed by the user). Permission check is performed on read/write access based on mode value, i.e., write call on a le which is created with O READ mode should return an EACCES error.
Now let us look at the second scenario of opening an existing le. The rst step here is look up the inode corresponding to the lename from the underlying FS layer by invoking lookup inode (in fs.c). The signature of lookup inode is
struct inode* lookup inode(char *filename).
A valid inode is returned on success (NULL on error) and you need to ensure that the access ags mentioned in open are compatible with the mode in which le was created. After getting the inode from the FS layer, you need to nd a free le descriptor, allocate a le object (using alloc file method in file.c) and ll-in the elds of corresponding struct le object which is pointed to by les (in context.h) eld of current execution context. Here you need to look for a free position in les array starting from index 3. Index positions 0, 1, 2 corresponds to stdin, stdout, stderr. You need to implement do regular read, do regular write and std close functions and assign them to read, write and close function pointers of struct fileops by accessing fops eld in the struct le. As last step of open call, you need to return the le descriptor which is returned back to the user and used for subsequent le operations.
The implementation of le objects and operations for STDIN, STDOUT and STDERR are already provided to help you with the understanding of the task.
3.2 read
You need to implement the do read regular function (in file.c). This function is to be assigned as the read handler in the le object while opening the le. The inode provides a read method (flat read) with the following signature
int flat read(struct inode *, char *buf, int count, int *offset).
where, buf and count are the user bu er and count, respectively, passed to do read regular from the read system call handler. The above function returns the number of bytes read from the underlying le. Read implementation for STDIN, do read kbd, is provided in file.c as an illustration.
6
You need to implement the do write regular function (in file.c). This function is to be assigned as the write handler in the le object while opening the le. The inode provides a write method (flat read) with the following signature
int flat write(struct inode *, char *buf, int count, int *offset).
where, buf and count are the user bu er and count, respectively, passed to do write regular from the read system call handler. The above function returns the number of bytes written to the underlying le. Write implementation for stdout/stderr, do write console, is provided in file.c.
Notes
You are required to modify only file.c.
Sample test case and expected output for this task are in src/user/test cases/testcase1.c and src/user/test cases/testcase1.output, respectively.
Refer to the section detailing the test procedure to know about the limits and assump-tions related to this task.
-
Task-2: More le operations (15 Marks)
List of Syscalls to implement
int dup(int oldfd)
int dup2(int oldfd, int newfd)
long lseek(int fd, long offset, int whence)
4.1 dup
You have to implement fd dup function (in le.c). It takes current execution context and oldfd as arguments. You need to return error codes (in entry.h) based on the error conditions as explained in the section detailing the error codes.
4.2 dup2
You have to implement fd dup2 function (in le.c). It takes current execution context, oldfd and newfd. Before making newfd as a copy of oldfd, you need to close newfd if it is open.
7
You need to implement do lseek regular (in le.c). It takes pointer to struct file, offset and whence as arguments. You need to implement the functionality for three whence options SEEK SET, SEEK CUR, SEEK END (in le.h). You need to return error codes (in en-try.h) based on the error conditions. Note that, if lseek results in taking the le o set beyond the le end, you need to return error code EINVAL.
Notes
You are required to modify only file.c.
Sample test case and expected output for this task are in src/user/test cases/testcase2.c and src/user/test cases/testcase2.output, respectively.
Refer to the section detailing the test procedure to know about the limits and assump-tions related to this task.
-
Task-3: Pipe it! (15 Marks)
List of Syscalls to implement
int pipe(int fd[2])
5.1 pipe
You need to implement create pipe, pipe read and pipe write functions (in pipe.c). create pipe, invoked to implement pipe system call, takes pointer to current execution context and le descriptor array (of two elements) as arguments. File descriptors for read and write ends of pipe are assigned by looking for available indices in files array of the current context. You can use alloc file API (in le.c) to allocate a file objects and associate it with read and write end of the pipe. Use alloc pipe info (in pipe.c) to get a pointer to a pipe object (struct pipe info) and attach it with the pipe eld of the file object. Note that, in struct pipe info, pipe buf can be used to implement data write and read. You should ll-in the elds of struct pipe info in alloc pipe info function. You need to implement pipe read and pipe write functions and assign them to read and write function pointers of struct fileops by accessing fops eld in struct le.
Notes
You are required to modify only pipe.c.
Sample test case and expected output for this task are in src/user/test cases/testcase3.c and src/user/test cases/testcase3.output, respectively.
Refer to the section detailing the test procedure to know about the limits and assump-tions related to this task.
8
(10 Marks)
List of functionalities to implement
System call int close(fd)
Handler for process exit void do file exit(struct exec context *ctx in file.c
Handler for process creation through fork void do file fork(struct exec context *child in file.c
Details on Implementation
You have to implement generic close (in le.c) as part of this task. Note that, the generic close implements closing regular les and pipes. You need to ensure that the reference count in the le object associated with the fd is maintained correctly. When the last reference to the le object is dropped, you need to invoke free file object(stuct file *) and free pipe object(struct pipe info *) as applicable. As a program may exit without closing the les, you need to perform le close on exit system call by appropriately im-plementing do file exit. When a child process is created using fork, the le objects are shared, as the FDs in les are already copied while creating the child process. You are required to adjust the reference counts as applicable (in do file fork).
Notes
You are required to modify only file.c.
Sample test case and expected output for this task are in src/user/test cases/testcase4.c and src/user/test cases/testcase4.output, respectively.
Refer to the section detailing the test procedure to know about the limits and assump-tions related to this task.
-
Task-5: Putting it all together!(40 Marks)
As part of this task, your implementation will be tested for all syscall APIs that you have implemented in Task 1-4. You need to verify that your implementation works in a holistic manner.
Notes
You may be required to modify file.c and pipe.c if your rst-cut logic is incorrect. If you have got everything correct, Bravo!
9
Sample test case and expected output for this task are in src/user/test cases/testcase5.c and src/user/test cases/testcase5.output, respectively.
Refer to the section detailing the test procedure to know about the limits and assump-tions related to this task.
Error codes
You should only use following error codes to mention error conditions. All these error codes should be negated before returning (Example: EINVAL should be returned as -EINVAL).
EINVAL(Invalid Argument) It should be used in-case of invalid argument such as lename does not exist, invalid le descriptor, accessing closed le or pipe.
EACCES(Invalid Access) It should be used in-case of invalid access such as writing to read-only le or pipe etc
ENOMEM(No Memory) It should be used if memory allocation function used to allocate le, pipe info fails.
EOTHERS(Others) In case of any other errors which is not speci ed above use
EOTHERS.
Test Procedure
We have provided you with ve test cases(test cases folder) to test Tasks 1-5. Apart from Task-5, We will be testing each Task individually and won’t be mixing it up with other Tasks. We will be following below assumptions when we are testing your code. So lets go thorough all the assumptions
There will be at-most four process that will be running at any point of time. No need to fork more than 4 process.
There can be at-most 16 les of each 4KB size at any point of time.
There can be at-most 16 le descriptor which can be created using dup or dup2 system calls.
The max length of pipe bu er will be 4KB. We wont read or write more than 4KB into the pipe.
We will be testing the error conditions using standard error codes which is speci ed above. Don’t forget to negate the error code before returning.
Don’t try to create or allocate memory by yourself. Try to use the speci ed function. In-case of any issues reach out to us.
10
Don’t modify any other function or le. We will be evaluating the changes in the les ( le.c and pipe.c)
You need not worry about concurrency as all accesses are guaranteed to be performed from a single process.
Submission guidelines
The assignment is to be done individually. You have to submit only two les( le.c and pipe.c). Put these two les in a directory named as your roll number. Create a zip archive of the directory and upload in canvas. Don’t modify any other les. We will not consider any le other than le.c and pipe.c for evaluation. In-case any issues you should reach out to us at the earliest. All the best!
11
