CS647 Assignment 1 (tinyblob): Perform and measure I/O on different devices

 

The goal of this assignment is to write a simple library that performs I/Os on files or block devices. The library is essentially a very simple store (tinyblob) that we will extent in subsequent assignments.  Although simple, the assignment will help us realize several aspects about device technology and a storage systems.

 

The assignment

 

1. The library

 

Implement a tinyblob library that performs accesses to fixed (but configurable) size blobs. For us, blobs are extents with size multiple of the (device) block size.  We will implement blobs in two ways: (a) using a separate file for each blob and (b) as continuous extents over a physical block device. The operations we will provide on blobs are: create, delete, read, and write full blobs. Our library will allow parameters to specify at initialization time the size of blobs (one size for all blocks in the store) and the maximum number of blobs in the store.

 

There are two versions of the library (either separate or configurable): (a) one that creates files in a directory, with each blob being a separate file and (b) one that allocates blocks over a block device of approximately [0, max*blob_size].  For both versions you can use an API, such as: 

 

·       bid_t tb_allocate_blob(void); // On success it returns the index/id of the allocated blob in the store. On failure it returns -1; When using files you can use a naming scheme “Bxxx” to name each blob file.

·       void tb_free_blob(bid_t); // Given a valid index from a successful allocate_blob call, it frees the blob.

·       int tb_write_blob(bid_t, void *data); // Perform a synchronous write (using issue_write() or otherwise) from the data buffer. Return success or failure.

·       int tb_read_blob(bid_t, void *data); // Perform a synchronous read (using issue_read() or otherwise) to the data buffer. Return success or failure.

 

·       (extra credit) iohandle_t tb_write_blob_issue(bid_t, void *data); // Issue an asynchronous a write operation (of the full blob) from the data buffer to blob_index. Return a handle to identify the I/O.

·       (extra credit) iohandle_t tb_read_blob_issue(bid_t, void *data) // Issue an asynchronous read  (of the full blob)  to blob_index a in buffer data (data must be a valid, preallocated buffer). Return a handle to identify the I/O.

·       (extra credit) void tb_wait_io(iohandle_t); // Wait for the asynchronous id to be performed.

 

·       void tb_flush(); // Flush all data and metadata to the disk.  NOTE! Be careful when you try to persist structs; Consider the use of add __attribute__((__packed__)), otherwise the compiler may add padding for alignment purposes (see https://gcc.gnu.org/onlinedocs/gcc-3.3/gcc/Type-Attributes.html).

·       void tb_init(char *location, size_t bsz, size_t nblobs); // Given a directory or a file, it reads the library state in memory and is able to perform I/O to the store. If the store does not exist, then create one with blob size bsz and maximum number of blocks nblobs .

·       void tb_shutdown(void); // Clean shutdown of the store; Flush all data and metadata so that it is possible to start from the same files/device.

 

Metadata: The file version of the store will probably require: a structure to maintain allocated files (one per blob), a structure for each blob to keep the required information, a structure to maintain pending I/Os (if you provide asynchronous I/O support), and a mutex.  The block version of the store will require in addition a free list (bitmap) for free blobs, since there is no filesystem now to keep track of the free device space. 

 

Implementation: Although the API above is given in C pseudo-form, you may use either C or C++. C++ has the benefit that if you wish to replace a standard library structure with another version, e.g. with a different memory/performance tradeoff, it will be easier to do so.

 

2. Driver program 

 

Implement a driver program that will create a blob store of configurable size, call the I/O library to perform sequential and random IO (over full blobs) with a variable and configurable number of threads. Threads can use synchronous or asynchronous I/O. The driver program will report latency for different patterns, IO sizes, number of threads, number of asynchronous IOs.

 

Measure and plot the throughput of sequential and random I/Os for different blob sizes on a ramdisk, HDD, SSD. As extra credit, provide also latency measurements. Also, measurements for asynchronous I/O with different queue depth.

 

Submission 

Turn in (by mail to h y 6 4 7 @ c s d . u o c . g r) a tar file that contains your solutions and a README file stating assumptions or special features.

___________________________________________________________________________________________________________________________________
(c) Copyright University of Crete, Greece, Last Modified: 26-Sep-2022