CS647 Assignment 4 (tinyperf): tinyPerf: Compare the performance of tinyStory to RocksDB.
The goal of this assignment is to evaluate your key-value store implementation and compare it with RocksDB.
The assignment:
For the evaluation of the RocksDB you will need to evaluate two setups.
A) The first setup will evaluate the performance of the key-value stores for a dataset that does not fit in memory. For RocksDB a representative configuration is the following:
L0 32MB, Growth Factor 4, Max Levels = 4, Block Cache 8MB, Bloom Filters with 10 bits per key:
rocksdb::Options options;
options.compression = rocksdb::kNoCompression;
options.create_if_missing = true;
options.max_open_files = -1;
options.IncreaseParallelism();
options.use_direct_reads = true;
options.use_direct_io_for_flush_and_compaction = true;
options.compaction_readahead_size = 0;
options.writable_file_max_buffer_size = 1024 * 1024;
options.compaction_style = rocksdb::CompactionStyle::kCompactionStyleLevel;
options.level0_file_num_compaction_trigger = 4;
options.compression = rocksdb::kNoCompression;
options.create_if_missing = true;
options.use_direct_io_for_flush_and_compaction = 1;
options.use_direct_reads = 1;
options.max_write_buffer_number_to_maintain = 4;
options.num_levels = 4;
options.max_bytes_for_level_multiplier = 4;
options.max_bytes_for_level_base = 32 * 1024 * 1024;
rocksdb::BlockBasedTableOptions table_options;
table_options.filter_policy.reset(rocksdb::NewBloomFilterPolicy(10, false));
table_options.block_cache = rocksdb::NewLRUCache(8 * 1024 * 1024);
options.table_factory.reset(NewBlockBasedTableFactory(table_options));
options.max_open_files = -1;
options.target_file_size_base = 64 * 1024 * 1024;
options.target_file_size_multiplier = 1;
options.table_factory.reset(NewBlockBasedTableFactory(table_options));
B) The second setup will evaluate the performance of the key-value stores for a dataset that does fit in memory. For RocksDB a representative configuration is the following:
L0 32MB, Growth Factor 4, Max Levels = 4, Block Cache 4GB, Bloom Filters with 10 bits per key:
rocksdb::Options options;
options.compression = rocksdb::kNoCompression;
options.create_if_missing = true;
options.max_open_files = -1;
options.IncreaseParallelism();
options.use_direct_reads = true;
options.use_direct_io_for_flush_and_compaction = true;
options.compaction_readahead_size = 0;
options.writable_file_max_buffer_size = 1024 * 1024;
options.compaction_style = rocksdb::CompactionStyle::kCompactionStyleLevel;
options.level0_file_num_compaction_trigger = 4;
options.compression = rocksdb::kNoCompression;
options.create_if_missing = true;
options.use_direct_io_for_flush_and_compaction = 1;
options.use_direct_reads = 1;
options.max_write_buffer_number_to_maintain = 4;
options.num_levels = 4;
options.max_bytes_for_level_multiplier = 4;
options.max_bytes_for_level_base = 32 * 1024 * 1024;
rocksdb::BlockBasedTableOptions table_options;
table_options.filter_policy.reset(rocksdb::NewBloomFilterPolicy(10, false));
table_options.block_cache = rocksdb::NewLRUCache(4 * 1024 * 1024 * 1024);
options.table_factory.reset(NewBlockBasedTableFactory(table_options));
options.max_open_files = -1;
options.target_file_size_base = 64 * 1024 * 1024;
options.target_file_size_multiplier = 1;
options.table_factory.reset(NewBlockBasedTableFactory(table_options));
Evaluate your key value store with the RocksDB configurations we provide using with 100 million small KVs (Key=20B,Value=20B), 8589934 medium KVs (Key=20B, Value=480B), 4194304 large KVs (Key=20B,Value=1000B). All datasets are calculated to take 4GB space on the device. Measure throughput for different number of threads for both systems and plot the results.
Submission
Turn in (by mail to h y 6 4 7 @ c s d . u o c . g r) a tar file that contains your solutions and a README file stating assumptions or special features.
___________________________________________________________________________________________________________________________________
(c) Copyright University of Crete, Greece, Last Modified: 10-Oct-2022