filedb

FileDB is a disk-based key-value store that draws inspiration from the Bitcask architecture, designed to provide efficient storage and retrieval of data. This Zig implementation leverages a log-structured hashtable for storing record metadata, while maintaining an open disk file for appending records. On reaching a predefined disk space limit or during system restarts, the system rotates disk files, ensuring that older files remain accessible exclusively for reading.

Features

• Efficient Record Fetching: By maintaining precise metadata that points to both the file and the specific record location, accessing records becomes an O(1) operation, significantly improving performance.
• Constant Metadata Size: The metadata for each record remains constant in size, regardless of the size of the actual value stored, ensuring efficient memory utilization.
• High Throughput: The append-only mode of operation on open files allows the system to handle a large volume of requests effectively.

Core Methods

• init(allocator: std.mem.Allocator, options: ?config.Options): Initializes the FileDB instance.
• deinit(): De-initializes the FileDB, releasing resources.
• put(key:[]const u8, value: []const u8): Inserts a key-value pair into the database.
• get(key:[]const u8): Retrieves the value associated with a given key.
• delete(key: []const u8): Removes a key-value pair from the database.
• list(allocator: std.mem.Allocator): Lists all keys stored in the database.
• sync(): Syncs the current open datafile with the disk.
• storeHashMap(): Creates a HINTS file for optimized data retrieval.
• loadKeyDir(): Loads the hashmap from the HINTS file for efficient access.

Redis Compatibility

A Redis-compatible client is provided alongside the FileDB library, allowing users to interact with the database using familiar Redis commands. Here are some command examples:

127.0.0.1:6379> RING
(error) ERR unknown command
127.0.0.1:6379> PING
PONG
127.0.0.1:6379> get abcd
(nil)
127.0.0.1:6379> set abcd def
OK
127.0.0.1:6379> get abcd
"def"

Performance Benchmarks

The performance of FileDB has been validated through comprehensive benchmarking against standard Redis commands. Below are some key performance results:

• Set Command (10,000 requests):
  • Throughput: 13,736.26 requests per second
  • Average Latency: 3.615 ms
• Get Command (1,000,000 requests):
  • Throughput: 104,876.77 requests per second
  • Average Latency: 0.405 ms

Additional Resources

• Bitcask Paper by Riak
• Go Implementation of Bitcask
• Zig Learning Resources:
  • Basic MetaProgramming in Zig
  • Zig Book
  • Zig Standard Library Documentation
  • Zig Help

FileDB is ideal for developers seeking a lightweight and high-performance key-value store solution that efficiently manages large datasets while remaining compatible with Redis tooling.