sboxdb

sboxdb (Sandbox Database) is a distributed SQL database implemented in Rust, crafted as a learning tool to delve into the intricacies of modern database systems. This project provides insight into various essential components such as storage engines, transaction management (MVCC), Raft-based replication, and SQL query execution.

Key Features

• Key-Value Storage:

  • In-memory key-value storage available.
  • Work in progress on LSM-based storage tailored for OLTP.
  • Plans to incorporate benchmarks and observability.
  • Future additions include Parquet-based storage for OLAP and vector-friendly formats akin to Lance.

• Replication:

  • Utilizes a Raft-based replicated state machine, focusing on data consistency.
  • Currently lacks support for cluster membership configuration changes.

• Transactional Storage:

  • Implements transactional MVCC storage with concurrency control via MVCC and OCC.
  • Upcoming features aim to add Write-Ahead-Log support and ARIES recovery mechanisms.

• Transactional Access Methods:

  • Includes catalog-related access methods, CRUD operations for tuples, and index-based access methods with Raft-backed support.

• SQL Parser:

  • A handcrafted SQL parser developed without the use of yacc or bison, supporting a variety of data types including null, boolean, i64, double, and UTF-8 strings.
  • Supports various SQL syntax operations such as BEGIN, COMMIT, and ROLLBACK, as well as table manipulation commands and data retrieval operations.
  • For a complete reference, see the SQL documentation.

• SQL Execution Engine:

  • Features a simple heuristic-based query planner and optimizer that supports expressions, functions, and joins.
  • Implements components such as a logical planner, physical planner, and executors, alongside function support and SQL logical test utilities, which can be explored in-depth within the project.

• Browser Support:

  • A WebAssembly deployment is available for browser use, hosted here.

Documentation

• Comprehensive Architecture to understand the structural design of sboxdb.

• Detailed SQL reference providing insight into supported SQL syntax.

• Examples of supported SQL use cases.

• Curated References detailing materials used during development.

Developer Tools

• Task runner: Just for managing project commands.
• Python package management: uv for the SQL Logical Test script generator.