Distributed Key-Value Store
NanoDB is what happens when you ask "How hard could building Redis be?" and actually follow through. Spoiler: pretty hard. But also incredibly rewarding.
🎯 The Challenge: Build a production-grade key-value store from absolute scratch—no libraries, no shortcuts, just raw C++17 and systems programming fundamentals.
🏗️ Architecture Highlights
| Write-Ahead Logging (WAL) | Crash recovery that actually works—every write is journaled before hitting memory |
| Sharded Locking | 16 independent lock partitions = threads don't fight over the same mutex |
| Log-Structured Merge | Deletions don't block writes—tombstones get compacted asynchronously |
💡 The Hard Problems I Solved
- Durability vs. Performance: Implemented group commit batching—flush 1000 writes in one fsync() instead of 1000 individual syscalls
- Concurrency Control: Lock-free reads using RCU (Read-Copy-Update) semantics for zero contention on GET operations
- Memory Management: Custom arena allocator reduces malloc() overhead by 70% compared to naive std::unordered_map
⚡ Performance: 50,000 writes/sec sustained throughput on a single thread. Scales linearly to 200k+ ops/sec with 8 threads.
🔍 What I Learned
Building a database is like building a house of cards in a wind tunnel. Every optimization creates new edge cases. Every edge case teaches you why PostgreSQL's source code is 1.5 million lines. Respect for database engineers: 📈 increased significantly.
Real-world use cases: Embedded analytics, IoT device state management, distributed cache nodes
- StackC++17, Multithreading, WAL, Sharding
- Sourcehttps://github.com/Shorya-agarwal/NanoDB-Distributed-Log-KV
