LogoLogo
Back to Speedb.io⭐ GitHubDiscord
  • 👋About
    • Speedb Use Cases
    • Speedb Communication Channels
    • Release Cadence
    • Releases
    • Roadmap
  • 💻Getting started
    • Speedb Quick Start Example
    • Dependencies
    • How to Compile Speedb
    • Drop-in Replacement
    • Use prebuilt binaries
    • How to contribute
      • Contribute code
      • Feature request process
      • Submit a pull request
      • Add or update documentation
      • Report bugs and other issues
      • Help with new and ongoing feature development
    • Kafka Streams: How to use Speedb instead of RocksDB?
  • ✨Speedb Features
    • Memory Tracking
    • Speedb Tuning Function
    • Table Pinning Policy
    • Snapshot Optimization
    • On Thread Start Callback
    • Write Flow
    • Global Delayed write
    • Live Configuration Changes
    • Report Index Size per Column Family
    • Proactive Flushing
    • Sorted Hash Memtable
    • Paired Bloom Filter
  • ➕Enhancements
    • Range Delete Improvement
    • Dynamic Delayed Writes
    • Reduce switch memtable latency
  • 🛠️Tools
    • Log Parser
    • DB_bench: Groups
    • Beezcli Tool
  • 🔦RocksDB Basics
  • 📈Performance testing
Powered by GitBook
On this page
  • Motivation
  • Implementation
  • Advantage
  • Disadvantage

Was this helpful?

Edit on GitHub
  1. Enhancements

Reduce switch memtable latency

Motivation

The MemTableRepFactory class (each memtable representation inherits it) can be set as a “PrepareMemtableCreation” which will create a switch background thread that is responsible to construct a new memtable object to be used when switch occurs. A new memtable will be created when we fetch this memtable, so we'll always have one memtable available for use.

In case we don't have a ready memtable (this can happen if the switch memtable happened before the previous trigger was triggered) we create the memtable immediately .

To be able support this ability , memtable representation needs to divide the memtable construction to pre/post phases . This allows us to get the correct memtable’s parameters as defined in the mutable CF configuration.

Implementation

The MemTableRepFactory class (each memtable representation inherits it) can be set as a “PrepareMemtableCreation” which will create a switch background thread that is responsible to construct a new memtable object to be used when switch occurs. A new memtable will be created when we fetch this memtable, so we'll always have one memtable available for use.

In case we don't have a ready memtable (this can happen if the switch memtable happened before the previous trigger was triggered) we create the memtable immediately .

To be able support this ability , memtable representation needs to divide the memtable construction to pre/post phases . This allows us to get the correct memtable’s parameters as defined in the mutable CF configuration.

Advantage

1 - A switch memtable (size of the write buffer, threshold of the write buffer manager, wall size etc.) does not have a memory construction penalty and will allow for fluent acceptance of new writes.

2 - Simplified code

Disadvantage

1 - Each CF will have two mutable memtables (one active and one on hold), so we multiply the memtable construct memory usage.

PreviousDynamic Delayed WritesNextTools

Last updated 2 years ago

Was this helpful?

➕