Memory Tracking

Objectives

The purpose of this feature is to provide insight about the memory consumption of each component. This should be useful when debugging memory issues

How to enable it?

In order to enable the memory reporting, you should add a compilation flag:

MEMORY_REPORTING=1 make -j $(NPROC)

Technical Details

An arena is a contiguous piece of memory, which Speedb allocates once and use part of the memory to minimize number of allocations requested from the OS

The memory counter is printed to the log by default every 10 minutes.

• This is a configurable parameter, you can change it under:

options.stats_dump_period_sec (the default of this option is 600, 10 minutes),

In order to get prints more often, the counter should be reduced.

Counters info:

• These metrics have been chosen to be part of the memory reporting feature because they represent most of the memory allocated in Speedb.

• Under Arena stats, there is memory allocated by the Arena, The total usage and per component that allocates using Arena.

• Each line under the Arena Stats describes the distribution of allocations requested (and not deallocated yet) per component in the code.

  • CF Stats - shows Memtable memory allocation per CF and summarizes all CFs together.

  • rocksdb.block-cache-usage - Total block cache usage reporting

  • rocksdb.estimate-table-readers-mem - How much memory is allocated to indexes and filter blocks

  • rocksdb.block-cache-pinned-usage - Pinned block cache usage

  • CacheAllocationUsage - Total memory allocated by BlockFetcher

Example for general output:

** Memory Reporting **

Arena Stats:

Total: 16M

ArenaWrappedDBIter: 0

FileIndexer::UpdateIndex: 0

MemTable::NewIterator: 0

LevelFileInit: 0

FindLevelFileTest::Add: 0

DoGenerateLevelFilesBrief: 0

TEST_GetLevelIterator: 0

Version::AddIteratorsForLevel: 0

Version::OverlapWithLevelIterator: 0

LogBuffer::AddLogToBuffer: 0

arena_test: 0

HashLinkList: 0

HashLinkListIterator: 0

HashLinkListDynamicIterator: 0

HashSkipList: 0

HashSkipListIterator: 0

HashSkipListDynamicIterator: 0

HashSpdb: 16M

HashSpdbIterator: 0

InlineSkipList: 0

SkipList: 0

SkipListIterator: 0

SkipListLookaheadIterator: 0

VectorMemtable: 0

CompactionMergingIterator: 0

NewErrorInternalIterator: 0

NewEmptyInternalIterator: 0

MergingIterator: 0

BlockBasedTableIterator: 0

BlockPrefixIndex::Builder: 0

CuckooTableIterator: 0

PlainTableBloomV1: 0

PlainTableIndexBuilder::FillIndexes: 0

PlainTableReader::NewIterator: 0

DynamicBloom: 0

DefaultMemtableImpl: 0

WriteBatchWithIndex: 0

CF Stats:

Total: 16M

[default]: 16M

rocksdb.block-cache-usage: 96

rocksdb.estimate-table-readers-mem: 659K

rocksdb.block-cache-pinned-usage: 96

Total CacheAllocationUsage: 655K

Jmalloc

In addition when Jemalloc is being used as an allocator for Speedb, there are more memory reporting metrics, in order to enable this dump please set option.dump_malloc_stats to true.

Allocated: 262920624, active: 264196096, metadata: 7739440 (n_thp 0), resident: 273063936, mapped: 307642368, retained: 79806464

Allocated - Number of bytes allocated by the Rocksdb

Active - Number of bytes allocated actively (includes the entire pages allocated size)

Metadata - Jemalloc itself metadata to manage memory allocation

Resident - Maximum overall size allocated includes zeroed pages including metadata (which may not reserve actual memory).

Mapped - Mapped is the sum of regions of virtual address space currently dedicated (internally) to serving some live allocation. - from jemalloc git

Retained - Freed memory that Jemalloc didnt return to the OS

Source: https://jemalloc.net/jemalloc.3.html#stats.allocated