An observability database aims to ingest, analyze and store Metrics, Tracing and Logging data.

Clone this repo:
  1. 05d858a Parameterize memory size and some key improvements (#153) by Gao Hongtao · 16 hours ago main
  2. 4391412 Update go 1.19 (#152) by Jiajing LU · 2 days ago
  3. b34d085 Merge primary index into lsm index (#150) by Gao Hongtao · 7 days ago
  4. 8c4add2 Introduce race detector and coverage profile (#149) by Gao Hongtao · 10 days ago
  5. 8e87c5f Remove term metadata store (#148) by Gao Hongtao · 12 days ago

BanyanDB

BanyanDB, as an observability database, aims to ingest, analyze and store Metrics, Tracing and Logging data. It's designed to handle observability data generated by observability platform and APM system, like Apache SkyWalking etc.

Introduction

BanyanDB, as an observability database, aims to ingest, analyze and store Metrics, Tracing, and Logging data. It's designed to handle observability data generated by Apache SkyWalking. Before BanyanDB emerges, the Databases that SkyWalking adopted are not ideal for the APM data model, especially for saving tracing and logging data. Consequently, There’s room to improve the performance and resource usage based on the nature of SkyWalking data patterns.

The database research community usually uses RUM conjecture to describe how a database access data. BanyanDB combines several access methods to build a comprehensive APM database to balance read cost, update cost, and memory overhead.

Documents

Documents

RoadMap

Client manager

  • [x] gRPC server
  • [ ] HTTP server (v0.2.0)

Distributed manager

  • [ ] Sharding
  • [ ] Replication and consistency model
  • [ ] Load balance
  • [ ] Distributed query optimizer
  • [ ] Node discovery
  • [ ] Data queue

Data processor

  • [x] Schema management
  • [x] Time-series abstract layer
  • [x] Stream data processor
  • [x] Measure data processor
  • [x] Property data processor
  • [ ] TopNAggregation processor (v0.2.0)
  • [x] Index processor
  • [ ] TTL (v0.2.0)
  • [ ] Cold data processor (v0.2.0)
  • [ ] WAL

Query processor

  • [x] Stream query processor
  • [x] Measure query processor
  • [x] Index reader
  • [ ] Streaming pipeline processor(OR and nested querying) (v0.2.0)
  • [ ] Parallel executor
  • [ ] Cost-based optimizer

Verification

  • [x] E2E with OAP and simulated data
  • [ ] E2E with showcases, agents and OAP (v0.2.0)
  • [ ] Space utilization rate (v0.2.0)
  • [ ] Leading and trailing zero (v0.2.0)
  • [ ] Stability (v0.2.0)
  • [ ] Crash recovery
  • [ ] Performance

Tools

  • [ ] Command-line (v0.2.0)
  • [ ] Webapp (v0.2.0)

Contributing

For developers who want to contribute to this project, see Contribution Guide

License

Apache 2.0 License.