Performance Documentation Index

This document provides an overview of all performance-related documentation in the IoTDB Node.js client.

📊 Performance Documentation Overview

Quick Navigation

Want to improve performance? → Start with Performance Guide
Understanding the optimizations? → See pg-Inspired Optimizations
Analyzing pool performance? → Check Performance Analysis Summary
Using redirection optimization? → Read Redirection Design

📚 Performance Documents

1. Performance Guide (User-Focused)

File: performance-guide.md
Audience: Users wanting to optimize their IoTDB client usage
Length: ~331 lines

Contents:

Overview of implemented optimizations
Configuration options (enableFastSerialization)
Usage examples and best practices
Performance benchmarks
Troubleshooting guide
When to use columnar vs iterator patterns

Key Features Covered:

Buffer Pooling (70-80% GC reduction)
Fast Serialization (1.5-2x faster)
Optimized Timestamp Handling (20-30% faster)
Columnar Result Format (2-3x faster queries)

2. pg-Inspired Optimizations (Developer-Focused)

File: pg-inspired-optimizations.md
Audience: Developers wanting to understand implementation details
Length: ~447 lines

Contents:

Problem statement and research analysis
What makes pg nodejs fast
Implementation details (Phase 1+2)
Architecture diagrams (before/after)
Code examples and patterns
Testing strategy
Future roadmap (Phase 3)

Technical Deep Dives:

Buffer pooling system design
Fast serializer implementation
Columnar result API design
Memory optimization techniques

3. Performance Analysis Summary (Post-Mortem)

File: ../PERFORMANCE_ANALYSIS_SUMMARY.md
Audience: Developers, contributors, and maintainers
Length: ~281 lines

Contents:

Executive summary of pool optimization testing
Performance regression data (7.4x degradation discovered)
Root cause analysis
Actions taken (reverted optimizations)
Lessons learned

Key Findings:

FIFO queue overhead analysis
Lifecycle tracking overhead impact
Why simple is sometimes better
Benchmark methodology issues

4. Redirection Design (Advanced Feature)

File: redirection-design.md
Audience: Users of multi-node clusters
Length: ~524 lines

Contents:

Client-side redirection optimization
How IoTDB distributes data
RedirectCache implementation
Configuration options
Usage examples
Performance impact

📈 Performance Improvements Summary

Implemented (Phase 1+2)

✅ 2-3x overall performance improvement

Feature	Improvement	Status
Buffer Pooling	70-80% GC reduction	✅ Implemented
Fast Serialization	1.5-2x faster writes	✅ Implemented
Timestamp Handling	20-30% faster	✅ Implemented
Columnar Results	2-3x faster queries	✅ Implemented
Redirection Caching	Reduces network hops	✅ Implemented

Tested & Reverted (Lessons Learned)

❌ Pool “optimizations” causing 7.4x degradation

Feature	Impact	Status
FIFO Queue	7.4x slower RPC	❌ Reverted
Lifecycle Tracking	Added overhead	❌ Reverted

Lesson: Sometimes simple is better. Complex optimizations need real-world benchmarking.

🎯 Performance Benchmarks

Write Performance

Scenario	Before	After	Improvement
Small batch (10 rows)	2.5ms	1.8ms	1.4x
Medium batch (100 rows)	15ms	6ms	2.5x
Large batch (1000 rows)	180ms	65ms	2.8x

Query Performance (with Columnar API)

Result Size	Iterator	Columnar	Improvement
1K rows	45ms	18ms	2.5x
10K rows	520ms	180ms	2.9x
100K rows	5800ms	1900ms	3.1x

Memory Usage

Operation	Before	After	Improvement
GC Events (10K writes)	150	45	70% reduction
GC Events (100K query)	280	60	78% reduction

🔧 Configuration & Usage

Enable Fast Serialization (Default)

import { Session } from 'iotdb-client-nodejs';

const session = new Session({
  host: 'localhost',
  port: 6667,
  enableFastSerialization: true,  // Default: true
});

Use Columnar Results for Analytics

const dataSet = await session.executeQueryStatement('SELECT temp FROM root.sensors');

// Columnar format: zero object allocation
const columnar = await dataSet.toColumnar();
const avg = columnar.values[0].reduce((a, b) => a + b) / columnar.values[0].length;

await dataSet.close();

Enable Redirection (Multi-Node)

import { SessionPool } from 'iotdb-client-nodejs';

const pool = new SessionPool({
  nodeUrls: ['node1:6667', 'node2:6667', 'node3:6667'],
  maxPoolSize: 20,
  enableRedirection: true,      // Default: true
  redirectCacheTTL: 300000,     // 5 minutes
});

📖 Reading Order Recommendation

For Users (Just Want Performance)

Start with Performance Guide
Check Redirection Design if using multi-node clusters
Reference Performance Analysis Summary to understand what NOT to do

For Developers (Want to Understand)

Read pg-Inspired Optimizations for implementation details
Check Performance Analysis Summary for lessons learned
Reference Performance Guide for usage patterns
See Redirection Design for advanced optimization

For Contributors (Want to Improve)

Start with Performance Analysis Summary to understand past mistakes
Read pg-Inspired Optimizations for current implementation
Check Performance Guide for expected behavior
Review benchmark code in ../benchmark/README.md

🚀 Future Work

Planned Improvements (Phase 3)

Streaming/cursor API with backpressure
Request pipelining
Prepared statement caching
Optional native bindings

Note: Pool FIFO queue and lifecycle management were attempted but reverted due to performance regression. See Performance Analysis Summary for details.

🐛 Troubleshooting

High Memory Usage

Clear buffer pool periodically: globalBufferPool.clear()
Use iterator pattern for large result sets instead of toColumnar()
Check buffer pool stats: globalBufferPool.getStats()

Slow Serialization

Enable debug logging: process.env.LOG_LEVEL = 'debug'
Check for disabled fast serialization
Verify batch sizes are appropriate (100-1000 rows)

Unexpected Results

Disable fast serialization for debugging: enableFastSerialization: false
Check Performance Guide troubleshooting section
Review Performance Analysis Summary for known issues

📞 Support

Documentation Issues: Check docs/README.md
Performance Issues: Open an issue with benchmark results
Questions: See Contributing Guidelines

Last Updated: 2026-02-03
Status: Phase 1+2 Complete, Phase 3 Planned