tree: 8417987a36f061416d9902b34c39c59fc5aa22c3

README.md

Apache IoTDB Node.js Client

A Node.js client for Apache IoTDB with support for SessionPool and TableSessionPool, providing efficient connection management and comprehensive query capabilities.

Overview
Features
Installation
Quick Start
Technical Architecture
API Reference
Development
Testing
Performance Testing
Examples
Documentation
Contributing
Release Process
License

Overview

The Apache IoTDB Node.js Client is a high-performance, feature-rich client library for interacting with Apache IoTDB, a time-series database designed for IoT data management. This client provides both tree model (timeseries) and table model (relational) APIs, enabling flexible data management strategies.

Features

Session Management: Single session with query, non-query, and insertTablet operations
SessionPool: Connection pooling for high-concurrency scenarios with automatic load balancing
- ✨ Enhanced Metrics: Comprehensive pool monitoring (totalCount, idleCount, activeCount, waitingCount)
- ✨ Wait Queue Tracking: Monitor requests waiting for connections
TableSessionPool: Specialized pool for table model operations with database context management
Multi-Node Support: Round-robin load balancing across multiple IoTDB nodes with failover
SSL/TLS Support: Secure connections with customizable SSL options and certificate validation
TypeScript Support: Full TypeScript definitions with strict type checking
Builder Pattern: Fluent API for elegant configuration management
Memory Efficient: SessionDataSet with lazy loading and pagination for large result sets
Comprehensive Testing: Unit tests, E2E tests, and benchmark tools included
Production Ready: Connection pooling, idle cleanup, health checks, and error handling

Installation

npm install @iotdb/client

Requirements

Node.js >= 14.0.0
Apache IoTDB >= 1.0.0

Quick Start

Basic Session Usage

import { Session } from '@iotdb/client';

const session = new Session({
  host: 'localhost',
  port: 6667,
  username: 'root',
  password: 'root',
});

await session.open();

// Execute non-query statement
await session.executeNonQueryStatement('CREATE DATABASE root.test');

// Execute query with SessionDataSet (iterator pattern - memory efficient)
const dataSet = await session.executeQueryStatement('SELECT * FROM root.test.**');
while (await dataSet.hasNext()) {
  const row = dataSet.next();
  console.log(row.getTimestamp(), row.getFields());
}
await dataSet.close();

// Or use toArray() helper for small result sets (loads all into memory)
const dataSet2 = await session.executeQueryStatement('SHOW DATABASES');
const allRows = await dataSet2.toArray(); // Returns [[timestamp, ...fields], ...]
console.log('All rows:', allRows);

// Execute query with custom timeout (30 seconds)
const customDataSet = await session.executeQueryStatement('SELECT * FROM root.test.**', 30000);
// ... iterate and close

// Insert tablet data (supports both tree and table models)
// Tree model example:
await session.insertTablet({
  deviceId: 'root.test.device1',
  measurements: ['temperature', 'humidity'],
  dataTypes: [3, 3], // FLOAT
  timestamps: [Date.now(), Date.now() + 1000],
  values: [
    [25.5, 60.0],
    [26.0, 61.5],
  ],
});

await session.close();

Using Builder Pattern (Recommended)

The Builder pattern provides a more elegant and fluent API for configuration:

import { Session, ConfigBuilder } from '@iotdb/client';

// Build a session configuration
const session = new Session(
  new ConfigBuilder()
    .host('localhost')
    .port(6667)
    .username('root')
    .password('root')
    .fetchSize(1024)
    .timezone('UTC+8')
    .build()
);

await session.open();
// ... use session
await session.close();

Query Results with SessionDataSet

The executeQueryStatement() method returns a SessionDataSet for efficient iteration through query results:

import { Session, SessionDataSet, RowRecord } from '@iotdb/client';

const session = new Session({
  host: 'localhost',
  port: 6667,
  username: 'root',
  password: 'root',
  fetchSize: 1024, // Rows per fetch
});

await session.open();

// Execute query and get SessionDataSet
const dataSet: SessionDataSet = await session.executeQueryStatement(
  'SELECT temperature, humidity FROM root.test.device1'
);

// Iterate through results efficiently
while (await dataSet.hasNext()) {
  const row: RowRecord = dataSet.next();
  
  // Access by column name
  const temp = row.getFloat('temperature');
  const humidity = row.getFloat('humidity');
  
  // Access by index
  const timestamp = row.getTimestamp();
  
  console.log(`${timestamp}: temp=${temp}, humidity=${humidity}`);
}

// Always close the dataset
await dataSet.close();
await session.close();

Benefits of SessionDataSet:

✅ Memory Efficient: Only keeps current batch in memory
✅ Lazy Loading: Fetches data on-demand
✅ Large Datasets: Can handle results larger than available RAM
✅ Type Safety: Typed getters prevent errors
✅ Resource Management: Proper cleanup with close()

See SessionDataSet Guide for complete documentation.

SessionPool Usage

import { SessionPool } from '@iotdb/client';

const pool = new SessionPool('localhost', 6667, {
  username: 'root',
  password: 'root',
  maxPoolSize: 10,
  minPoolSize: 2,
  maxIdleTime: 60000,
  waitTimeout: 60000,
});

await pool.init();

// Execute queries using the pool
const result = await pool.executeQueryStatement('SELECT * FROM root.test.**');

// Execute non-query statements
await pool.executeNonQueryStatement(
  'CREATE TIMESERIES root.test.device1.temperature WITH DATATYPE=FLOAT'
);

// Insert data
await pool.insertTablet({
  deviceId: 'root.test.device1',
  measurements: ['temperature'],
  dataTypes: [3], // FLOAT
  timestamps: [Date.now()],
  values: [[25.5]],
});

// Get pool statistics
console.log('Pool size:', pool.getPoolSize());
console.log('Available:', pool.getAvailableSize());

// Enhanced metrics for better monitoring
console.log('Waiting requests:', pool.waitingCount);
const stats = pool.getPoolStats();
console.log('Comprehensive stats:', stats);
// { total, idle, active, waiting, endpoints, redirectCacheSize }

await pool.close();

Enhanced Pool Metrics

The SessionPool provides comprehensive metrics for monitoring pool health:

const pool = new SessionPool({
  host: 'localhost',
  port: 6667,
  maxPoolSize: 10,
  minPoolSize: 2,
  waitTimeout: 60000,        // Timeout for waiting requests (ms)
  maxIdleTime: 60000,        // Idle connection timeout (ms)
});

// New metric getters (backward compatible with old methods)
console.log('Total connections:', pool.totalCount);
console.log('Idle connections:', pool.idleCount);
console.log('Active connections:', pool.activeCount);
console.log('Waiting requests:', pool.waitingCount);

// Comprehensive stats object
const stats = pool.getPoolStats();
// { total, idle, active, waiting, endpoints, redirectCacheSize }

Key Features:

✅ Enhanced Metrics: Comprehensive pool health monitoring
✅ Wait Queue Tracking: Monitor requests waiting for connections
✅ Backward Compatible: All existing code works unchanged

await pool.close();


### Explicit Session Management

For more control, you can explicitly get and release sessions from the pool:

```typescript
import { SessionPool } from '@iotdb/client';

const pool = new SessionPool('localhost', 6667, {
  username: 'root',
  password: 'root',
  maxPoolSize: 10,
});

await pool.init();

// Get a session from the pool
const session = await pool.getSession();

try {
  // Use the session for multiple operations
  await session.executeNonQueryStatement('CREATE DATABASE root.test');
  const result = await session.executeQueryStatement('SHOW DATABASES');
  await session.insertTablet({
    deviceId: 'root.test.device1',
    measurements: ['temperature'],
    dataTypes: [3],
    timestamps: [Date.now()],
    values: [[25.5]],
  });
} finally {
  // Always release the session back to the pool
  pool.releaseSession(session);
}

await pool.close();

Concurrent Operations (High Throughput)

For high-throughput scenarios, use the concurrent APIs optimized for Node.js:

Batch Insert - Single RPC Call

Insert multiple tablets in a single RPC call (most efficient for tree model):

import { Session, TreeTablet, TSDataType } from '@iotdb/client';

const session = new Session({ host: 'localhost', port: 6667 });
await session.open();

// Create multiple tablets
const tablets = [];
for (let i = 0; i < 100; i++) {
  tablets.push({
    deviceId: `root.test.device${i}`,
    measurements: ['temperature'],
    dataTypes: [TSDataType.FLOAT],
    timestamps: [Date.now() + i * 1000],
    values: [[25.0 + i]],
  });
}

// Insert all tablets in single RPC call
await session.insertTablets(tablets);

await session.close();

Concurrent Pool Insertion

Use pool-level concurrent insertion for maximum throughput:

import { SessionPool } from '@iotdb/client';

const pool = new SessionPool({
  host: 'localhost',
  port: 6667,
  maxPoolSize: 20,
  minPoolSize: 5,
});
await pool.init();

// Generate tablets
const tablets = generateTablets(1000);

// Insert concurrently with controlled parallelism
// Pre-acquires sessions to avoid contention
await pool.insertTabletsParallel(tablets, { concurrency: 20 });

await pool.close();

Generic Parallel Execution

Execute any operations in parallel using the pool:

import { SessionPool } from '@iotdb/client';

const pool = new SessionPool({ host: 'localhost', port: 6667, maxPoolSize: 10 });
await pool.init();

const deviceNames = ['d1', 'd2', 'd3', 'd4', 'd5'];

// Execute operations in parallel
const results = await pool.executeParallel(
  deviceNames,
  async (session, deviceName) => {
    await session.executeNonQueryStatement(
      `CREATE TIMESERIES root.test.${deviceName}.value WITH DATATYPE=FLOAT`
    );
    return `Created ${deviceName}`;
  },
  { concurrency: 5 }
);

console.log(results); // ['Created d1', 'Created d2', ...]

await pool.close();

Utility Functions

Standalone utilities for concurrent execution:

import { executeConcurrent, chunkArray, createSemaphore } from '@iotdb/client';

// Execute any async operations with controlled concurrency
const result = await executeConcurrent(
  items,
  async (item) => await processItem(item),
  { concurrency: 10, logProgressEvery: 100 }
);
console.log(`Success: ${result.successCount}, Failed: ${result.failureCount}`);

// Split arrays for batch processing
const chunks = chunkArray(largeArray, 100);

// Fine-grained concurrency control
const sem = createSemaphore(5);
await sem.acquire();
try {
  await doWork();
} finally {
  sem.release();
}

Multi-Node Support

Method 1: Using nodeUrls with String Format (Recommended)

When nodes have different host:port combinations, use the nodeUrls configuration with string array format:

import { SessionPool, PoolConfigBuilder } from '@iotdb/client';

// Using config object with string array (RECOMMENDED)
const pool1 = new SessionPool({
  nodeUrls: [
    'node1.example.com:6667',
    'node2.example.com:6668',
    'node3.example.com:6669',
  ],
  username: 'root',
  password: 'root',
  maxPoolSize: 15,
  minPoolSize: 3,
});

// Or using Builder pattern with string array
const pool2 = new SessionPool(
  new PoolConfigBuilder()
    .nodeUrls([
      'node1.example.com:6667',
      'node2.example.com:6668',
      'node3.example.com:6669',
    ])
    .username('root')
    .password('root')
    .maxPoolSize(15)
    .minPoolSize(3)
    .build()
);

await pool1.init();
// Connections will be distributed across all nodes using round-robin

Method 2: Using nodeUrls with Object Format (Also Supported)

You can also use the object format for nodeUrls:

const pool = new SessionPool({
  nodeUrls: [
    { host: 'node1.example.com', port: 6667 },
    { host: 'node2.example.com', port: 6668 },
    { host: 'node3.example.com', port: 6669 },
  ],
  username: 'root',
  password: 'root',
  maxPoolSize: 15,
});

Method 3: Traditional API (For Same Port)

When all nodes share the same port:

import { SessionPool } from '@iotdb/client';

const pool = new SessionPool(
  ['node1.example.com', 'node2.example.com', 'node3.example.com'],
  6667,
  {
    username: 'root',
    password: 'root',
    maxPoolSize: 15,
  }
);

await pool.init();
// Connections will be distributed across all nodes using round-robin

SSL/TLS Support

import { Session } from '@iotdb/client';
import * as fs from 'fs';

const session = new Session({
  host: 'localhost',
  port: 6667,
  username: 'root',
  password: 'root',
  enableSSL: true,
  sslOptions: {
    ca: fs.readFileSync('/path/to/ca.crt'),
    cert: fs.readFileSync('/path/to/client.crt'),
    key: fs.readFileSync('/path/to/client.key'),
    rejectUnauthorized: true,
  },
});

await session.open();

TableSessionPool Usage

import { TableSessionPool } from '@iotdb/client';

const tablePool = new TableSessionPool('localhost', 6667, {
  username: 'root',
  password: 'root',
  database: 'my_database', // Set default database for table model
  maxPoolSize: 10,
  minPoolSize: 2,
});

await tablePool.init();

// Execute queries in table mode
const result = await tablePool.executeQueryStatement('SHOW DATABASES');

await tablePool.close();

Redirection Support ✅ (Implemented)

Status: ✅ Fully Implemented

The client now supports automatic write redirection for multi-node IoTDB clusters. When a write operation is sent to a node that doesn‘t own the device’s data, the server responds with a redirect recommendation (status code 400). The client automatically:

Caches the device→endpoint mapping
Creates/reuses a connection to the optimal endpoint
Retries the operation on the correct node
Uses the cached mapping for future writes to the same device

Benefits:

30-50% throughput improvement by avoiding cross-node data forwarding
Reduced network latency
Better resource utilization

Configuration:

import { SessionPool, TableSessionPool } from '@iotdb/client';

// Tree model pool with redirection
const treePool = new SessionPool({
  nodeUrls: ['192.168.1.100:6667', '192.168.1.101:6667', '192.168.1.102:6667'],
  username: 'root',
  password: 'root',
  maxPoolSize: 10,
  enableRedirection: true,        // Enable redirection (default: true)
  redirectCacheTTL: 300000,       // Cache TTL in ms (default: 5 minutes)
});

// Table model pool with redirection
const tablePool = new TableSessionPool({
  nodeUrls: ['192.168.1.100:6667', '192.168.1.101:6667', '192.168.1.102:6667'],
  enableRedirection: true,
});

How It Works:

// First write to a device - server returns redirect recommendation
const tablet = {
  deviceId: 'root.sg.device1',
  measurements: ['temperature'],
  dataTypes: [TSDataType.FLOAT],
  timestamps: [Date.now()],
  values: [[25.5]],
};

await pool.insertTablet(tablet);
// → Writes to Node A (via round-robin)
// → Write succeeds!
// → Server responds with code 400: "Recommend using Node B for this device in the future"
// → Client caches: device1 → Node B (for next write)

// Second write to same device - uses cached endpoint
await pool.insertTablet({
  deviceId: 'root.sg.device1',
  measurements: ['temperature'],
  dataTypes: [TSDataType.FLOAT],
  timestamps: [Date.now() + 1000],
  values: [[26.0]],
});
// → Client checks cache: device1 → Node B
// → Writes directly to Node B
// → No redirect needed!

Testing:

Redirection support has been tested with the 1C3D (1 ConfigNode, 3 DataNodes) cluster configuration. Run E2E tests:

# Start 1C3D cluster
docker-compose -f docker-compose-1c3d.yml up -d

# Run redirection tests
MULTI_NODE=true npm run test:e2e

Implementation Details:

See docs/redirection-design.md for detailed design documentation.

API Reference

Configuration Builders

ConfigBuilder

Fluent API for building Session configurations:

import { ConfigBuilder } from '@iotdb/client';

const config = new ConfigBuilder()
  .host('localhost')
  .port(6667)
  .username('root')
  .password('root')
  .database('mydb')
  .timezone('UTC+8')
  .fetchSize(2048)
  .enableSSL(false)
  .build();

Methods:

host(host: string): this - Set the host
port(port: number): this - Set the port
nodeUrls(nodeUrls: EndPoint[]): this - Set multiple node URLs
username(username: string): this - Set the username
password(password: string): this - Set the password
database(database: string): this - Set the database
timezone(timezone: string): this - Set the timezone
fetchSize(fetchSize: number): this - Set the fetch size
enableSSL(enable: boolean): this - Enable or disable SSL
sslOptions(sslOptions: SSLOptions): this - Set SSL options
build(): Config - Build and return the configuration

PoolConfigBuilder

Fluent API for building SessionPool configurations (extends ConfigBuilder):

import { PoolConfigBuilder } from '@iotdb/client';

const config = new PoolConfigBuilder()
  .host('localhost')
  .port(6667)
  .username('root')
  .password('root')
  .maxPoolSize(20)
  .minPoolSize(5)
  .maxIdleTime(30000)
  .waitTimeout(45000)
  .build();

Additional Methods:

maxPoolSize(size: number): this - Set maximum pool size
minPoolSize(size: number): this - Set minimum pool size
maxIdleTime(time: number): this - Set maximum idle time (ms)
waitTimeout(timeout: number): this - Set wait timeout (ms)
build(): PoolConfig - Build and return the pool configuration

Data Types

IoTDB Node.js client supports all IoTDB data types including BOOLEAN, INT32, INT64, FLOAT, DOUBLE, TEXT, BLOB, STRING, DATE, and TIMESTAMP. See Data Types Reference for comprehensive documentation on:

Type mappings between JavaScript and IoTDB
Usage examples for each data type
Best practices and encoding options
Null value handling

Session

Constructor

Option 1: Using config object

new Session(config: Config)

Option 2: Using Builder pattern (Recommended)

new Session(new ConfigBuilder()...build())

The config must include either:

host and port for single node connection
nodeUrls for multi-node connection (uses first node)

Methods

async open(): Promise<void> - Open the session
async close(): Promise<void> - Close the session
async executeQueryStatement(sql: string, timeoutMs?: number): Promise<QueryResult> - Execute a query with optional timeout (default: 60000ms)
async executeNonQueryStatement(sql: string): Promise<void> - Execute a non-query statement
async insertTablet(tablet: Tablet): Promise<void> - Insert tablet data
isOpen(): boolean - Check if session is open

SessionPool

Constructor

Option 1: Traditional API (Backward compatible)

new SessionPool(hosts: string | string[], port: number, config?: Partial<PoolConfig>)

Option 2: Using config object with nodeUrls

new SessionPool(config: PoolConfig)

Option 3: Using Builder pattern (Recommended)

new SessionPool(new PoolConfigBuilder()...build())

Methods

Connection Management:

async init(): Promise<void> - Initialize the pool
async close(): Promise<void> - Close all connections

Automatic Session Management:

async executeQueryStatement(sql: string, timeoutMs?: number): Promise<QueryResult> - Execute a query with optional timeout (default: 60000ms)
async executeNonQueryStatement(sql: string): Promise<void> - Execute a non-query statement
async insertTablet(tablet: Tablet): Promise<void> - Insert tablet data

Explicit Session Management:

async getSession(): Promise<Session> - Get a session from the pool (must be released)
releaseSession(session: Session): void - Release a session back to the pool

Pool Statistics:

getPoolSize(): number - Get current pool size
getAvailableSize(): number - Get available connections
getInUseSize(): number - Get number of sessions currently in use

TableSessionPool

Same as SessionPool but optimized for table model operations. Automatically executes USE DATABASE when configured with a database. All query methods support the same timeout parameter (default: 60000ms).

Constructor

Same constructor options as SessionPool.

Types

Config

interface Config {
  host?: string;
  port?: number;
  nodeUrls?: string[] | EndPoint[];  // String array format: ["host1:6667", "host2:6668"]
  username?: string;
  password?: string;
  database?: string;
  timezone?: string;
  fetchSize?: number;
  enableSSL?: boolean;
  sslOptions?: SSLOptions;
}

Note: Either host/port OR nodeUrls must be provided.

Use nodeUrls in string array format (e.g., ["host1:6667", "host2:6668"]) for nodes with different ports (RECOMMENDED)
Object format [{ host, port }] is also supported for backward compatibility

EndPoint

interface EndPoint {
  host: string;
  port: number;
}

PoolConfig

interface PoolConfig extends Config {
  maxPoolSize?: number;
  minPoolSize?: number;
  maxIdleTime?: number;
  waitTimeout?: number;
}

Tablet

#### TreeTablet (Tree Model / Timeseries Model)

**Interface (for plain objects):**
```typescript
interface ITreeTablet {
  deviceId: string;           // Full path like "root.test.device1"
  measurements: string[];     // Sensor names
  dataTypes: number[];        // TSDataType for each measurement
  timestamps: number[];       // Array of timestamps
  values: any[][];           // 2D array: [rows][columns]
}

Class (with helper methods):

import { TreeTablet, TSDataType } from '@iotdb/client';

// Create a tablet
const tablet = new TreeTablet(
  'root.test.device1',
  ['temperature', 'humidity'],
  [TSDataType.FLOAT, TSDataType.DOUBLE]
);

// Add rows one at a time using addRow method
tablet.addRow(Date.now(), [25.5, 60.0]);
tablet.addRow(Date.now() + 1000, [26.0, 61.5]);
tablet.addRow(Date.now() + 2000, [26.5, 62.0]);

// Insert the tablet
await session.insertTablet(tablet);

Alternative: Plain object approach (still supported)

await session.insertTablet({
  deviceId: 'root.test.device1',
  measurements: ['temperature', 'humidity'],
  dataTypes: [TSDataType.FLOAT, TSDataType.DOUBLE],
  timestamps: [Date.now(), Date.now() + 1000],
  values: [[25.5, 60.0], [26.0, 61.5]],
});

TableTablet (Table Model / Relational Model)

Interface (for plain objects):

interface ITableTablet {
  tableName: string;          // Table name
  columnNames: string[];      // Column names (excluding timestamp)
  columnTypes: number[];      // TSDataType for each column
  columnCategories: ColumnCategory[];  // Category for each column
  timestamps: number[];       // Array of timestamps
  values: any[][];           // 2D array: [rows][columns]
}

enum ColumnCategory {
  TAG = 0,         // Tag column - indexed for WHERE clause filtering
  FIELD = 1,       // Field column - measurement values
  ATTRIBUTE = 2,   // Attribute column - metadata not indexed
  TIME = 3,        // Time column (reserved for internal use, do not use in columnCategories)
}

Class (with helper methods):

import { TableTablet, ColumnCategory, TSDataType } from '@iotdb/client';

// Create a tablet
const tablet = new TableTablet(
  'sensor_data',
  ['device_id', 'temperature', 'humidity'],
  [TSDataType.TEXT, TSDataType.FLOAT, TSDataType.DOUBLE],
  [ColumnCategory.TAG, ColumnCategory.FIELD, ColumnCategory.FIELD]
);

// Add rows one at a time using addRow method
tablet.addRow(Date.now(), ['device_001', 25.5, 60.0]);
tablet.addRow(Date.now() + 1000, ['device_002', 26.0, 61.5]);
tablet.addRow(Date.now() + 2000, ['device_003', 26.5, 62.0]);

// Insert the tablet
await pool.insertTablet(tablet);

Alternative: Plain object approach (still supported)

await pool.insertTablet({
  tableName: 'sensor_data',
  columnNames: ['device_id', 'temperature', 'humidity'],
  columnTypes: [TSDataType.TEXT, TSDataType.FLOAT, TSDataType.DOUBLE],
  columnCategories: [ColumnCategory.TAG, ColumnCategory.FIELD, ColumnCategory.FIELD],
  timestamps: [Date.now(), Date.now() + 1000],
  values: [['device_001', 25.5, 60.0], ['device_002', 26.0, 61.5]],
});

Note: TIME is reserved for internal use. When specifying columnCategories in TableTablet, only use TAG, FIELD, and ATTRIBUTE. Timestamps are handled separately via the timestamps array.

Deprecated Tablet (for backward compatibility)

interface Tablet {
  deviceId: string;
  measurements: string[];
  dataTypes: number[]; // 0=BOOLEAN, 1=INT32, 2=INT64, 3=FLOAT, 4=DOUBLE, 5=TEXT
  timestamps: number[];
  values: any[][];
}
// Note: Use TreeTablet instead


#### QueryResult
```typescript
interface QueryResult {
  columns: string[];
  dataTypes: string[];
  rows: any[][];
  queryId?: number;
}

Data Types

When using insertTablet, specify data types using these constants:

0 - BOOLEAN
1 - INT32
2 - INT64
3 - FLOAT
4 - DOUBLE
5 - TEXT

Migration Guide

Upgrading to the New API

The new version maintains full backward compatibility while adding new features. No changes are required for existing code, but you may want to adopt the new features:

Multi-Node with Different Ports

Old way (still works, but limited to same port):

const pool = new SessionPool(
  ['node1', 'node2', 'node3'],
  6667,
  { username: 'root', password: 'root' }
);

New way (supports different ports per node with string format - RECOMMENDED):

const pool = new SessionPool({
  nodeUrls: [
    'node1:6667',
    'node2:6668',
    'node3:6669',
  ],
  username: 'root',
  password: 'root',
});

Alternative (object format also supported):

const pool = new SessionPool({
  nodeUrls: [
    { host: 'node1', port: 6667 },
    { host: 'node2', port: 6668 },
    { host: 'node3', port: 6669 },
  ],
  username: 'root',
  password: 'root',
});

Using Builder Pattern

Old way (still works):

const session = new Session({
  host: 'localhost',
  port: 6667,
  username: 'root',
  password: 'root',
  fetchSize: 2048,
});

New way (more fluent):

import { ConfigBuilder } from '@iotdb/client';

const session = new Session(
  new ConfigBuilder()
    .host('localhost')
    .port(6667)
    .username('root')
    .password('root')
    .fetchSize(2048)
    .build()
);

Explicit Session Management

Old way (still works):

// Pool automatically manages sessions
const result = await pool.executeQueryStatement('SELECT ...');

New way (more control):

// Explicitly get and release sessions
const session = await pool.getSession();
try {
  const result1 = await session.executeQueryStatement('SELECT ...');
  const result2 = await session.executeQueryStatement('SELECT ...');
  // ... multiple operations with same session
} finally {
  pool.releaseSession(session);
}

Technical Architecture

Overview

The IoTDB Node.js client follows a three-layer architecture design, optimized for both single-session and high-concurrency scenarios:

┌─────────────────────────────────────────────────────┐
│  Application Layer (Your Code)                      │
└─────────────────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────────────┐
│  Pool Layer                                         │
│  ┌──────────────────┐    ┌──────────────────────┐  │
│  │  SessionPool     │    │ TableSessionPool     │  │
│  │  - Load Balance  │    │ - Database Context   │  │
│  │  - Pool Mgmt     │    │ - Pool Mgmt          │  │
│  └──────────────────┘    └──────────────────────┘  │
└─────────────────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────────────┐
│  Session Layer                                      │
│  ┌──────────────────────────────────────────────┐  │
│  │  Session                                     │  │
│  │  - Query / Non-Query                         │  │
│  │  - InsertTablet                              │  │
│  │  - Result Parsing                            │  │
│  └──────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────┘
                    ↓
┌─────────────────────────────────────────────────────┐
│  Connection Layer                                   │
│  ┌──────────────────────────────────────────────┐  │
│  │  Connection (Thrift)                         │  │
│  │  - TCP/SSL Transport                         │  │
│  │  - Session Lifecycle                         │  │
│  │  - Low-level Protocol                        │  │
│  └──────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────┘
                    ↓
              Apache IoTDB

Core Components

1. Connection Layer (`src/connection/Connection.ts`)

Manages low-level Thrift connections over TCP or SSL/TLS
Handles session lifecycle (open/close with sessionId/statementId)
Implements TFramedTransport and TBinaryProtocol
Supports single endpoint connections
Key Pattern: One connection = one IoTDB node endpoint

2. Session Layer (`src/client/Session.ts`)

High-level API for database operations
Methods: executeQueryStatement(), executeNonQueryStatement(), insertTablet()
Handles query result parsing with SessionDataSet
Supports pagination with configurable fetchSize
Key Pattern: Uses first node from nodeUrls for single-session scenarios

3. Pool Layer (`src/client/BaseSessionPool.ts`, `SessionPool.ts`, `TableSessionPool.ts`)

Connection pooling with configurable min/max sizes
Round-robin load balancing across multiple endpoints
Automatic idle connection cleanup (maxIdleTime)
Wait queue when pool exhausted (waitTimeout)
Health checks and connection recycling
Key Pattern: Distributes connections across all nodes in nodeUrls

4. Configuration System (`src/utils/Config.ts`)

Builder pattern for fluent configuration
Support for both old (host/port) and new (nodeUrls) formats

5. Tablet Type System

The client provides distinct tablet types for tree and table models:

TreeTablet (Timeseries Model):

deviceId: Full path (e.g., “root.sg.device”)
measurements: Sensor names
dataTypes: Type for each measurement
timestamps and values: Time-series data

TableTablet (Relational Model):

tableName: Table name (not a path)
columnNames: All columns including tags, time, fields, attributes
columnTypes: Type for each column
columnCategories: Category (TAG, TIME, FIELD, ATTRIBUTE) for each column
timestamps and values: Includes tag values in data

Polymorphic insertTablet(): Both Session and TableSession use the same method name with runtime type dispatch:

// Works with TreeTablet
await session.insertTablet({ deviceId: '...', measurements: [...], ...});

// Works with TableTablet
await tableSession.insertTablet({ tableName: '...', columnNames: [...], ...});

Type-safe with TypeScript interfaces
Validation and default values

Data Flow

Query Execution Flow

1. Application calls pool.executeQueryStatement()
2. Pool acquires available Session (round-robin)
3. Session sends query via Connection to IoTDB
4. IoTDB returns SessionDataSet with queryId
5. SessionDataSet fetches data in batches (fetchSize)
6. Application iterates results with hasNext()/next()
7. Session released back to pool
8. SessionDataSet.close() releases server resources

Insert Flow

1. Application calls pool.insertTablet()
2. Pool acquires available Session (round-robin)
3. Session serializes Tablet data by column
4. Data sent via Connection to IoTDB
5. IoTDB acknowledges write
6. Session released back to pool

Thread Safety & Concurrency

Sessions: Not thread-safe; use SessionPool for concurrency
SessionPool: Thread-safe; internal locking for session management
Connection Lifecycle: Managed automatically by pool
Load Balancing: Round-robin assignment on session acquisition
Idle Cleanup: Background task removes idle connections

Thrift Integration

The client uses Apache Thrift for RPC communication:

Generated Code: src/thrift/generated/ from IoTDB's .thrift files
Protocol: TBinaryProtocol (compact, efficient)
Transport: TFramedTransport (message boundaries)
SSL Support: Configurable TLS transport layer
Version: Compatible with Apache IoTDB 1.0+

Memory Management

SessionDataSet: Lazy loading with pagination (default: 1024 rows/fetch)
Connection Pool: Bounded size prevents resource exhaustion
Idle Cleanup: Automatic connection cleanup after maxIdleTime
Result Sets: Must call close() to release server resources

Error Handling

Connection Errors: Automatic retry with next node in pool
Timeout Handling: Configurable query timeouts (default: 60s)
Pool Exhaustion: Wait queue with timeout
Thrift Errors: Wrapped in JavaScript errors with stack traces

Configuration Patterns

Constructor Overloading

SessionPool/TableSessionPool support two constructor signatures:

// New format (recommended):
new SessionPool({ nodeUrls: ["host1:6667", "host2:6667"] });

// Old format (backward compatible):
new SessionPool(["host1", "host2"], 6667, { /* options */ });

Builder Pattern

const session = new Session(
  new ConfigBuilder()
    .host('localhost')
    .port(6667)
    .fetchSize(2048)
    .build()
);

Development

Prerequisites

Node.js >= 14.0.0
npm >= 6.0.0
Apache Thrift compiler (optional, for regenerating Thrift files)
Git

Development Setup

Clone the repository:

git clone https://github.com/CritasWang/@iotdb/client.git
cd @iotdb/client

Install dependencies:

npm install

Build the project:

npm run build

Build System

The project uses a two-step build process:

esbuild: Fast TypeScript compilation
- Configured in esbuild.config.js
- Compiles src/ to dist/ directory
- Excludes type declaration files
tsc: Type declaration generation
- Generates .d.ts files for TypeScript support
- Run with --emitDeclarationOnly flag
- Ensures type safety for consumers
copy:thrift: Copy generated Thrift files
- Copies .js files from src/thrift/generated/ to dist/thrift/generated/
- Required because Thrift code uses require() statements

Build commands:

npm run build          # Complete build (esbuild + tsc + copy)
npm run build:esbuild  # Only esbuild compilation
npm run build:types    # Only type declarations

Development Workflow

Make changes in src/ directory
Build with npm run build
Test with npm test
Lint with npm run lint
Format with npm run format

Code Style

Use TypeScript strict mode
Follow existing code formatting (Prettier)
Add JSDoc comments for public APIs
Keep functions focused and concise
Use async/await instead of callbacks
Handle errors appropriately
Prefer explicit types over any

Regenerating Thrift Files

If you need to update to a newer version of IoTDB's Thrift definitions:

Download the latest Thrift files from Apache IoTDB:

git clone --depth 1 https://github.com/apache/iotdb.git /tmp/iotdb

Copy the Thrift files:

cp /tmp/iotdb/iotdb-protocol/thrift-datanode/src/main/thrift/client.thrift thrift/
cp /tmp/iotdb/iotdb-protocol/thrift-commons/src/main/thrift/common.thrift thrift/

Regenerate the Node.js client:

npm run generate:thrift

Test thoroughly to ensure compatibility

Testing

Test Structure

tests/
├── unit/           # Unit tests (fast, no external dependencies)
│   ├── Config.test.ts
│   ├── Logger.test.ts
│   └── ...
└── e2e/            # End-to-end tests (require IoTDB)
    ├── Session.test.ts
    ├── SessionPool.test.ts
    ├── TableSessionPool.test.ts
    └── ...

Running Tests

Run all tests:

npm test

Run only unit tests:

npm run test:unit

Run only E2E tests (requires IoTDB instance):

export IOTDB_HOST=localhost
export IOTDB_PORT=6667
export IOTDB_USER=root
export IOTDB_PASSWORD=root
npm run test:e2e

E2E Test Setup

E2E tests require a running IoTDB instance. You can use Docker Compose:

Single Node (1c1d):

docker-compose -f docker-compose-1c1d.yml up -d

3-Node Cluster (3c3d):

docker-compose -f docker-compose-3c3d.yml up -d

Stop containers:

docker-compose -f docker-compose-1c1d.yml down

Test Patterns

Unit Test Example

describe('ConfigBuilder', () => {
  test('should build config with all options', () => {
    const config = new ConfigBuilder()
      .host('localhost')
      .port(6667)
      .username('root')
      .password('root')
      .build();
    
    expect(config.host).toBe('localhost');
    expect(config.port).toBe(6667);
  });
});

E2E Test Example

describe('Session E2E Tests', () => {
  let session: Session;

  beforeAll(async () => {
    session = new Session({
      host: process.env.IOTDB_HOST || 'localhost',
      port: parseInt(process.env.IOTDB_PORT || '6667'),
      username: process.env.IOTDB_USER || 'root',
      password: process.env.IOTDB_PASSWORD || 'root',
    });
    await session.open();
  }, 60000); // 60s timeout for connection

  afterAll(async () => {
    if (session?.isOpen()) {
      await session.close();
    }
  });

  test('should execute query', async () => {
    if (!session.isOpen()) return; // Skip if no connection
    
    const dataSet = await session.executeQueryStatement('SHOW DATABASES');
    const rows = await dataSet.toArray();
    expect(Array.isArray(rows)).toBe(true);
    await dataSet.close();
  });
});

Test Coverage

Current test coverage:

Unit tests: Core utilities (Config, Logger, data serialization)
E2E tests: Session, SessionPool, TableSessionPool
All data types tested simultaneously
Multi-node scenarios tested
Pool behavior tested (size limits, timeouts, cleanup)

Debugging Tests

Debug single test:

npm run test:debug

Debug E2E tests:

npm run test:e2e:debug

Check for open handles:

npm run test:e2e:check-handles

Performance Testing

Comprehensive benchmark tools are available in the benchmark/ directory for performance testing and optimization.

Overview

Tree Model Benchmark: Tests timeseries data model using insertTablet API
Table Model Benchmark: Tests relational data model using insertTablet API
Pre-generated Data: Eliminates data generation overhead during testing
Concurrent Clients: Simulates real-world high-concurrency scenarios
Detailed Metrics: Throughput, latency, percentiles (P50, P90, P95, P99)

Quick Start

Test benchmark infrastructure (no IoTDB required):

node benchmark/test-benchmark.js

Run tree model benchmark:

CLIENT_NUMBER=10 DEVICE_NUMBER=100 node benchmark/benchmark-tree.js

Run table model benchmark:

CLIENT_NUMBER=10 DEVICE_NUMBER=100 node benchmark/benchmark-table.js

Key Configuration Parameters

Parameter	Default	Description
`CLIENT_NUMBER`	10	Number of concurrent clients
`DEVICE_NUMBER`	100	Number of devices to simulate
`SENSOR_NUMBER`	10	Number of sensors per device
`BATCH_SIZE_PER_WRITE`	100	Data rows per write operation
`TOTAL_DATA_POINTS`	100000	Total data points to generate
`POOL_MAX_SIZE`	20	Maximum connections in pool

Example: High Concurrency Test

CLIENT_NUMBER=50 \
DEVICE_NUMBER=1000 \
SENSOR_NUMBER=10 \
BATCH_SIZE_PER_WRITE=1000 \
TOTAL_DATA_POINTS=1000000 \
node benchmark/benchmark-tree.js

Performance Metrics

The benchmark reports:

Execution Time: Total test duration
Operations: Total, successful, failed, success rate
Data Points: Total points written
Throughput: Operations/sec, Points/sec
Latency: Min, Max, Average, P50, P90, P95, P99

Sample Output

================================================================================
BENCHMARK RESULTS
================================================================================

[Execution Time]
  Duration:              45.23s (45234ms)

[Operations]
  Total Operations:      1000
  Successful:            998
  Failed:                2
  Success Rate:          99.80%

[Data Points]
  Total Points Written:  100,000

[Throughput]
  Operations/sec:        22.11
  Points/sec:            2,210

[Latency (ms)]
  Min:                   15.23ms
  Max:                   1250.45ms
  Average:               45.23ms
  P50 (Median):          42.15ms
  P90:                   78.45ms
  P95:                   95.23ms
  P99:                   125.67ms
================================================================================

Performance Tuning Tips

Optimize Batch Size: Test different values (100-1000 rows)
Adjust Concurrency: Start with 10-20 clients, adjust based on results
Use Connection Pooling: Set appropriate POOL_MIN_SIZE and POOL_MAX_SIZE
Pre-generate Data: Use cached data for accurate results
Monitor Resources: Watch CPU, memory, disk I/O, and network

For complete documentation, see benchmark/README.md.

Examples

See the examples/ directory for more usage examples:

examples/basic-session.ts - Basic session usage
examples/session-pool.ts - SessionPool usage
examples/table-session-pool.ts - TableSessionPool usage
examples/multi-node.ts - Multi-node configuration
examples/ssl-connection.ts - SSL/TLS connection

Documentation

Comprehensive documentation is available in the docs/ directory:

User Guides

Documentation Index - Complete documentation overview and navigation
Tree Model User Guide - Complete guide for timeseries data model
Table Model User Guide - Complete guide for relational data model
SessionDataSet Guide - Working with query results
Data Types Reference - Complete data type documentation
TypeScript Examples - TypeScript usage guide

Performance Documentation

Performance Documentation Index ⭐ START HERE for performance
Performance Guide - User-focused optimization guide with benchmarks
pg-Inspired Optimizations - Developer-focused implementation details
Performance Analysis Summary - Pool optimization testing analysis
Redirection Design - Client-side redirection optimization

Technical Documentation

Implementation Guide - Architecture and core components
Tablet Interfaces - TreeTablet vs TableTablet guide
Thrift Documentation - Thrift code generation
Build Infrastructure - Build system details

For Contributors

Contributing Guidelines - How to contribute
Debugging E2E Tests - Testing guide
Test Database Reference - Test setup

Additional Resources

Project Status - Implementation status and roadmap
Changelog - Version history
GitHub Workflows - CI/CD documentation
E2E Test Status - End-to-end testing status
Tablet Refactoring Summary - Summary of tablet interface changes

Contributing

We welcome contributions from the community! Whether you're fixing bugs, adding features, improving documentation, or reporting issues, your help is appreciated.

How to Contribute

Fork the repository on GitHub
Create a feature branch from main
Make your changes following our code style guidelines
Add tests for new functionality
Update documentation as needed
Submit a pull request with a clear description

Development Guidelines

Follow existing code style and conventions
Write clear, concise commit messages
Add unit tests for new features
Ensure all tests pass before submitting PR
Update CHANGELOG.md for notable changes
Keep PRs focused on a single feature or fix

Code Review Process

All submissions require review before merging:

Automated tests must pass (CI/CD)
Code review by maintainers
Documentation review (if applicable)
Final approval and merge

Reporting Issues

When reporting bugs, please include:

Node.js version
IoTDB version
Operating system
Steps to reproduce
Expected vs actual behavior
Error messages and stack traces

For detailed contribution guidelines, see CONTRIBUTING.md.

Release Process

This project follows semantic versioning (SemVer) and maintains a regular release cycle.

Version Numbering

Given a version number MAJOR.MINOR.PATCH:

MAJOR: Breaking API changes
MINOR: New features, backward compatible
PATCH: Bug fixes, backward compatible

Release Workflow

1. Pre-release Preparation

Update version and changelog:

# Update version in package.json
npm version [major|minor|patch] --no-git-tag-version

# Update CHANGELOG.md with release notes
# - New features
# - Bug fixes
# - Breaking changes
# - Deprecations

2. Testing

Run comprehensive tests:

# Unit tests
npm run test:unit

# E2E tests (requires IoTDB)
export IOTDB_HOST=localhost
export IOTDB_PORT=6667
npm run test:e2e

# Linting
npm run lint

# Build verification
npm run build

3. Version Tagging

Create and push version tag:

# Commit version bump
git add package.json CHANGELOG.md
git commit -m "chore: bump version to X.Y.Z"

# Create tag
git tag -a vX.Y.Z -m "Release vX.Y.Z"

# Push to remote
git push origin main
git push origin vX.Y.Z

4. Publishing to npm

Build and publish:

# Build production assets
npm run build

# Publish to npm (requires npm account)
npm publish

# For beta/RC releases
npm publish --tag beta

5. GitHub Release

Create GitHub release:

Go to GitHub Releases page
Click “Create a new release”
Select the version tag
Add release title: v X.Y.Z - Release Name
Copy changelog entries to release notes
Attach build artifacts (if applicable)
Publish release

Release Checklist

[ ] All tests passing
[ ] CHANGELOG.md updated
[ ] Version bumped in package.json
[ ] Documentation updated
[ ] Breaking changes documented
[ ] Migration guide (for major versions)
[ ] Git tag created
[ ] npm package published
[ ] GitHub release created
[ ] Release announcement (if major)

Release Schedule

Patch releases: As needed for critical bugs
Minor releases: Monthly or when features are ready
Major releases: When breaking changes are necessary

Beta/RC Releases

For testing before stable release:

# Create beta version
npm version 1.2.0-beta.1 --no-git-tag-version

# Publish with beta tag
npm publish --tag beta

# Install beta version
npm install @iotdb/client@beta

Hotfix Process

For critical production issues:

Create hotfix branch from release tag
Fix the issue
Bump patch version
Tag and publish immediately
Merge back to main

Post-release Tasks

Update documentation site (if applicable)
Announce on project channels
Monitor for issues and feedback
Prepare next release milestone

License

Apache License 2.0

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Apache IoTDB Node.js Client

Table of Contents

Overview

Features

Installation

Requirements

Quick Start

Basic Session Usage

Using Builder Pattern (Recommended)

Query Results with SessionDataSet

SessionPool Usage

Enhanced Pool Metrics

Concurrent Operations (High Throughput)

Batch Insert - Single RPC Call

Concurrent Pool Insertion

Generic Parallel Execution

Utility Functions

Multi-Node Support

Method 1: Using nodeUrls with String Format (Recommended)

Method 2: Using nodeUrls with Object Format (Also Supported)

Method 3: Traditional API (For Same Port)

SSL/TLS Support

TableSessionPool Usage

Redirection Support ✅ (Implemented)

API Reference

Configuration Builders

ConfigBuilder

PoolConfigBuilder

Data Types

Session

Constructor

Methods

SessionPool

Constructor

Methods

TableSessionPool

Constructor

Types

Config

EndPoint

PoolConfig

Tablet

TableTablet (Table Model / Relational Model)

Deprecated Tablet (for backward compatibility)

Data Types

Migration Guide

Upgrading to the New API

Multi-Node with Different Ports

Using Builder Pattern

Explicit Session Management

Technical Architecture

Overview

Core Components

1. Connection Layer (src/connection/Connection.ts)

2. Session Layer (src/client/Session.ts)

3. Pool Layer (src/client/BaseSessionPool.ts, SessionPool.ts, TableSessionPool.ts)

4. Configuration System (src/utils/Config.ts)

5. Tablet Type System

Data Flow

Query Execution Flow

Insert Flow

Thread Safety & Concurrency

Thrift Integration

Memory Management

Error Handling

Configuration Patterns

Constructor Overloading

Builder Pattern

Development

Prerequisites

Development Setup

Build System

Development Workflow

Code Style

Regenerating Thrift Files

Testing

Test Structure

Running Tests

E2E Test Setup

Test Patterns

1. Connection Layer (`src/connection/Connection.ts`)

2. Session Layer (`src/client/Session.ts`)

3. Pool Layer (`src/client/BaseSessionPool.ts`, `SessionPool.ts`, `TableSessionPool.ts`)

4. Configuration System (`src/utils/Config.ts`)