| # Apache IoTDB Node.js Client |
| |
| [](https://www.apache.org/licenses/LICENSE-2.0.html) |
| [](https://www.npmjs.com/package/@iotdb/client) |
| [](https://nodejs.org/) |
| |
| A Node.js client for Apache IoTDB with support for SessionPool and TableSessionPool, providing efficient connection management and comprehensive query capabilities. |
| |
| ## Table of Contents |
| |
| - [Overview](#overview) |
| - [Features](#features) |
| - [Installation](#installation) |
| - [Quick Start](#quick-start) |
| - [Technical Architecture](#technical-architecture) |
| - [API Reference](#api-reference) |
| - [Development](#development) |
| - [Testing](#testing) |
| - [Performance Testing](#performance-testing) |
| - [Examples](#examples) |
| - [Documentation](#documentation) |
| - [Contributing](#contributing) |
| - [Release Process](#release-process) |
| - [License](#license) |
| |
| ## Overview |
| |
| The Apache IoTDB Node.js Client is a high-performance, feature-rich client library for interacting with Apache IoTDB, a time-series database designed for IoT data management. This client provides both tree model (timeseries) and table model (relational) APIs, enabling flexible data management strategies. |
| |
| ## Features |
| |
| - **Session Management**: Single session with query, non-query, and insertTablet operations |
| - **SessionPool**: Connection pooling for high-concurrency scenarios with automatic load balancing |
| - ✨ **Enhanced Metrics**: Comprehensive pool monitoring (totalCount, idleCount, activeCount, waitingCount) |
| - ✨ **Wait Queue Tracking**: Monitor requests waiting for connections |
| - **TableSessionPool**: Specialized pool for table model operations with database context management |
| - **Multi-Node Support**: Round-robin load balancing across multiple IoTDB nodes with failover |
| - **SSL/TLS Support**: Secure connections with customizable SSL options and certificate validation |
| - **TypeScript Support**: Full TypeScript definitions with strict type checking |
| - **Builder Pattern**: Fluent API for elegant configuration management |
| - **Memory Efficient**: SessionDataSet with lazy loading and pagination for large result sets |
| - **Comprehensive Testing**: Unit tests, E2E tests, and benchmark tools included |
| - **Production Ready**: Connection pooling, idle cleanup, health checks, and error handling |
| |
| ## Installation |
| |
| ```bash |
| npm install @iotdb/client |
| ``` |
| |
| ## Requirements |
| |
| - Node.js >= 14.0.0 |
| - Apache IoTDB >= 1.0.0 |
| |
| ## Quick Start |
| |
| ### Basic Session Usage |
| |
| ```typescript |
| import { Session } from '@iotdb/client'; |
| |
| const session = new Session({ |
| host: 'localhost', |
| port: 6667, |
| username: 'root', |
| password: 'root', |
| }); |
| |
| await session.open(); |
| |
| // Execute non-query statement |
| await session.executeNonQueryStatement('CREATE DATABASE root.test'); |
| |
| // Execute query with SessionDataSet (iterator pattern - memory efficient) |
| const dataSet = await session.executeQueryStatement('SELECT * FROM root.test.**'); |
| while (await dataSet.hasNext()) { |
| const row = dataSet.next(); |
| console.log(row.getTimestamp(), row.getFields()); |
| } |
| await dataSet.close(); |
| |
| // Or use toArray() helper for small result sets (loads all into memory) |
| const dataSet2 = await session.executeQueryStatement('SHOW DATABASES'); |
| const allRows = await dataSet2.toArray(); // Returns [[timestamp, ...fields], ...] |
| console.log('All rows:', allRows); |
| |
| // Execute query with custom timeout (30 seconds) |
| const customDataSet = await session.executeQueryStatement('SELECT * FROM root.test.**', 30000); |
| // ... iterate and close |
| |
| // Insert tablet data (supports both tree and table models) |
| // Tree model example: |
| await session.insertTablet({ |
| deviceId: 'root.test.device1', |
| measurements: ['temperature', 'humidity'], |
| dataTypes: [3, 3], // FLOAT |
| timestamps: [Date.now(), Date.now() + 1000], |
| values: [ |
| [25.5, 60.0], |
| [26.0, 61.5], |
| ], |
| }); |
| |
| await session.close(); |
| ``` |
| |
| ### Using Builder Pattern (Recommended) |
| |
| The Builder pattern provides a more elegant and fluent API for configuration: |
| |
| ```typescript |
| import { Session, ConfigBuilder } from '@iotdb/client'; |
| |
| // Build a session configuration |
| const session = new Session( |
| new ConfigBuilder() |
| .host('localhost') |
| .port(6667) |
| .username('root') |
| .password('root') |
| .fetchSize(1024) |
| .timezone('UTC+8') |
| .build() |
| ); |
| |
| await session.open(); |
| // ... use session |
| await session.close(); |
| ``` |
| |
| ### Query Results with SessionDataSet |
| |
| The `executeQueryStatement()` method returns a SessionDataSet for efficient iteration through query results: |
| |
| ```typescript |
| import { Session, SessionDataSet, RowRecord } from '@iotdb/client'; |
| |
| const session = new Session({ |
| host: 'localhost', |
| port: 6667, |
| username: 'root', |
| password: 'root', |
| fetchSize: 1024, // Rows per fetch |
| }); |
| |
| await session.open(); |
| |
| // Execute query and get SessionDataSet |
| const dataSet: SessionDataSet = await session.executeQueryStatement( |
| 'SELECT temperature, humidity FROM root.test.device1' |
| ); |
| |
| // Iterate through results efficiently |
| while (await dataSet.hasNext()) { |
| const row: RowRecord = dataSet.next(); |
| |
| // Access by column name |
| const temp = row.getFloat('temperature'); |
| const humidity = row.getFloat('humidity'); |
| |
| // Access by index |
| const timestamp = row.getTimestamp(); |
| |
| console.log(`${timestamp}: temp=${temp}, humidity=${humidity}`); |
| } |
| |
| // Always close the dataset |
| await dataSet.close(); |
| await session.close(); |
| ``` |
| |
| **Benefits of SessionDataSet:** |
| - ✅ **Memory Efficient**: Only keeps current batch in memory |
| - ✅ **Lazy Loading**: Fetches data on-demand |
| - ✅ **Large Datasets**: Can handle results larger than available RAM |
| - ✅ **Type Safety**: Typed getters prevent errors |
| - ✅ **Resource Management**: Proper cleanup with `close()` |
| |
| See [SessionDataSet Guide](docs/sessiondataset-guide.md) for complete documentation. |
| |
| ### SessionPool Usage |
| |
| ```typescript |
| import { SessionPool } from '@iotdb/client'; |
| |
| const pool = new SessionPool('localhost', 6667, { |
| username: 'root', |
| password: 'root', |
| maxPoolSize: 10, |
| minPoolSize: 2, |
| maxIdleTime: 60000, |
| waitTimeout: 60000, |
| }); |
| |
| await pool.init(); |
| |
| // Execute queries using the pool |
| const result = await pool.executeQueryStatement('SELECT * FROM root.test.**'); |
| |
| // Execute non-query statements |
| await pool.executeNonQueryStatement( |
| 'CREATE TIMESERIES root.test.device1.temperature WITH DATATYPE=FLOAT' |
| ); |
| |
| // Insert data |
| await pool.insertTablet({ |
| deviceId: 'root.test.device1', |
| measurements: ['temperature'], |
| dataTypes: [3], // FLOAT |
| timestamps: [Date.now()], |
| values: [[25.5]], |
| }); |
| |
| // Get pool statistics |
| console.log('Pool size:', pool.getPoolSize()); |
| console.log('Available:', pool.getAvailableSize()); |
| |
| // Enhanced metrics for better monitoring |
| console.log('Waiting requests:', pool.waitingCount); |
| const stats = pool.getPoolStats(); |
| console.log('Comprehensive stats:', stats); |
| // { total, idle, active, waiting, endpoints, redirectCacheSize } |
| |
| await pool.close(); |
| ``` |
| |
| ### Enhanced Pool Metrics |
| |
| The SessionPool provides comprehensive metrics for monitoring pool health: |
| |
| ```typescript |
| const pool = new SessionPool({ |
| host: 'localhost', |
| port: 6667, |
| maxPoolSize: 10, |
| minPoolSize: 2, |
| waitTimeout: 60000, // Timeout for waiting requests (ms) |
| maxIdleTime: 60000, // Idle connection timeout (ms) |
| }); |
| |
| // New metric getters (backward compatible with old methods) |
| console.log('Total connections:', pool.totalCount); |
| console.log('Idle connections:', pool.idleCount); |
| console.log('Active connections:', pool.activeCount); |
| console.log('Waiting requests:', pool.waitingCount); |
| |
| // Comprehensive stats object |
| const stats = pool.getPoolStats(); |
| // { total, idle, active, waiting, endpoints, redirectCacheSize } |
| ``` |
| |
| **Key Features:** |
| - ✅ **Enhanced Metrics**: Comprehensive pool health monitoring |
| - ✅ **Wait Queue Tracking**: Monitor requests waiting for connections |
| - ✅ **Backward Compatible**: All existing code works unchanged |
| |
| await pool.close(); |
| ``` |
| |
| ### Explicit Session Management |
| |
| For more control, you can explicitly get and release sessions from the pool: |
| |
| ```typescript |
| import { SessionPool } from '@iotdb/client'; |
| |
| const pool = new SessionPool('localhost', 6667, { |
| username: 'root', |
| password: 'root', |
| maxPoolSize: 10, |
| }); |
| |
| await pool.init(); |
| |
| // Get a session from the pool |
| const session = await pool.getSession(); |
| |
| try { |
| // Use the session for multiple operations |
| await session.executeNonQueryStatement('CREATE DATABASE root.test'); |
| const result = await session.executeQueryStatement('SHOW DATABASES'); |
| await session.insertTablet({ |
| deviceId: 'root.test.device1', |
| measurements: ['temperature'], |
| dataTypes: [3], |
| timestamps: [Date.now()], |
| values: [[25.5]], |
| }); |
| } finally { |
| // Always release the session back to the pool |
| pool.releaseSession(session); |
| } |
| |
| await pool.close(); |
| ``` |
| |
| ### Concurrent Operations (High Throughput) |
| |
| For high-throughput scenarios, use the concurrent APIs optimized for Node.js: |
| |
| #### Batch Insert - Single RPC Call |
| |
| Insert multiple tablets in a single RPC call (most efficient for tree model): |
| |
| ```typescript |
| import { Session, TreeTablet, TSDataType } from '@iotdb/client'; |
| |
| const session = new Session({ host: 'localhost', port: 6667 }); |
| await session.open(); |
| |
| // Create multiple tablets |
| const tablets = []; |
| for (let i = 0; i < 100; i++) { |
| tablets.push({ |
| deviceId: `root.test.device${i}`, |
| measurements: ['temperature'], |
| dataTypes: [TSDataType.FLOAT], |
| timestamps: [Date.now() + i * 1000], |
| values: [[25.0 + i]], |
| }); |
| } |
| |
| // Insert all tablets in single RPC call |
| await session.insertTablets(tablets); |
| |
| await session.close(); |
| ``` |
| |
| #### Concurrent Pool Insertion |
| |
| Use pool-level concurrent insertion for maximum throughput: |
| |
| ```typescript |
| import { SessionPool } from '@iotdb/client'; |
| |
| const pool = new SessionPool({ |
| host: 'localhost', |
| port: 6667, |
| maxPoolSize: 20, |
| minPoolSize: 5, |
| }); |
| await pool.init(); |
| |
| // Generate tablets |
| const tablets = generateTablets(1000); |
| |
| // Insert concurrently with controlled parallelism |
| // Pre-acquires sessions to avoid contention |
| await pool.insertTabletsParallel(tablets, { concurrency: 20 }); |
| |
| await pool.close(); |
| ``` |
| |
| #### Generic Parallel Execution |
| |
| Execute any operations in parallel using the pool: |
| |
| ```typescript |
| import { SessionPool } from '@iotdb/client'; |
| |
| const pool = new SessionPool({ host: 'localhost', port: 6667, maxPoolSize: 10 }); |
| await pool.init(); |
| |
| const deviceNames = ['d1', 'd2', 'd3', 'd4', 'd5']; |
| |
| // Execute operations in parallel |
| const results = await pool.executeParallel( |
| deviceNames, |
| async (session, deviceName) => { |
| await session.executeNonQueryStatement( |
| `CREATE TIMESERIES root.test.${deviceName}.value WITH DATATYPE=FLOAT` |
| ); |
| return `Created ${deviceName}`; |
| }, |
| { concurrency: 5 } |
| ); |
| |
| console.log(results); // ['Created d1', 'Created d2', ...] |
| |
| await pool.close(); |
| ``` |
| |
| #### Utility Functions |
| |
| Standalone utilities for concurrent execution: |
| |
| ```typescript |
| import { executeConcurrent, chunkArray, createSemaphore } from '@iotdb/client'; |
| |
| // Execute any async operations with controlled concurrency |
| const result = await executeConcurrent( |
| items, |
| async (item) => await processItem(item), |
| { concurrency: 10, logProgressEvery: 100 } |
| ); |
| console.log(`Success: ${result.successCount}, Failed: ${result.failureCount}`); |
| |
| // Split arrays for batch processing |
| const chunks = chunkArray(largeArray, 100); |
| |
| // Fine-grained concurrency control |
| const sem = createSemaphore(5); |
| await sem.acquire(); |
| try { |
| await doWork(); |
| } finally { |
| sem.release(); |
| } |
| ``` |
| |
| ### Multi-Node Support |
| |
| #### Method 1: Using nodeUrls with String Format (Recommended) |
| |
| When nodes have different host:port combinations, use the `nodeUrls` configuration with string array format: |
| |
| ```typescript |
| import { SessionPool, PoolConfigBuilder } from '@iotdb/client'; |
| |
| // Using config object with string array (RECOMMENDED) |
| const pool1 = new SessionPool({ |
| nodeUrls: [ |
| 'node1.example.com:6667', |
| 'node2.example.com:6668', |
| 'node3.example.com:6669', |
| ], |
| username: 'root', |
| password: 'root', |
| maxPoolSize: 15, |
| minPoolSize: 3, |
| }); |
| |
| // Or using Builder pattern with string array |
| const pool2 = new SessionPool( |
| new PoolConfigBuilder() |
| .nodeUrls([ |
| 'node1.example.com:6667', |
| 'node2.example.com:6668', |
| 'node3.example.com:6669', |
| ]) |
| .username('root') |
| .password('root') |
| .maxPoolSize(15) |
| .minPoolSize(3) |
| .build() |
| ); |
| |
| await pool1.init(); |
| // Connections will be distributed across all nodes using round-robin |
| ``` |
| |
| #### Method 2: Using nodeUrls with Object Format (Also Supported) |
| |
| You can also use the object format for `nodeUrls`: |
| |
| ```typescript |
| const pool = new SessionPool({ |
| nodeUrls: [ |
| { host: 'node1.example.com', port: 6667 }, |
| { host: 'node2.example.com', port: 6668 }, |
| { host: 'node3.example.com', port: 6669 }, |
| ], |
| username: 'root', |
| password: 'root', |
| maxPoolSize: 15, |
| }); |
| ``` |
| |
| #### Method 3: Traditional API (For Same Port) |
| |
| When all nodes share the same port: |
| |
| ```typescript |
| import { SessionPool } from '@iotdb/client'; |
| |
| const pool = new SessionPool( |
| ['node1.example.com', 'node2.example.com', 'node3.example.com'], |
| 6667, |
| { |
| username: 'root', |
| password: 'root', |
| maxPoolSize: 15, |
| } |
| ); |
| |
| await pool.init(); |
| // Connections will be distributed across all nodes using round-robin |
| ``` |
| |
| ### SSL/TLS Support |
| |
| ```typescript |
| import { Session } from '@iotdb/client'; |
| import * as fs from 'fs'; |
| |
| const session = new Session({ |
| host: 'localhost', |
| port: 6667, |
| username: 'root', |
| password: 'root', |
| enableSSL: true, |
| sslOptions: { |
| ca: fs.readFileSync('/path/to/ca.crt'), |
| cert: fs.readFileSync('/path/to/client.crt'), |
| key: fs.readFileSync('/path/to/client.key'), |
| rejectUnauthorized: true, |
| }, |
| }); |
| |
| await session.open(); |
| ``` |
| |
| ### TableSessionPool Usage |
| |
| ```typescript |
| import { TableSessionPool } from '@iotdb/client'; |
| |
| const tablePool = new TableSessionPool('localhost', 6667, { |
| username: 'root', |
| password: 'root', |
| database: 'my_database', // Set default database for table model |
| maxPoolSize: 10, |
| minPoolSize: 2, |
| }); |
| |
| await tablePool.init(); |
| |
| // Execute queries in table mode |
| const result = await tablePool.executeQueryStatement('SHOW DATABASES'); |
| |
| await tablePool.close(); |
| ``` |
| |
| ### Redirection Support ✅ (Implemented) |
| |
| **Status: ✅ Fully Implemented** |
| |
| The client now supports automatic write redirection for multi-node IoTDB clusters. When a write operation is sent to a node that doesn't own the device's data, the server responds with a redirect recommendation (status code 400). The client automatically: |
| |
| 1. Caches the device→endpoint mapping |
| 2. Creates/reuses a connection to the optimal endpoint |
| 3. Retries the operation on the correct node |
| 4. Uses the cached mapping for future writes to the same device |
| |
| **Benefits:** |
| - 30-50% throughput improvement by avoiding cross-node data forwarding |
| - Reduced network latency |
| - Better resource utilization |
| |
| **Configuration:** |
| |
| ```typescript |
| import { SessionPool, TableSessionPool } from '@iotdb/client'; |
| |
| // Tree model pool with redirection |
| const treePool = new SessionPool({ |
| nodeUrls: ['192.168.1.100:6667', '192.168.1.101:6667', '192.168.1.102:6667'], |
| username: 'root', |
| password: 'root', |
| maxPoolSize: 10, |
| enableRedirection: true, // Enable redirection (default: true) |
| redirectCacheTTL: 300000, // Cache TTL in ms (default: 5 minutes) |
| }); |
| |
| // Table model pool with redirection |
| const tablePool = new TableSessionPool({ |
| nodeUrls: ['192.168.1.100:6667', '192.168.1.101:6667', '192.168.1.102:6667'], |
| enableRedirection: true, |
| }); |
| ``` |
| |
| **How It Works:** |
| |
| ```typescript |
| // First write to a device - server returns redirect recommendation |
| const tablet = { |
| deviceId: 'root.sg.device1', |
| measurements: ['temperature'], |
| dataTypes: [TSDataType.FLOAT], |
| timestamps: [Date.now()], |
| values: [[25.5]], |
| }; |
| |
| await pool.insertTablet(tablet); |
| // → Writes to Node A (via round-robin) |
| // → Write succeeds! |
| // → Server responds with code 400: "Recommend using Node B for this device in the future" |
| // → Client caches: device1 → Node B (for next write) |
| |
| // Second write to same device - uses cached endpoint |
| await pool.insertTablet({ |
| deviceId: 'root.sg.device1', |
| measurements: ['temperature'], |
| dataTypes: [TSDataType.FLOAT], |
| timestamps: [Date.now() + 1000], |
| values: [[26.0]], |
| }); |
| // → Client checks cache: device1 → Node B |
| // → Writes directly to Node B |
| // → No redirect needed! |
| ``` |
| |
| **Testing:** |
| |
| Redirection support has been tested with the 1C3D (1 ConfigNode, 3 DataNodes) cluster configuration. Run E2E tests: |
| |
| ```bash |
| # Start 1C3D cluster |
| docker-compose -f docker-compose-1c3d.yml up -d |
| |
| # Run redirection tests |
| MULTI_NODE=true npm run test:e2e |
| ``` |
| |
| **Implementation Details:** |
| |
| See [docs/redirection-design.md](docs/redirection-design.md) for detailed design documentation. |
| |
| ## API Reference |
| |
| ### Configuration Builders |
| |
| #### ConfigBuilder |
| |
| Fluent API for building Session configurations: |
| |
| ```typescript |
| import { ConfigBuilder } from '@iotdb/client'; |
| |
| const config = new ConfigBuilder() |
| .host('localhost') |
| .port(6667) |
| .username('root') |
| .password('root') |
| .database('mydb') |
| .timezone('UTC+8') |
| .fetchSize(2048) |
| .enableSSL(false) |
| .build(); |
| ``` |
| |
| **Methods:** |
| - `host(host: string): this` - Set the host |
| - `port(port: number): this` - Set the port |
| - `nodeUrls(nodeUrls: EndPoint[]): this` - Set multiple node URLs |
| - `username(username: string): this` - Set the username |
| - `password(password: string): this` - Set the password |
| - `database(database: string): this` - Set the database |
| - `timezone(timezone: string): this` - Set the timezone |
| - `fetchSize(fetchSize: number): this` - Set the fetch size |
| - `enableSSL(enable: boolean): this` - Enable or disable SSL |
| - `sslOptions(sslOptions: SSLOptions): this` - Set SSL options |
| - `build(): Config` - Build and return the configuration |
| |
| #### PoolConfigBuilder |
| |
| Fluent API for building SessionPool configurations (extends ConfigBuilder): |
| |
| ```typescript |
| import { PoolConfigBuilder } from '@iotdb/client'; |
| |
| const config = new PoolConfigBuilder() |
| .host('localhost') |
| .port(6667) |
| .username('root') |
| .password('root') |
| .maxPoolSize(20) |
| .minPoolSize(5) |
| .maxIdleTime(30000) |
| .waitTimeout(45000) |
| .build(); |
| ``` |
| |
| **Additional Methods:** |
| - `maxPoolSize(size: number): this` - Set maximum pool size |
| - `minPoolSize(size: number): this` - Set minimum pool size |
| - `maxIdleTime(time: number): this` - Set maximum idle time (ms) |
| - `waitTimeout(timeout: number): this` - Set wait timeout (ms) |
| - `build(): PoolConfig` - Build and return the pool configuration |
| |
| ### Data Types |
| |
| IoTDB Node.js client supports all IoTDB data types including BOOLEAN, INT32, INT64, FLOAT, DOUBLE, TEXT, BLOB, STRING, DATE, and TIMESTAMP. See [Data Types Reference](docs/data-types.md) for comprehensive documentation on: |
| - Type mappings between JavaScript and IoTDB |
| - Usage examples for each data type |
| - Best practices and encoding options |
| - Null value handling |
| |
| ### Session |
| |
| #### Constructor |
| |
| **Option 1: Using config object** |
| ```typescript |
| new Session(config: Config) |
| ``` |
| |
| **Option 2: Using Builder pattern** (Recommended) |
| ```typescript |
| new Session(new ConfigBuilder()...build()) |
| ``` |
| |
| The config must include either: |
| - `host` and `port` for single node connection |
| - `nodeUrls` for multi-node connection (uses first node) |
| |
| #### Methods |
| - `async open(): Promise<void>` - Open the session |
| - `async close(): Promise<void>` - Close the session |
| - `async executeQueryStatement(sql: string, timeoutMs?: number): Promise<QueryResult>` - Execute a query with optional timeout (default: 60000ms) |
| - `async executeNonQueryStatement(sql: string): Promise<void>` - Execute a non-query statement |
| - `async insertTablet(tablet: Tablet): Promise<void>` - Insert tablet data |
| - `isOpen(): boolean` - Check if session is open |
| |
| ### SessionPool |
| |
| #### Constructor |
| |
| **Option 1: Traditional API** (Backward compatible) |
| ```typescript |
| new SessionPool(hosts: string | string[], port: number, config?: Partial<PoolConfig>) |
| ``` |
| |
| **Option 2: Using config object with nodeUrls** |
| ```typescript |
| new SessionPool(config: PoolConfig) |
| ``` |
| |
| **Option 3: Using Builder pattern** (Recommended) |
| ```typescript |
| new SessionPool(new PoolConfigBuilder()...build()) |
| ``` |
| |
| #### Methods |
| |
| **Connection Management:** |
| - `async init(): Promise<void>` - Initialize the pool |
| - `async close(): Promise<void>` - Close all connections |
| |
| **Automatic Session Management:** |
| - `async executeQueryStatement(sql: string, timeoutMs?: number): Promise<QueryResult>` - Execute a query with optional timeout (default: 60000ms) |
| - `async executeNonQueryStatement(sql: string): Promise<void>` - Execute a non-query statement |
| - `async insertTablet(tablet: Tablet): Promise<void>` - Insert tablet data |
| |
| **Explicit Session Management:** |
| - `async getSession(): Promise<Session>` - Get a session from the pool (must be released) |
| - `releaseSession(session: Session): void` - Release a session back to the pool |
| |
| **Pool Statistics:** |
| - `getPoolSize(): number` - Get current pool size |
| - `getAvailableSize(): number` - Get available connections |
| - `getInUseSize(): number` - Get number of sessions currently in use |
| |
| ### TableSessionPool |
| |
| Same as SessionPool but optimized for table model operations. Automatically executes `USE DATABASE` when configured with a database. All query methods support the same timeout parameter (default: 60000ms). |
| |
| #### Constructor |
| |
| Same constructor options as SessionPool. |
| |
| ### Types |
| |
| #### Config |
| ```typescript |
| interface Config { |
| host?: string; |
| port?: number; |
| nodeUrls?: string[] | EndPoint[]; // String array format: ["host1:6667", "host2:6668"] |
| username?: string; |
| password?: string; |
| database?: string; |
| timezone?: string; |
| fetchSize?: number; |
| enableSSL?: boolean; |
| sslOptions?: SSLOptions; |
| } |
| ``` |
| |
| **Note:** Either `host`/`port` OR `nodeUrls` must be provided. |
| - Use `nodeUrls` in string array format (e.g., `["host1:6667", "host2:6668"]`) for nodes with different ports (RECOMMENDED) |
| - Object format `[{ host, port }]` is also supported for backward compatibility |
| |
| #### EndPoint |
| ```typescript |
| interface EndPoint { |
| host: string; |
| port: number; |
| } |
| ``` |
| |
| #### PoolConfig |
| ```typescript |
| interface PoolConfig extends Config { |
| maxPoolSize?: number; |
| minPoolSize?: number; |
| maxIdleTime?: number; |
| waitTimeout?: number; |
| } |
| ``` |
| |
| #### Tablet |
| ```typescript |
| #### TreeTablet (Tree Model / Timeseries Model) |
| |
| **Interface (for plain objects):** |
| ```typescript |
| interface ITreeTablet { |
| deviceId: string; // Full path like "root.test.device1" |
| measurements: string[]; // Sensor names |
| dataTypes: number[]; // TSDataType for each measurement |
| timestamps: number[]; // Array of timestamps |
| values: any[][]; // 2D array: [rows][columns] |
| } |
| ``` |
| |
| **Class (with helper methods):** |
| ```typescript |
| import { TreeTablet, TSDataType } from '@iotdb/client'; |
| |
| // Create a tablet |
| const tablet = new TreeTablet( |
| 'root.test.device1', |
| ['temperature', 'humidity'], |
| [TSDataType.FLOAT, TSDataType.DOUBLE] |
| ); |
| |
| // Add rows one at a time using addRow method |
| tablet.addRow(Date.now(), [25.5, 60.0]); |
| tablet.addRow(Date.now() + 1000, [26.0, 61.5]); |
| tablet.addRow(Date.now() + 2000, [26.5, 62.0]); |
| |
| // Insert the tablet |
| await session.insertTablet(tablet); |
| ``` |
| |
| **Alternative: Plain object approach (still supported)** |
| ```typescript |
| await session.insertTablet({ |
| deviceId: 'root.test.device1', |
| measurements: ['temperature', 'humidity'], |
| dataTypes: [TSDataType.FLOAT, TSDataType.DOUBLE], |
| timestamps: [Date.now(), Date.now() + 1000], |
| values: [[25.5, 60.0], [26.0, 61.5]], |
| }); |
| ``` |
| |
| #### TableTablet (Table Model / Relational Model) |
| |
| **Interface (for plain objects):** |
| ```typescript |
| interface ITableTablet { |
| tableName: string; // Table name |
| columnNames: string[]; // Column names (excluding timestamp) |
| columnTypes: number[]; // TSDataType for each column |
| columnCategories: ColumnCategory[]; // Category for each column |
| timestamps: number[]; // Array of timestamps |
| values: any[][]; // 2D array: [rows][columns] |
| } |
| |
| enum ColumnCategory { |
| TAG = 0, // Tag column - indexed for WHERE clause filtering |
| FIELD = 1, // Field column - measurement values |
| ATTRIBUTE = 2, // Attribute column - metadata not indexed |
| TIME = 3, // Time column (reserved for internal use, do not use in columnCategories) |
| } |
| ``` |
| |
| **Class (with helper methods):** |
| ```typescript |
| import { TableTablet, ColumnCategory, TSDataType } from '@iotdb/client'; |
| |
| // Create a tablet |
| const tablet = new TableTablet( |
| 'sensor_data', |
| ['device_id', 'temperature', 'humidity'], |
| [TSDataType.TEXT, TSDataType.FLOAT, TSDataType.DOUBLE], |
| [ColumnCategory.TAG, ColumnCategory.FIELD, ColumnCategory.FIELD] |
| ); |
| |
| // Add rows one at a time using addRow method |
| tablet.addRow(Date.now(), ['device_001', 25.5, 60.0]); |
| tablet.addRow(Date.now() + 1000, ['device_002', 26.0, 61.5]); |
| tablet.addRow(Date.now() + 2000, ['device_003', 26.5, 62.0]); |
| |
| // Insert the tablet |
| await pool.insertTablet(tablet); |
| ``` |
| |
| **Alternative: Plain object approach (still supported)** |
| ```typescript |
| await pool.insertTablet({ |
| tableName: 'sensor_data', |
| columnNames: ['device_id', 'temperature', 'humidity'], |
| columnTypes: [TSDataType.TEXT, TSDataType.FLOAT, TSDataType.DOUBLE], |
| columnCategories: [ColumnCategory.TAG, ColumnCategory.FIELD, ColumnCategory.FIELD], |
| timestamps: [Date.now(), Date.now() + 1000], |
| values: [['device_001', 25.5, 60.0], ['device_002', 26.0, 61.5]], |
| }); |
| ``` |
| |
| **Note:** TIME is reserved for internal use. When specifying columnCategories in TableTablet, only use TAG, FIELD, and ATTRIBUTE. Timestamps are handled separately via the timestamps array. |
| |
| #### Deprecated Tablet (for backward compatibility) |
| ```typescript |
| interface Tablet { |
| deviceId: string; |
| measurements: string[]; |
| dataTypes: number[]; // 0=BOOLEAN, 1=INT32, 2=INT64, 3=FLOAT, 4=DOUBLE, 5=TEXT |
| timestamps: number[]; |
| values: any[][]; |
| } |
| // Note: Use TreeTablet instead |
| ``` |
| ``` |
| |
| #### QueryResult |
| ```typescript |
| interface QueryResult { |
| columns: string[]; |
| dataTypes: string[]; |
| rows: any[][]; |
| queryId?: number; |
| } |
| ``` |
| |
| ## Data Types |
| |
| When using `insertTablet`, specify data types using these constants: |
| |
| - `0` - BOOLEAN |
| - `1` - INT32 |
| - `2` - INT64 |
| - `3` - FLOAT |
| - `4` - DOUBLE |
| - `5` - TEXT |
| |
| ## Migration Guide |
| |
| ### Upgrading to the New API |
| |
| The new version maintains full backward compatibility while adding new features. No changes are required for existing code, but you may want to adopt the new features: |
| |
| #### Multi-Node with Different Ports |
| |
| **Old way** (still works, but limited to same port): |
| ```typescript |
| const pool = new SessionPool( |
| ['node1', 'node2', 'node3'], |
| 6667, |
| { username: 'root', password: 'root' } |
| ); |
| ``` |
| |
| **New way** (supports different ports per node with string format - RECOMMENDED): |
| ```typescript |
| const pool = new SessionPool({ |
| nodeUrls: [ |
| 'node1:6667', |
| 'node2:6668', |
| 'node3:6669', |
| ], |
| username: 'root', |
| password: 'root', |
| }); |
| ``` |
| |
| **Alternative** (object format also supported): |
| ```typescript |
| const pool = new SessionPool({ |
| nodeUrls: [ |
| { host: 'node1', port: 6667 }, |
| { host: 'node2', port: 6668 }, |
| { host: 'node3', port: 6669 }, |
| ], |
| username: 'root', |
| password: 'root', |
| }); |
| ``` |
| |
| #### Using Builder Pattern |
| |
| **Old way** (still works): |
| ```typescript |
| const session = new Session({ |
| host: 'localhost', |
| port: 6667, |
| username: 'root', |
| password: 'root', |
| fetchSize: 2048, |
| }); |
| ``` |
| |
| **New way** (more fluent): |
| ```typescript |
| import { ConfigBuilder } from '@iotdb/client'; |
| |
| const session = new Session( |
| new ConfigBuilder() |
| .host('localhost') |
| .port(6667) |
| .username('root') |
| .password('root') |
| .fetchSize(2048) |
| .build() |
| ); |
| ``` |
| |
| #### Explicit Session Management |
| |
| **Old way** (still works): |
| ```typescript |
| // Pool automatically manages sessions |
| const result = await pool.executeQueryStatement('SELECT ...'); |
| ``` |
| |
| **New way** (more control): |
| ```typescript |
| // Explicitly get and release sessions |
| const session = await pool.getSession(); |
| try { |
| const result1 = await session.executeQueryStatement('SELECT ...'); |
| const result2 = await session.executeQueryStatement('SELECT ...'); |
| // ... multiple operations with same session |
| } finally { |
| pool.releaseSession(session); |
| } |
| ``` |
| |
| ## Technical Architecture |
| |
| ### Overview |
| |
| The IoTDB Node.js client follows a three-layer architecture design, optimized for both single-session and high-concurrency scenarios: |
| |
| ``` |
| ┌─────────────────────────────────────────────────────┐ |
| │ Application Layer (Your Code) │ |
| └─────────────────────────────────────────────────────┘ |
| ↓ |
| ┌─────────────────────────────────────────────────────┐ |
| │ Pool Layer │ |
| │ ┌──────────────────┐ ┌──────────────────────┐ │ |
| │ │ SessionPool │ │ TableSessionPool │ │ |
| │ │ - Load Balance │ │ - Database Context │ │ |
| │ │ - Pool Mgmt │ │ - Pool Mgmt │ │ |
| │ └──────────────────┘ └──────────────────────┘ │ |
| └─────────────────────────────────────────────────────┘ |
| ↓ |
| ┌─────────────────────────────────────────────────────┐ |
| │ Session Layer │ |
| │ ┌──────────────────────────────────────────────┐ │ |
| │ │ Session │ │ |
| │ │ - Query / Non-Query │ │ |
| │ │ - InsertTablet │ │ |
| │ │ - Result Parsing │ │ |
| │ └──────────────────────────────────────────────┘ │ |
| └─────────────────────────────────────────────────────┘ |
| ↓ |
| ┌─────────────────────────────────────────────────────┐ |
| │ Connection Layer │ |
| │ ┌──────────────────────────────────────────────┐ │ |
| │ │ Connection (Thrift) │ │ |
| │ │ - TCP/SSL Transport │ │ |
| │ │ - Session Lifecycle │ │ |
| │ │ - Low-level Protocol │ │ |
| │ └──────────────────────────────────────────────┘ │ |
| └─────────────────────────────────────────────────────┘ |
| ↓ |
| Apache IoTDB |
| ``` |
| |
| ### Core Components |
| |
| #### 1. Connection Layer (`src/connection/Connection.ts`) |
| - Manages low-level Thrift connections over TCP or SSL/TLS |
| - Handles session lifecycle (open/close with sessionId/statementId) |
| - Implements TFramedTransport and TBinaryProtocol |
| - Supports single endpoint connections |
| - **Key Pattern**: One connection = one IoTDB node endpoint |
| |
| #### 2. Session Layer (`src/client/Session.ts`) |
| - High-level API for database operations |
| - Methods: `executeQueryStatement()`, `executeNonQueryStatement()`, `insertTablet()` |
| - Handles query result parsing with SessionDataSet |
| - Supports pagination with configurable fetchSize |
| - **Key Pattern**: Uses first node from nodeUrls for single-session scenarios |
| |
| #### 3. Pool Layer (`src/client/BaseSessionPool.ts`, `SessionPool.ts`, `TableSessionPool.ts`) |
| - Connection pooling with configurable min/max sizes |
| - Round-robin load balancing across multiple endpoints |
| - Automatic idle connection cleanup (maxIdleTime) |
| - Wait queue when pool exhausted (waitTimeout) |
| - Health checks and connection recycling |
| - **Key Pattern**: Distributes connections across all nodes in nodeUrls |
| |
| #### 4. Configuration System (`src/utils/Config.ts`) |
| - Builder pattern for fluent configuration |
| - Support for both old (host/port) and new (nodeUrls) formats |
| |
| #### 5. Tablet Type System |
| |
| The client provides distinct tablet types for tree and table models: |
| |
| **TreeTablet** (Timeseries Model): |
| - `deviceId`: Full path (e.g., "root.sg.device") |
| - `measurements`: Sensor names |
| - `dataTypes`: Type for each measurement |
| - `timestamps` and `values`: Time-series data |
| |
| **TableTablet** (Relational Model): |
| - `tableName`: Table name (not a path) |
| - `columnNames`: All columns including tags, time, fields, attributes |
| - `columnTypes`: Type for each column |
| - `columnCategories`: Category (TAG, TIME, FIELD, ATTRIBUTE) for each column |
| - `timestamps` and `values`: Includes tag values in data |
| |
| **Polymorphic insertTablet():** |
| Both `Session` and `TableSession` use the same method name with runtime type dispatch: |
| ```typescript |
| // Works with TreeTablet |
| await session.insertTablet({ deviceId: '...', measurements: [...], ...}); |
| |
| // Works with TableTablet |
| await tableSession.insertTablet({ tableName: '...', columnNames: [...], ...}); |
| ``` |
| - Type-safe with TypeScript interfaces |
| - Validation and default values |
| |
| ### Data Flow |
| |
| #### Query Execution Flow |
| ``` |
| 1. Application calls pool.executeQueryStatement() |
| 2. Pool acquires available Session (round-robin) |
| 3. Session sends query via Connection to IoTDB |
| 4. IoTDB returns SessionDataSet with queryId |
| 5. SessionDataSet fetches data in batches (fetchSize) |
| 6. Application iterates results with hasNext()/next() |
| 7. Session released back to pool |
| 8. SessionDataSet.close() releases server resources |
| ``` |
| |
| #### Insert Flow |
| ``` |
| 1. Application calls pool.insertTablet() |
| 2. Pool acquires available Session (round-robin) |
| 3. Session serializes Tablet data by column |
| 4. Data sent via Connection to IoTDB |
| 5. IoTDB acknowledges write |
| 6. Session released back to pool |
| ``` |
| |
| ### Thread Safety & Concurrency |
| |
| - **Sessions**: Not thread-safe; use SessionPool for concurrency |
| - **SessionPool**: Thread-safe; internal locking for session management |
| - **Connection Lifecycle**: Managed automatically by pool |
| - **Load Balancing**: Round-robin assignment on session acquisition |
| - **Idle Cleanup**: Background task removes idle connections |
| |
| ### Thrift Integration |
| |
| The client uses Apache Thrift for RPC communication: |
| |
| - **Generated Code**: `src/thrift/generated/` from IoTDB's `.thrift` files |
| - **Protocol**: TBinaryProtocol (compact, efficient) |
| - **Transport**: TFramedTransport (message boundaries) |
| - **SSL Support**: Configurable TLS transport layer |
| - **Version**: Compatible with Apache IoTDB 1.0+ |
| |
| ### Memory Management |
| |
| - **SessionDataSet**: Lazy loading with pagination (default: 1024 rows/fetch) |
| - **Connection Pool**: Bounded size prevents resource exhaustion |
| - **Idle Cleanup**: Automatic connection cleanup after maxIdleTime |
| - **Result Sets**: Must call `close()` to release server resources |
| |
| ### Error Handling |
| |
| - **Connection Errors**: Automatic retry with next node in pool |
| - **Timeout Handling**: Configurable query timeouts (default: 60s) |
| - **Pool Exhaustion**: Wait queue with timeout |
| - **Thrift Errors**: Wrapped in JavaScript errors with stack traces |
| |
| ### Configuration Patterns |
| |
| #### Constructor Overloading |
| SessionPool/TableSessionPool support two constructor signatures: |
| |
| ```typescript |
| // New format (recommended): |
| new SessionPool({ nodeUrls: ["host1:6667", "host2:6667"] }); |
| |
| // Old format (backward compatible): |
| new SessionPool(["host1", "host2"], 6667, { /* options */ }); |
| ``` |
| |
| #### Builder Pattern |
| ```typescript |
| const session = new Session( |
| new ConfigBuilder() |
| .host('localhost') |
| .port(6667) |
| .fetchSize(2048) |
| .build() |
| ); |
| ``` |
| |
| ## Development |
| |
| ### Prerequisites |
| |
| - Node.js >= 14.0.0 |
| - npm >= 6.0.0 |
| - Apache Thrift compiler (optional, for regenerating Thrift files) |
| - Git |
| |
| ### Development Setup |
| |
| 1. Clone the repository: |
| ```bash |
| git clone https://github.com/CritasWang/@iotdb/client.git |
| cd @iotdb/client |
| ``` |
| |
| 2. Install dependencies: |
| ```bash |
| npm install |
| ``` |
| |
| 3. Build the project: |
| ```bash |
| npm run build |
| ``` |
| |
| ### Build System |
| |
| The project uses a two-step build process: |
| |
| 1. **esbuild**: Fast TypeScript compilation |
| - Configured in `esbuild.config.js` |
| - Compiles `src/` to `dist/` directory |
| - Excludes type declaration files |
| |
| 2. **tsc**: Type declaration generation |
| - Generates `.d.ts` files for TypeScript support |
| - Run with `--emitDeclarationOnly` flag |
| - Ensures type safety for consumers |
| |
| 3. **copy:thrift**: Copy generated Thrift files |
| - Copies `.js` files from `src/thrift/generated/` to `dist/thrift/generated/` |
| - Required because Thrift code uses `require()` statements |
| |
| Build commands: |
| ```bash |
| npm run build # Complete build (esbuild + tsc + copy) |
| npm run build:esbuild # Only esbuild compilation |
| npm run build:types # Only type declarations |
| ``` |
| |
| ### Development Workflow |
| |
| 1. **Make changes** in `src/` directory |
| 2. **Build** with `npm run build` |
| 3. **Test** with `npm test` |
| 4. **Lint** with `npm run lint` |
| 5. **Format** with `npm run format` |
| |
| ### Code Style |
| |
| - Use TypeScript strict mode |
| - Follow existing code formatting (Prettier) |
| - Add JSDoc comments for public APIs |
| - Keep functions focused and concise |
| - Use async/await instead of callbacks |
| - Handle errors appropriately |
| - Prefer explicit types over `any` |
| |
| ### Regenerating Thrift Files |
| |
| If you need to update to a newer version of IoTDB's Thrift definitions: |
| |
| 1. Download the latest Thrift files from Apache IoTDB: |
| ```bash |
| git clone --depth 1 https://github.com/apache/iotdb.git /tmp/iotdb |
| ``` |
| |
| 2. Copy the Thrift files: |
| ```bash |
| cp /tmp/iotdb/iotdb-protocol/thrift-datanode/src/main/thrift/client.thrift thrift/ |
| cp /tmp/iotdb/iotdb-protocol/thrift-commons/src/main/thrift/common.thrift thrift/ |
| ``` |
| |
| 3. Regenerate the Node.js client: |
| ```bash |
| npm run generate:thrift |
| ``` |
| |
| 4. Test thoroughly to ensure compatibility |
| |
| ## Testing |
| |
| ### Test Structure |
| |
| ``` |
| tests/ |
| ├── unit/ # Unit tests (fast, no external dependencies) |
| │ ├── Config.test.ts |
| │ ├── Logger.test.ts |
| │ └── ... |
| └── e2e/ # End-to-end tests (require IoTDB) |
| ├── Session.test.ts |
| ├── SessionPool.test.ts |
| ├── TableSessionPool.test.ts |
| └── ... |
| ``` |
| |
| ### Running Tests |
| |
| Run all tests: |
| ```bash |
| npm test |
| ``` |
| |
| Run only unit tests: |
| ```bash |
| npm run test:unit |
| ``` |
| |
| Run only E2E tests (requires IoTDB instance): |
| ```bash |
| export IOTDB_HOST=localhost |
| export IOTDB_PORT=6667 |
| export IOTDB_USER=root |
| export IOTDB_PASSWORD=root |
| npm run test:e2e |
| ``` |
| |
| ### E2E Test Setup |
| |
| E2E tests require a running IoTDB instance. You can use Docker Compose: |
| |
| **Single Node (1c1d):** |
| ```bash |
| docker-compose -f docker-compose-1c1d.yml up -d |
| ``` |
| |
| **3-Node Cluster (3c3d):** |
| ```bash |
| docker-compose -f docker-compose-3c3d.yml up -d |
| ``` |
| |
| **Stop containers:** |
| ```bash |
| docker-compose -f docker-compose-1c1d.yml down |
| ``` |
| |
| ### Test Patterns |
| |
| #### Unit Test Example |
| ```typescript |
| describe('ConfigBuilder', () => { |
| test('should build config with all options', () => { |
| const config = new ConfigBuilder() |
| .host('localhost') |
| .port(6667) |
| .username('root') |
| .password('root') |
| .build(); |
| |
| expect(config.host).toBe('localhost'); |
| expect(config.port).toBe(6667); |
| }); |
| }); |
| ``` |
| |
| #### E2E Test Example |
| ```typescript |
| describe('Session E2E Tests', () => { |
| let session: Session; |
| |
| beforeAll(async () => { |
| session = new Session({ |
| host: process.env.IOTDB_HOST || 'localhost', |
| port: parseInt(process.env.IOTDB_PORT || '6667'), |
| username: process.env.IOTDB_USER || 'root', |
| password: process.env.IOTDB_PASSWORD || 'root', |
| }); |
| await session.open(); |
| }, 60000); // 60s timeout for connection |
| |
| afterAll(async () => { |
| if (session?.isOpen()) { |
| await session.close(); |
| } |
| }); |
| |
| test('should execute query', async () => { |
| if (!session.isOpen()) return; // Skip if no connection |
| |
| const dataSet = await session.executeQueryStatement('SHOW DATABASES'); |
| const rows = await dataSet.toArray(); |
| expect(Array.isArray(rows)).toBe(true); |
| await dataSet.close(); |
| }); |
| }); |
| ``` |
| |
| ### Test Coverage |
| |
| Current test coverage: |
| - Unit tests: Core utilities (Config, Logger, data serialization) |
| - E2E tests: Session, SessionPool, TableSessionPool |
| - All data types tested simultaneously |
| - Multi-node scenarios tested |
| - Pool behavior tested (size limits, timeouts, cleanup) |
| |
| ### Debugging Tests |
| |
| Debug single test: |
| ```bash |
| npm run test:debug |
| ``` |
| |
| Debug E2E tests: |
| ```bash |
| npm run test:e2e:debug |
| ``` |
| |
| Check for open handles: |
| ```bash |
| npm run test:e2e:check-handles |
| ``` |
| |
| ## Performance Testing |
| |
| Comprehensive benchmark tools are available in the `benchmark/` directory for performance testing and optimization. |
| |
| ### Overview |
| |
| - **Tree Model Benchmark**: Tests timeseries data model using `insertTablet` API |
| - **Table Model Benchmark**: Tests relational data model using `insertTablet` API |
| - **Pre-generated Data**: Eliminates data generation overhead during testing |
| - **Concurrent Clients**: Simulates real-world high-concurrency scenarios |
| - **Detailed Metrics**: Throughput, latency, percentiles (P50, P90, P95, P99) |
| |
| ### Quick Start |
| |
| Test benchmark infrastructure (no IoTDB required): |
| ```bash |
| node benchmark/test-benchmark.js |
| ``` |
| |
| Run tree model benchmark: |
| ```bash |
| CLIENT_NUMBER=10 DEVICE_NUMBER=100 node benchmark/benchmark-tree.js |
| ``` |
| |
| Run table model benchmark: |
| ```bash |
| CLIENT_NUMBER=10 DEVICE_NUMBER=100 node benchmark/benchmark-table.js |
| ``` |
| |
| ### Key Configuration Parameters |
| |
| | Parameter | Default | Description | |
| |-----------|---------|-------------| |
| | `CLIENT_NUMBER` | 10 | Number of concurrent clients | |
| | `DEVICE_NUMBER` | 100 | Number of devices to simulate | |
| | `SENSOR_NUMBER` | 10 | Number of sensors per device | |
| | `BATCH_SIZE_PER_WRITE` | 100 | Data rows per write operation | |
| | `TOTAL_DATA_POINTS` | 100000 | Total data points to generate | |
| | `POOL_MAX_SIZE` | 20 | Maximum connections in pool | |
| |
| ### Example: High Concurrency Test |
| |
| ```bash |
| CLIENT_NUMBER=50 \ |
| DEVICE_NUMBER=1000 \ |
| SENSOR_NUMBER=10 \ |
| BATCH_SIZE_PER_WRITE=1000 \ |
| TOTAL_DATA_POINTS=1000000 \ |
| node benchmark/benchmark-tree.js |
| ``` |
| |
| ### Performance Metrics |
| |
| The benchmark reports: |
| - **Execution Time**: Total test duration |
| - **Operations**: Total, successful, failed, success rate |
| - **Data Points**: Total points written |
| - **Throughput**: Operations/sec, Points/sec |
| - **Latency**: Min, Max, Average, P50, P90, P95, P99 |
| |
| ### Sample Output |
| |
| ``` |
| ================================================================================ |
| BENCHMARK RESULTS |
| ================================================================================ |
| |
| [Execution Time] |
| Duration: 45.23s (45234ms) |
| |
| [Operations] |
| Total Operations: 1000 |
| Successful: 998 |
| Failed: 2 |
| Success Rate: 99.80% |
| |
| [Data Points] |
| Total Points Written: 100,000 |
| |
| [Throughput] |
| Operations/sec: 22.11 |
| Points/sec: 2,210 |
| |
| [Latency (ms)] |
| Min: 15.23ms |
| Max: 1250.45ms |
| Average: 45.23ms |
| P50 (Median): 42.15ms |
| P90: 78.45ms |
| P95: 95.23ms |
| P99: 125.67ms |
| ================================================================================ |
| ``` |
| |
| ### Performance Tuning Tips |
| |
| 1. **Optimize Batch Size**: Test different values (100-1000 rows) |
| 2. **Adjust Concurrency**: Start with 10-20 clients, adjust based on results |
| 3. **Use Connection Pooling**: Set appropriate `POOL_MIN_SIZE` and `POOL_MAX_SIZE` |
| 4. **Pre-generate Data**: Use cached data for accurate results |
| 5. **Monitor Resources**: Watch CPU, memory, disk I/O, and network |
| |
| For complete documentation, see [benchmark/README.md](benchmark/README.md). |
| |
| ## Examples |
| |
| See the `examples/` directory for more usage examples: |
| |
| - `examples/basic-session.ts` - Basic session usage |
| - `examples/session-pool.ts` - SessionPool usage |
| - `examples/table-session-pool.ts` - TableSessionPool usage |
| - `examples/multi-node.ts` - Multi-node configuration |
| - `examples/ssl-connection.ts` - SSL/TLS connection |
| |
| ## Documentation |
| |
| Comprehensive documentation is available in the [docs/](docs/) directory: |
| |
| ### User Guides |
| |
| - **[Documentation Index](docs/README.md)** - Complete documentation overview and navigation |
| - **[Tree Model User Guide](docs/user-guide-tree.md)** - Complete guide for timeseries data model |
| - **[Table Model User Guide](docs/user-guide-table.md)** - Complete guide for relational data model |
| - **[SessionDataSet Guide](docs/sessiondataset-guide.md)** - Working with query results |
| - **[Data Types Reference](docs/data-types.md)** - Complete data type documentation |
| - **[TypeScript Examples](docs/typescript-examples.md)** - TypeScript usage guide |
| |
| ### Performance Documentation |
| |
| - **[Performance Documentation Index](docs/PERFORMANCE_INDEX.md)** ⭐ **START HERE for performance** |
| - **[Performance Guide](docs/performance-guide.md)** - User-focused optimization guide with benchmarks |
| - **[pg-Inspired Optimizations](docs/pg-inspired-optimizations.md)** - Developer-focused implementation details |
| - **[Performance Analysis Summary](PERFORMANCE_ANALYSIS_SUMMARY.md)** - Pool optimization testing analysis |
| - **[Redirection Design](docs/redirection-design.md)** - Client-side redirection optimization |
| |
| ### Technical Documentation |
| |
| - **[Implementation Guide](docs/implementation.md)** - Architecture and core components |
| - **[Tablet Interfaces](docs/tablet-interfaces.md)** - TreeTablet vs TableTablet guide |
| - **[Thrift Documentation](docs/thrift.md)** - Thrift code generation |
| - **[Build Infrastructure](docs/development/build-infrastructure.md)** - Build system details |
| |
| ### For Contributors |
| |
| - **[Contributing Guidelines](CONTRIBUTING.md)** - How to contribute |
| - **[Debugging E2E Tests](docs/development/debugging-e2e.md)** - Testing guide |
| - **[Test Database Reference](docs/development/test-database.md)** - Test setup |
| |
| ### Additional Resources |
| |
| - **[Project Status](docs/project-status.md)** - Implementation status and roadmap |
| - **[Changelog](CHANGELOG.md)** - Version history |
| - **[GitHub Workflows](.github/workflows/README.md)** - CI/CD documentation |
| - **[E2E Test Status](E2E_TEST_STATUS.md)** - End-to-end testing status |
| - **[Tablet Refactoring Summary](TABLET_REFACTORING_SUMMARY.md)** - Summary of tablet interface changes |
| |
| ## Contributing |
| |
| We welcome contributions from the community! Whether you're fixing bugs, adding features, improving documentation, or reporting issues, your help is appreciated. |
| |
| ### How to Contribute |
| |
| 1. **Fork the repository** on GitHub |
| 2. **Create a feature branch** from `main` |
| 3. **Make your changes** following our code style guidelines |
| 4. **Add tests** for new functionality |
| 5. **Update documentation** as needed |
| 6. **Submit a pull request** with a clear description |
| |
| ### Development Guidelines |
| |
| - Follow existing code style and conventions |
| - Write clear, concise commit messages |
| - Add unit tests for new features |
| - Ensure all tests pass before submitting PR |
| - Update CHANGELOG.md for notable changes |
| - Keep PRs focused on a single feature or fix |
| |
| ### Code Review Process |
| |
| All submissions require review before merging: |
| 1. Automated tests must pass (CI/CD) |
| 2. Code review by maintainers |
| 3. Documentation review (if applicable) |
| 4. Final approval and merge |
| |
| ### Reporting Issues |
| |
| When reporting bugs, please include: |
| - Node.js version |
| - IoTDB version |
| - Operating system |
| - Steps to reproduce |
| - Expected vs actual behavior |
| - Error messages and stack traces |
| |
| For detailed contribution guidelines, see [CONTRIBUTING.md](CONTRIBUTING.md). |
| |
| ## Release Process |
| |
| This project follows semantic versioning (SemVer) and maintains a regular release cycle. |
| |
| ### Version Numbering |
| |
| Given a version number `MAJOR.MINOR.PATCH`: |
| - **MAJOR**: Breaking API changes |
| - **MINOR**: New features, backward compatible |
| - **PATCH**: Bug fixes, backward compatible |
| |
| ### Release Workflow |
| |
| #### 1. Pre-release Preparation |
| |
| Update version and changelog: |
| ```bash |
| # Update version in package.json |
| npm version [major|minor|patch] --no-git-tag-version |
| |
| # Update CHANGELOG.md with release notes |
| # - New features |
| # - Bug fixes |
| # - Breaking changes |
| # - Deprecations |
| ``` |
| |
| #### 2. Testing |
| |
| Run comprehensive tests: |
| ```bash |
| # Unit tests |
| npm run test:unit |
| |
| # E2E tests (requires IoTDB) |
| export IOTDB_HOST=localhost |
| export IOTDB_PORT=6667 |
| npm run test:e2e |
| |
| # Linting |
| npm run lint |
| |
| # Build verification |
| npm run build |
| ``` |
| |
| #### 3. Version Tagging |
| |
| Create and push version tag: |
| ```bash |
| # Commit version bump |
| git add package.json CHANGELOG.md |
| git commit -m "chore: bump version to X.Y.Z" |
| |
| # Create tag |
| git tag -a vX.Y.Z -m "Release vX.Y.Z" |
| |
| # Push to remote |
| git push origin main |
| git push origin vX.Y.Z |
| ``` |
| |
| #### 4. Publishing to npm |
| |
| Build and publish: |
| ```bash |
| # Build production assets |
| npm run build |
| |
| # Publish to npm (requires npm account) |
| npm publish |
| |
| # For beta/RC releases |
| npm publish --tag beta |
| ``` |
| |
| #### 5. GitHub Release |
| |
| Create GitHub release: |
| 1. Go to GitHub Releases page |
| 2. Click "Create a new release" |
| 3. Select the version tag |
| 4. Add release title: `v X.Y.Z - Release Name` |
| 5. Copy changelog entries to release notes |
| 6. Attach build artifacts (if applicable) |
| 7. Publish release |
| |
| ### Release Checklist |
| |
| - [ ] All tests passing |
| - [ ] CHANGELOG.md updated |
| - [ ] Version bumped in package.json |
| - [ ] Documentation updated |
| - [ ] Breaking changes documented |
| - [ ] Migration guide (for major versions) |
| - [ ] Git tag created |
| - [ ] npm package published |
| - [ ] GitHub release created |
| - [ ] Release announcement (if major) |
| |
| ### Release Schedule |
| |
| - **Patch releases**: As needed for critical bugs |
| - **Minor releases**: Monthly or when features are ready |
| - **Major releases**: When breaking changes are necessary |
| |
| ### Beta/RC Releases |
| |
| For testing before stable release: |
| |
| ```bash |
| # Create beta version |
| npm version 1.2.0-beta.1 --no-git-tag-version |
| |
| # Publish with beta tag |
| npm publish --tag beta |
| |
| # Install beta version |
| npm install @iotdb/client@beta |
| ``` |
| |
| ### Hotfix Process |
| |
| For critical production issues: |
| |
| 1. Create hotfix branch from release tag |
| 2. Fix the issue |
| 3. Bump patch version |
| 4. Tag and publish immediately |
| 5. Merge back to main |
| |
| ### Post-release Tasks |
| |
| - Update documentation site (if applicable) |
| - Announce on project channels |
| - Monitor for issues and feedback |
| - Prepare next release milestone |
| |
| ## License |
| |
| Apache License 2.0 |
| |
| Copyright © 2024 Apache IoTDB |
| |
| Licensed under the Apache License, Version 2.0 (the "License"); |
| you may not use this file except in compliance with the License. |
| You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| |
| ## References |
| |
| - [Apache IoTDB](https://iotdb.apache.org/) |
| - [Apache IoTDB GitHub](https://github.com/apache/iotdb) |
| - [Apache IoTDB Documentation](https://iotdb.apache.org/UserGuide/Master/QuickStart/QuickStart.html) |
| - [Apache IoTDB C# Client](https://github.com/apache/iotdb-client-csharp) |
| - [Apache Thrift](https://thrift.apache.org/) |