Add benchmark suite and tablet API improvements for IoTDB client (#6) * Initial plan * Add benchmark testing tools for IoTDB client Implemented comprehensive benchmark suite inspired by thulab/iot-benchmark: - Configuration module with validation - Data generator for pre-generating test data - Schema manager for metadata registration - Core benchmark engine with metrics collection - Tree model benchmark entry point - Table model benchmark entry point - Comprehensive README with usage examples * Add benchmark infrastructure test and documentation updates - Added test-benchmark.js to verify benchmark infrastructure without IoTDB - Updated benchmark README with testing instructions and prerequisites - Updated main README with benchmark tools section - All benchmark components tested and working - Added debug output to tree benchmark * Add LOOP-based execution and device-session binding Implemented features requested by @CritasWang: - Added LOOP parameter as alternative to TOTAL_DATA_POINTS - Total points = DEVICE_NUMBER × BATCH_SIZE × SENSOR_NUMBER × LOOP - Each loop writes one complete batch for all devices (one tablet per device) - Added ENABLE_DEVICE_SESSION_BINDING to bind devices to sessions - Device-session binding requires DEVICE_NUMBER % POOL_MAX_SIZE == 0 - Updated configuration validation and documentation - Modified data generator to support loop-based mode - Updated benchmark-core with executeConcurrentWithBinding function - Updated both tree and table benchmarks to support new execution model * Add comprehensive documentation: enhanced README, bilingual user guides (tree/table models) (#7) * Initial plan * Enhanced README with comprehensive sections * Add Chinese README translation * Add Tree Model User Guide (English) * Add Table Model User Guide (English) * Add Chinese translations for user guides (tree and table model) --------- * Convert benchmark-table.js to use insertTablet API Changes: - Modified data-generator.js to generate device-oriented structure for table model (matching tree model format) - Updated benchmark-table.js to use insertTablet instead of SQL INSERT statements - Modified generateWorkload to work with device-oriented data structure - Updated executeWrite to use pool.insertTablet() with tablet format - Removed formatValue function (no longer needed) - Updated schema-manager.js to build table schema from device structure - Updated test-benchmark.js to work with new table model data structure - Updated README.md and benchmark/README.md to reflect that both models use insertTablet Both tree and table benchmarks now use the same insertTablet API, with table model automatically using table mode through TableSessionPool. * Add TreeTablet and TableTablet interfaces with separate methods - Added ColumnCategory enum for table model columns (ID, TIME, MEASUREMENT, ATTRIBUTE) - Created TreeTablet interface for tree/timeseries model (uses deviceId) - Created TableTablet interface for table/relational model (uses tableName, includes columnCategories) - Deprecated old Tablet interface for backward compatibility - Added insertTreeTablet() method to Session class - Added insertTableTablet() method to Session class - Updated insertTablet() to use insertTreeTablet internally (backward compatible) - Added insertTreeTablet() to BaseSessionPool - Added insertTableTablet() to TableSessionPool - Updated all exports to include new types and enum - Refactored serializeTabletValues to accept parameters instead of tablet object * Update examples to use new TreeTablet and TableTablet interfaces - Updated basic-session.ts to use insertTreeTablet() - Updated session-pool.ts to use insertTreeTablet() - Updated table-session-pool.ts to use insertTableTablet() with ColumnCategory - Added ColumnCategory import where needed - Demonstrated proper table tablet structure with device_id, timestamp, and measurement columns * Update benchmark code to use new TreeTablet and TableTablet interfaces - Updated benchmark-tree.js to use insertTreeTablet() - Updated benchmark-table.js to use insertTableTablet() with ColumnCategory - Modified table benchmark to construct proper TableTablet with: - device_id as ID column - timestamp as TIME column - measurements as MEASUREMENT columns - Added tableName to workload generation - Updated documentation strings * Fix ColumnCategory enum to match C# and Java clients BREAKING FIX: Corrected ColumnCategory enum values to match official IoTDB clients: - Changed ID to TAG (for device identification columns) - Changed MEASUREMENT to FIELD (for measurement value columns) - Kept ATTRIBUTE unchanged - Reordered TIME to position 3 (was 1) Updated enum definition: - TAG = 0 (was ID = 0) - FIELD = 1 (was MEASUREMENT = 2) - ATTRIBUTE = 2 (was ATTRIBUTE = 3) - TIME = 3 (was TIME = 1) Updated all usage across: - Examples (table-session-pool.ts) - Benchmarks (benchmark-table.js) - E2E tests (TableSessionPool.test.ts) - Session implementation (insertTableTablet logic) This ensures compatibility with C# and Java IoTDB client definitions. * Add comprehensive tablet interfaces documentation - Created docs/tablet-interfaces.md with complete guide for TreeTablet and TableTablet - Documented ColumnCategory enum (TAG, FIELD, ATTRIBUTE, TIME) matching C#/Java - Added examples for both tree and table models - Included migration guide from old Tablet interface - Updated README.md to use insertTreeTablet - Explained column categories: TAG for identification, FIELD for measurements, ATTRIBUTE for metadata, TIME for timestamps Documentation covers: - Interface definitions and structure - Usage examples for both models - ColumnCategory enum explanation - Migration from deprecated Tablet interface - Best practices for each model * Refactor: Create TableSession as composition over inheritance Due to TypeScript type system limitations with method overriding, changing approach to use composition instead of inheritance. TableSession will wrap Session functionality for table-specific operations. * Implement polymorphic insertTablet with runtime type checking - Session.insertTablet now accepts TreeTablet | TableTablet union type - Runtime checking: 'tableName' property → TableTablet, 'deviceId' → TreeTablet - TableSession extends Session without overriding (clean inheritance) - BaseSessionPool and TableSessionPool use same insertTablet signature - Updated all examples, tests, and benchmarks to use unified insertTablet method - Simplified API: one method name for both models, no interface expansion * Update README with comprehensive tablet documentation - Added TreeTablet and TableTablet interface documentation - Updated quick start examples to use unified insertTablet method - Added Tablet Type System section explaining polymorphism - Documented ColumnCategory enum for table model - Added runtime type checking explanation - Updated all code examples to reflect new API - Clarified tree vs table model tablet structures * Add comprehensive refactoring summary documentation Added TABLET_REFACTORING_SUMMARY.md documenting: - Complete architecture overview with class hierarchy - TreeTablet and TableTablet interface details - ColumnCategory enum explanation - Polymorphic insertTablet implementation details - All code changes and design decisions - Migration guide for users - Testing and validation status This completes the tablet refactoring project. * Update user manuals and add ColumnCategory documentation - Updated all 4 user manuals (tree/table, English/Chinese) with: - TreeTablet interface documentation with TSDataType examples - TableTablet interface documentation with ColumnCategory enum - Detailed ColumnCategory enum explanation (TAG, FIELD, ATTRIBUTE, TIME) - Code examples showing both enum and numeric usage - Updated TableSessionPool.test.ts: - Changed insertTableTablet to insertTablet (unified API) - Added detailed comments explaining column categories - Explicit ColumnCategory enum usage with explanations - Updated examples/table-session-pool.ts: - Fixed ColumnCategory.ID to ColumnCategory.TAG (correct enum) - Added explanatory comments for each column category - Both pool and explicit session examples updated * Fix ColumnCategory documentation - clarify TIME is internal only Based on review feedback, corrected ColumnCategory enum usage: Changes made: 1. Updated Session.ts enum comment to clarify TIME is reserved for internal use 2. Removed ColumnCategory.TIME from all user-facing examples and tests 3. Updated all documentation (4 user guides + README + other docs): - English: user-guide-table.md, tablet-interfaces.md, README.md - Chinese: user-guide-table-zh.md - Summary: TABLET_REFACTORING_SUMMARY.md 4. Fixed examples/table-session-pool.ts - removed timestamp column from columnNames/columnCategories 5. Fixed tests/e2e/TableSessionPool.test.ts - removed TIME usage in tests 6. Fixed benchmark/benchmark-table.js - removed TIME from benchmark Key points clarified in documentation: - TIME enum value exists (matches Java/C# clients) but is for internal use only - Users should only use TAG, FIELD, and ATTRIBUTE in columnCategories arrays - Timestamps are handled separately via the timestamps array parameter - Do not include timestamp column in columnNames or columnCategories All code now properly demonstrates that timestamps are separate from columnCategories. * Add comprehensive ColumnCategory usage guide Created detailed documentation explaining: - Why TIME exists but shouldn't be used by users - Correct vs incorrect usage with examples - Purpose of each column category (TAG, FIELD, ATTRIBUTE) - Common mistakes and how to fix them - Migration guide from incorrect usage - Array alignment requirements This addresses the documentation issue identified in the review. * Add TreeTablet and TableTablet classes with addRow method Implemented tablet classes with convenient addRow() method for building tablets row-by-row: Classes added: - TreeTablet class for tree/timeseries model - Constructor: new TreeTablet(deviceId, measurements, dataTypes) - Method: addRow(timestamp, values[]) - Validates values array length matches measurements - TableTablet class for table/relational model - Constructor: new TableTablet(tableName, columnNames, columnTypes, columnCategories) - Method: addRow(timestamp, values[]) - Validates values array length matches columnNames - Validates constructor parameter alignment Backward compatibility: - Kept ITreeTablet and ITableTablet interfaces for plain objects - Session.insertTablet() accepts both classes and plain objects - BaseSessionPool.insertTablet() accepts both classes and plain objects Exports: - TreeTablet and TableTablet exported as classes - ITreeTablet and ITableTablet exported as type interfaces - Updated all imports in BaseSessionPool Build tested and working correctly. * Update examples to demonstrate addRow method Updated all three examples to show the new TreeTablet and TableTablet classes with addRow method: 1. basic-session.ts: - Added TreeTablet import - Demonstrated addRow() usage for building tablets row-by-row - Showed both new class approach and old plain object approach 2. session-pool.ts: - Added TreeTablet import - Updated explicit session example to use addRow() - Shows convenient streaming data pattern 3. table-session-pool.ts: - Added TableTablet import - Updated pool and explicit session examples - Demonstrates addRow() with table model column categories - Shows multiple rows being added incrementally All examples build successfully and demonstrate best practices for using the new tablet classes. * Update README and tree model user guides with addRow documentation Updated documentation to include TreeTablet class with addRow method: 1. README.md: - Changed TreeTablet from interface to showing both interface and class - Added comprehensive examples of TreeTablet.addRow() usage - Added TableTablet.addRow() examples - Showed both class-based and plain object approaches - Clarified backward compatibility 2. docs/user-guide-tree.md: - Updated imports to include TreeTablet and TSDataType - Modified Quick Start section to use TreeTablet class - Enhanced Data Insertion section with detailed addRow examples - Added benefits list for TreeTablet class usage - Kept plain object examples for backward compatibility 3. Built successfully with no errors Next: Update Chinese tree guide and both table model guides (English and Chinese) * docs: Update user guides with TreeTablet/TableTablet class and addRow() method - Updated docs/user-guide-tree-zh.md with TreeTablet class examples - Updated docs/user-guide-table.md with TableTablet class examples - Updated docs/user-guide-table-zh.md with TableTablet class examples - Added import statements for TreeTablet/TableTablet, TSDataType, ColumnCategory - Changed Quick Start sections to use class-based approach with addRow() - Added Data Insertion sections with both class-based (recommended) and plain object (legacy) approaches - Included benefits sections highlighting convenience, type-safety, validation, and streaming-friendly features - Maintained backward compatibility examples * docs: Remove standalone Session usage from tree model user guides - Removed Section 4 "Session API" entirely from both English and Chinese guides - Updated Section 3 "Quick Start" to show only SessionPool example - Renumbered all sections (old 5-10 → new 4-9) - Updated Table of Contents with new section numbers - Replaced all Session usage with SessionPool throughout: - Changed imports to use only SessionPool - Updated all code examples to use pool.init() instead of session.open() - Updated query examples to use pool.executeQueryStatement() - Removed "Session vs SessionPool" comparison from Best Practices - Updated Configuration Builders section to focus on PoolConfigBuilder - Removed "Session Already Exists" error from Troubleshooting - Updated Appendix B API Quick Reference to show only SessionPool methods - Maintained consistency between English and Chinese versions ---------
A Node.js client for Apache IoTDB with support for SessionPool and TableSessionPool, providing efficient connection management and comprehensive query capabilities.
The Apache IoTDB Node.js Client is a high-performance, feature-rich client library for interacting with Apache IoTDB, a time-series database designed for IoT data management. This client provides both tree model (timeseries) and table model (relational) APIs, enabling flexible data management strategies.
npm install iotdb-client-nodejs
import { Session } from 'iotdb-client-nodejs'; const session = new Session({ host: 'localhost', port: 6667, username: 'root', password: 'root', }); await session.open(); // Execute non-query statement await session.executeNonQueryStatement('CREATE DATABASE root.test'); // Execute query with SessionDataSet (iterator pattern - memory efficient) const dataSet = await session.executeQueryStatement('SELECT * FROM root.test.**'); while (await dataSet.hasNext()) { const row = dataSet.next(); console.log(row.getTimestamp(), row.getFields()); } await dataSet.close(); // Or use toArray() helper for small result sets (loads all into memory) const dataSet2 = await session.executeQueryStatement('SHOW DATABASES'); const allRows = await dataSet2.toArray(); // Returns [[timestamp, ...fields], ...] console.log('All rows:', allRows); // Execute query with custom timeout (30 seconds) const customDataSet = await session.executeQueryStatement('SELECT * FROM root.test.**', 30000); // ... iterate and close // Insert tablet data (supports both tree and table models) // Tree model example: await session.insertTablet({ deviceId: 'root.test.device1', measurements: ['temperature', 'humidity'], dataTypes: [3, 3], // FLOAT timestamps: [Date.now(), Date.now() + 1000], values: [ [25.5, 60.0], [26.0, 61.5], ], }); await session.close();
The Builder pattern provides a more elegant and fluent API for configuration:
import { Session, ConfigBuilder } from 'iotdb-client-nodejs'; // Build a session configuration const session = new Session( new ConfigBuilder() .host('localhost') .port(6667) .username('root') .password('root') .fetchSize(1024) .timezone('UTC+8') .build() ); await session.open(); // ... use session await session.close();
The executeQueryStatement() method returns a SessionDataSet for efficient iteration through query results:
import { Session, SessionDataSet, RowRecord } from 'iotdb-client-nodejs'; const session = new Session({ host: 'localhost', port: 6667, username: 'root', password: 'root', fetchSize: 1024, // Rows per fetch }); await session.open(); // Execute query and get SessionDataSet const dataSet: SessionDataSet = await session.executeQueryStatement( 'SELECT temperature, humidity FROM root.test.device1' ); // Iterate through results efficiently while (await dataSet.hasNext()) { const row: RowRecord = dataSet.next(); // Access by column name const temp = row.getFloat('temperature'); const humidity = row.getFloat('humidity'); // Access by index const timestamp = row.getTimestamp(); console.log(`${timestamp}: temp=${temp}, humidity=${humidity}`); } // Always close the dataset await dataSet.close(); await session.close();
Benefits of SessionDataSet:
close()See SessionDataSet Guide for complete documentation.
import { SessionPool } from 'iotdb-client-nodejs'; const pool = new SessionPool('localhost', 6667, { username: 'root', password: 'root', maxPoolSize: 10, minPoolSize: 2, maxIdleTime: 60000, waitTimeout: 60000, }); await pool.init(); // Execute queries using the pool const result = await pool.executeQueryStatement('SELECT * FROM root.test.**'); // Execute non-query statements await pool.executeNonQueryStatement( 'CREATE TIMESERIES root.test.device1.temperature WITH DATATYPE=FLOAT' ); // Insert data await pool.insertTablet({ deviceId: 'root.test.device1', measurements: ['temperature'], dataTypes: [3], // FLOAT timestamps: [Date.now()], values: [[25.5]], }); // Get pool statistics console.log('Pool size:', pool.getPoolSize()); console.log('Available:', pool.getAvailableSize()); await pool.close();
For more control, you can explicitly get and release sessions from the pool:
import { SessionPool } from 'iotdb-client-nodejs'; const pool = new SessionPool('localhost', 6667, { username: 'root', password: 'root', maxPoolSize: 10, }); await pool.init(); // Get a session from the pool const session = await pool.getSession(); try { // Use the session for multiple operations await session.executeNonQueryStatement('CREATE DATABASE root.test'); const result = await session.executeQueryStatement('SHOW DATABASES'); await session.insertTablet({ deviceId: 'root.test.device1', measurements: ['temperature'], dataTypes: [3], timestamps: [Date.now()], values: [[25.5]], }); } finally { // Always release the session back to the pool pool.releaseSession(session); } await pool.close();
When nodes have different host:port combinations, use the nodeUrls configuration with string array format:
import { SessionPool, PoolConfigBuilder } from 'iotdb-client-nodejs'; // Using config object with string array (RECOMMENDED) const pool1 = new SessionPool({ nodeUrls: [ 'node1.example.com:6667', 'node2.example.com:6668', 'node3.example.com:6669', ], username: 'root', password: 'root', maxPoolSize: 15, minPoolSize: 3, }); // Or using Builder pattern with string array const pool2 = new SessionPool( new PoolConfigBuilder() .nodeUrls([ 'node1.example.com:6667', 'node2.example.com:6668', 'node3.example.com:6669', ]) .username('root') .password('root') .maxPoolSize(15) .minPoolSize(3) .build() ); await pool1.init(); // Connections will be distributed across all nodes using round-robin
You can also use the object format for nodeUrls:
const pool = new SessionPool({ nodeUrls: [ { host: 'node1.example.com', port: 6667 }, { host: 'node2.example.com', port: 6668 }, { host: 'node3.example.com', port: 6669 }, ], username: 'root', password: 'root', maxPoolSize: 15, });
When all nodes share the same port:
import { SessionPool } from 'iotdb-client-nodejs'; const pool = new SessionPool( ['node1.example.com', 'node2.example.com', 'node3.example.com'], 6667, { username: 'root', password: 'root', maxPoolSize: 15, } ); await pool.init(); // Connections will be distributed across all nodes using round-robin
import { Session } from 'iotdb-client-nodejs'; import * as fs from 'fs'; const session = new Session({ host: 'localhost', port: 6667, username: 'root', password: 'root', enableSSL: true, sslOptions: { ca: fs.readFileSync('/path/to/ca.crt'), cert: fs.readFileSync('/path/to/client.crt'), key: fs.readFileSync('/path/to/client.key'), rejectUnauthorized: true, }, }); await session.open();
import { TableSessionPool } from 'iotdb-client-nodejs'; const tablePool = new TableSessionPool('localhost', 6667, { username: 'root', password: 'root', database: 'my_database', // Set default database for table model maxPoolSize: 10, minPoolSize: 2, }); await tablePool.init(); // Execute queries in table mode const result = await tablePool.executeQueryStatement('SHOW DATABASES'); await tablePool.close();
Fluent API for building Session configurations:
import { ConfigBuilder } from 'iotdb-client-nodejs'; const config = new ConfigBuilder() .host('localhost') .port(6667) .username('root') .password('root') .database('mydb') .timezone('UTC+8') .fetchSize(2048) .enableSSL(false) .build();
Methods:
host(host: string): this - Set the hostport(port: number): this - Set the portnodeUrls(nodeUrls: EndPoint[]): this - Set multiple node URLsusername(username: string): this - Set the usernamepassword(password: string): this - Set the passworddatabase(database: string): this - Set the databasetimezone(timezone: string): this - Set the timezonefetchSize(fetchSize: number): this - Set the fetch sizeenableSSL(enable: boolean): this - Enable or disable SSLsslOptions(sslOptions: SSLOptions): this - Set SSL optionsbuild(): Config - Build and return the configurationFluent API for building SessionPool configurations (extends ConfigBuilder):
import { PoolConfigBuilder } from 'iotdb-client-nodejs'; const config = new PoolConfigBuilder() .host('localhost') .port(6667) .username('root') .password('root') .maxPoolSize(20) .minPoolSize(5) .maxIdleTime(30000) .waitTimeout(45000) .build();
Additional Methods:
maxPoolSize(size: number): this - Set maximum pool sizeminPoolSize(size: number): this - Set minimum pool sizemaxIdleTime(time: number): this - Set maximum idle time (ms)waitTimeout(timeout: number): this - Set wait timeout (ms)build(): PoolConfig - Build and return the pool configurationIoTDB Node.js client supports all IoTDB data types including BOOLEAN, INT32, INT64, FLOAT, DOUBLE, TEXT, BLOB, STRING, DATE, and TIMESTAMP. See DATA_TYPES.md for comprehensive documentation on:
Option 1: Using config object
new Session(config: Config)
Option 2: Using Builder pattern (Recommended)
new Session(new ConfigBuilder()...build())
The config must include either:
host and port for single node connectionnodeUrls for multi-node connection (uses first node)async open(): Promise<void> - Open the sessionasync close(): Promise<void> - Close the sessionasync executeQueryStatement(sql: string, timeoutMs?: number): Promise<QueryResult> - Execute a query with optional timeout (default: 60000ms)async executeNonQueryStatement(sql: string): Promise<void> - Execute a non-query statementasync insertTablet(tablet: Tablet): Promise<void> - Insert tablet dataisOpen(): boolean - Check if session is openOption 1: Traditional API (Backward compatible)
new SessionPool(hosts: string | string[], port: number, config?: Partial<PoolConfig>)
Option 2: Using config object with nodeUrls
new SessionPool(config: PoolConfig)
Option 3: Using Builder pattern (Recommended)
new SessionPool(new PoolConfigBuilder()...build())
Connection Management:
async init(): Promise<void> - Initialize the poolasync close(): Promise<void> - Close all connectionsAutomatic Session Management:
async executeQueryStatement(sql: string, timeoutMs?: number): Promise<QueryResult> - Execute a query with optional timeout (default: 60000ms)async executeNonQueryStatement(sql: string): Promise<void> - Execute a non-query statementasync insertTablet(tablet: Tablet): Promise<void> - Insert tablet dataExplicit Session Management:
async getSession(): Promise<Session> - Get a session from the pool (must be released)releaseSession(session: Session): void - Release a session back to the poolPool Statistics:
getPoolSize(): number - Get current pool sizegetAvailableSize(): number - Get available connectionsgetInUseSize(): number - Get number of sessions currently in useSame as SessionPool but optimized for table model operations. Automatically executes USE DATABASE when configured with a database. All query methods support the same timeout parameter (default: 60000ms).
Same constructor options as SessionPool.
interface Config { host?: string; port?: number; nodeUrls?: string[] | EndPoint[]; // String array format: ["host1:6667", "host2:6668"] username?: string; password?: string; database?: string; timezone?: string; fetchSize?: number; enableSSL?: boolean; sslOptions?: SSLOptions; }
Note: Either host/port OR nodeUrls must be provided.
nodeUrls in string array format (e.g., ["host1:6667", "host2:6668"]) for nodes with different ports (RECOMMENDED)[{ host, port }] is also supported for backward compatibilityinterface EndPoint { host: string; port: number; }
interface PoolConfig extends Config { maxPoolSize?: number; minPoolSize?: number; maxIdleTime?: number; waitTimeout?: number; }
#### TreeTablet (Tree Model / Timeseries Model) **Interface (for plain objects):** ```typescript interface ITreeTablet { deviceId: string; // Full path like "root.test.device1" measurements: string[]; // Sensor names dataTypes: number[]; // TSDataType for each measurement timestamps: number[]; // Array of timestamps values: any[][]; // 2D array: [rows][columns] }
Class (with helper methods):
import { TreeTablet, TSDataType } from 'iotdb-client-nodejs'; // Create a tablet const tablet = new TreeTablet( 'root.test.device1', ['temperature', 'humidity'], [TSDataType.FLOAT, TSDataType.DOUBLE] ); // Add rows one at a time using addRow method tablet.addRow(Date.now(), [25.5, 60.0]); tablet.addRow(Date.now() + 1000, [26.0, 61.5]); tablet.addRow(Date.now() + 2000, [26.5, 62.0]); // Insert the tablet await session.insertTablet(tablet);
Alternative: Plain object approach (still supported)
await session.insertTablet({ deviceId: 'root.test.device1', measurements: ['temperature', 'humidity'], dataTypes: [TSDataType.FLOAT, TSDataType.DOUBLE], timestamps: [Date.now(), Date.now() + 1000], values: [[25.5, 60.0], [26.0, 61.5]], });
Interface (for plain objects):
interface ITableTablet { tableName: string; // Table name columnNames: string[]; // Column names (excluding timestamp) columnTypes: number[]; // TSDataType for each column columnCategories: ColumnCategory[]; // Category for each column timestamps: number[]; // Array of timestamps values: any[][]; // 2D array: [rows][columns] } enum ColumnCategory { TAG = 0, // Tag column - indexed for WHERE clause filtering FIELD = 1, // Field column - measurement values ATTRIBUTE = 2, // Attribute column - metadata not indexed TIME = 3, // Time column (reserved for internal use, do not use in columnCategories) }
Class (with helper methods):
import { TableTablet, ColumnCategory, TSDataType } from 'iotdb-client-nodejs'; // Create a tablet const tablet = new TableTablet( 'sensor_data', ['device_id', 'temperature', 'humidity'], [TSDataType.TEXT, TSDataType.FLOAT, TSDataType.DOUBLE], [ColumnCategory.TAG, ColumnCategory.FIELD, ColumnCategory.FIELD] ); // Add rows one at a time using addRow method tablet.addRow(Date.now(), ['device_001', 25.5, 60.0]); tablet.addRow(Date.now() + 1000, ['device_002', 26.0, 61.5]); tablet.addRow(Date.now() + 2000, ['device_003', 26.5, 62.0]); // Insert the tablet await pool.insertTablet(tablet);
Alternative: Plain object approach (still supported)
await pool.insertTablet({ tableName: 'sensor_data', columnNames: ['device_id', 'temperature', 'humidity'], columnTypes: [TSDataType.TEXT, TSDataType.FLOAT, TSDataType.DOUBLE], columnCategories: [ColumnCategory.TAG, ColumnCategory.FIELD, ColumnCategory.FIELD], timestamps: [Date.now(), Date.now() + 1000], values: [['device_001', 25.5, 60.0], ['device_002', 26.0, 61.5]], });
Note: TIME is reserved for internal use. When specifying columnCategories in TableTablet, only use TAG, FIELD, and ATTRIBUTE. Timestamps are handled separately via the timestamps array.
interface Tablet { deviceId: string; measurements: string[]; dataTypes: number[]; // 0=BOOLEAN, 1=INT32, 2=INT64, 3=FLOAT, 4=DOUBLE, 5=TEXT timestamps: number[]; values: any[][]; } // Note: Use TreeTablet instead
#### QueryResult ```typescript interface QueryResult { columns: string[]; dataTypes: string[]; rows: any[][]; queryId?: number; }
When using insertTablet, specify data types using these constants:
0 - BOOLEAN1 - INT322 - INT643 - FLOAT4 - DOUBLE5 - TEXTThe new version maintains full backward compatibility while adding new features. No changes are required for existing code, but you may want to adopt the new features:
Old way (still works, but limited to same port):
const pool = new SessionPool( ['node1', 'node2', 'node3'], 6667, { username: 'root', password: 'root' } );
New way (supports different ports per node with string format - RECOMMENDED):
const pool = new SessionPool({ nodeUrls: [ 'node1:6667', 'node2:6668', 'node3:6669', ], username: 'root', password: 'root', });
Alternative (object format also supported):
const pool = new SessionPool({ nodeUrls: [ { host: 'node1', port: 6667 }, { host: 'node2', port: 6668 }, { host: 'node3', port: 6669 }, ], username: 'root', password: 'root', });
Old way (still works):
const session = new Session({ host: 'localhost', port: 6667, username: 'root', password: 'root', fetchSize: 2048, });
New way (more fluent):
import { ConfigBuilder } from 'iotdb-client-nodejs'; const session = new Session( new ConfigBuilder() .host('localhost') .port(6667) .username('root') .password('root') .fetchSize(2048) .build() );
Old way (still works):
// Pool automatically manages sessions const result = await pool.executeQueryStatement('SELECT ...');
New way (more control):
// Explicitly get and release sessions const session = await pool.getSession(); try { const result1 = await session.executeQueryStatement('SELECT ...'); const result2 = await session.executeQueryStatement('SELECT ...'); // ... multiple operations with same session } finally { pool.releaseSession(session); }
The IoTDB Node.js client follows a three-layer architecture design, optimized for both single-session and high-concurrency scenarios:
┌─────────────────────────────────────────────────────┐
│ Application Layer (Your Code) │
└─────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────┐
│ Pool Layer │
│ ┌──────────────────┐ ┌──────────────────────┐ │
│ │ SessionPool │ │ TableSessionPool │ │
│ │ - Load Balance │ │ - Database Context │ │
│ │ - Pool Mgmt │ │ - Pool Mgmt │ │
│ └──────────────────┘ └──────────────────────┘ │
└─────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────┐
│ Session Layer │
│ ┌──────────────────────────────────────────────┐ │
│ │ Session │ │
│ │ - Query / Non-Query │ │
│ │ - InsertTablet │ │
│ │ - Result Parsing │ │
│ └──────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────┐
│ Connection Layer │
│ ┌──────────────────────────────────────────────┐ │
│ │ Connection (Thrift) │ │
│ │ - TCP/SSL Transport │ │
│ │ - Session Lifecycle │ │
│ │ - Low-level Protocol │ │
│ └──────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
↓
Apache IoTDB
src/connection/Connection.ts)src/client/Session.ts)executeQueryStatement(), executeNonQueryStatement(), insertTablet()src/client/BaseSessionPool.ts, SessionPool.ts, TableSessionPool.ts)src/utils/Config.ts)The client provides distinct tablet types for tree and table models:
TreeTablet (Timeseries Model):
deviceId: Full path (e.g., “root.sg.device”)measurements: Sensor namesdataTypes: Type for each measurementtimestamps and values: Time-series dataTableTablet (Relational Model):
tableName: Table name (not a path)columnNames: All columns including tags, time, fields, attributescolumnTypes: Type for each columncolumnCategories: Category (TAG, TIME, FIELD, ATTRIBUTE) for each columntimestamps and values: Includes tag values in dataPolymorphic insertTablet(): Both Session and TableSession use the same method name with runtime type dispatch:
// Works with TreeTablet await session.insertTablet({ deviceId: '...', measurements: [...], ...}); // Works with TableTablet await tableSession.insertTablet({ tableName: '...', columnNames: [...], ...});
1. Application calls pool.executeQueryStatement() 2. Pool acquires available Session (round-robin) 3. Session sends query via Connection to IoTDB 4. IoTDB returns SessionDataSet with queryId 5. SessionDataSet fetches data in batches (fetchSize) 6. Application iterates results with hasNext()/next() 7. Session released back to pool 8. SessionDataSet.close() releases server resources
1. Application calls pool.insertTablet() 2. Pool acquires available Session (round-robin) 3. Session serializes Tablet data by column 4. Data sent via Connection to IoTDB 5. IoTDB acknowledges write 6. Session released back to pool
The client uses Apache Thrift for RPC communication:
src/thrift/generated/ from IoTDB's .thrift filesclose() to release server resourcesSessionPool/TableSessionPool support two constructor signatures:
// New format (recommended): new SessionPool({ nodeUrls: ["host1:6667", "host2:6667"] }); // Old format (backward compatible): new SessionPool(["host1", "host2"], 6667, { /* options */ });
const session = new Session( new ConfigBuilder() .host('localhost') .port(6667) .fetchSize(2048) .build() );
git clone https://github.com/CritasWang/iotdb-client-nodejs.git cd iotdb-client-nodejs
npm install
npm run build
The project uses a two-step build process:
esbuild: Fast TypeScript compilation
esbuild.config.jssrc/ to dist/ directorytsc: Type declaration generation
.d.ts files for TypeScript support--emitDeclarationOnly flagcopy:thrift: Copy generated Thrift files
.js files from src/thrift/generated/ to dist/thrift/generated/require() statementsBuild commands:
npm run build # Complete build (esbuild + tsc + copy) npm run build:esbuild # Only esbuild compilation npm run build:types # Only type declarations
src/ directorynpm run buildnpm testnpm run lintnpm run formatanyIf you need to update to a newer version of IoTDB's Thrift definitions:
git clone --depth 1 https://github.com/apache/iotdb.git /tmp/iotdb
cp /tmp/iotdb/iotdb-protocol/thrift-datanode/src/main/thrift/client.thrift thrift/ cp /tmp/iotdb/iotdb-protocol/thrift-commons/src/main/thrift/common.thrift thrift/
npm run generate:thrift
tests/
├── unit/ # Unit tests (fast, no external dependencies)
│ ├── Config.test.ts
│ ├── Logger.test.ts
│ └── ...
└── e2e/ # End-to-end tests (require IoTDB)
├── Session.test.ts
├── SessionPool.test.ts
├── TableSessionPool.test.ts
└── ...
Run all tests:
npm test
Run only unit tests:
npm run test:unit
Run only E2E tests (requires IoTDB instance):
export IOTDB_HOST=localhost export IOTDB_PORT=6667 export IOTDB_USER=root export IOTDB_PASSWORD=root npm run test:e2e
E2E tests require a running IoTDB instance. You can use Docker Compose:
Single Node (1c1d):
docker-compose -f docker-compose-1c1d.yml up -d
3-Node Cluster (3c3d):
docker-compose -f docker-compose-3c3d.yml up -d
Stop containers:
docker-compose -f docker-compose-1c1d.yml down
describe('ConfigBuilder', () => { test('should build config with all options', () => { const config = new ConfigBuilder() .host('localhost') .port(6667) .username('root') .password('root') .build(); expect(config.host).toBe('localhost'); expect(config.port).toBe(6667); }); });
describe('Session E2E Tests', () => { let session: Session; beforeAll(async () => { session = new Session({ host: process.env.IOTDB_HOST || 'localhost', port: parseInt(process.env.IOTDB_PORT || '6667'), username: process.env.IOTDB_USER || 'root', password: process.env.IOTDB_PASSWORD || 'root', }); await session.open(); }, 60000); // 60s timeout for connection afterAll(async () => { if (session?.isOpen()) { await session.close(); } }); test('should execute query', async () => { if (!session.isOpen()) return; // Skip if no connection const dataSet = await session.executeQueryStatement('SHOW DATABASES'); const rows = await dataSet.toArray(); expect(Array.isArray(rows)).toBe(true); await dataSet.close(); }); });
Current test coverage:
Debug single test:
npm run test:debug
Debug E2E tests:
npm run test:e2e:debug
Check for open handles:
npm run test:e2e:check-handles
Comprehensive benchmark tools are available in the benchmark/ directory for performance testing and optimization.
insertTablet APIinsertTablet APITest benchmark infrastructure (no IoTDB required):
node benchmark/test-benchmark.js
Run tree model benchmark:
CLIENT_NUMBER=10 DEVICE_NUMBER=100 node benchmark/benchmark-tree.js
Run table model benchmark:
CLIENT_NUMBER=10 DEVICE_NUMBER=100 node benchmark/benchmark-table.js
| Parameter | Default | Description |
|---|---|---|
CLIENT_NUMBER | 10 | Number of concurrent clients |
DEVICE_NUMBER | 100 | Number of devices to simulate |
SENSOR_NUMBER | 10 | Number of sensors per device |
BATCH_SIZE_PER_WRITE | 100 | Data rows per write operation |
TOTAL_DATA_POINTS | 100000 | Total data points to generate |
POOL_MAX_SIZE | 20 | Maximum connections in pool |
CLIENT_NUMBER=50 \ DEVICE_NUMBER=1000 \ SENSOR_NUMBER=10 \ BATCH_SIZE_PER_WRITE=1000 \ TOTAL_DATA_POINTS=1000000 \ node benchmark/benchmark-tree.js
The benchmark reports:
================================================================================ BENCHMARK RESULTS ================================================================================ [Execution Time] Duration: 45.23s (45234ms) [Operations] Total Operations: 1000 Successful: 998 Failed: 2 Success Rate: 99.80% [Data Points] Total Points Written: 100,000 [Throughput] Operations/sec: 22.11 Points/sec: 2,210 [Latency (ms)] Min: 15.23ms Max: 1250.45ms Average: 45.23ms P50 (Median): 42.15ms P90: 78.45ms P95: 95.23ms P99: 125.67ms ================================================================================
POOL_MIN_SIZE and POOL_MAX_SIZEFor complete documentation, see benchmark/README.md.
See the examples/ directory for more usage examples:
examples/basic-session.ts - Basic session usageexamples/session-pool.ts - SessionPool usageexamples/table-session-pool.ts - TableSessionPool usageexamples/multi-node.ts - Multi-node configurationexamples/ssl-connection.ts - SSL/TLS connectionComprehensive documentation is available in the docs/ directory:
We welcome contributions from the community! Whether you're fixing bugs, adding features, improving documentation, or reporting issues, your help is appreciated.
mainAll submissions require review before merging:
When reporting bugs, please include:
For detailed contribution guidelines, see CONTRIBUTING.md.
This project follows semantic versioning (SemVer) and maintains a regular release cycle.
Given a version number MAJOR.MINOR.PATCH:
Update version and changelog:
# Update version in package.json npm version [major|minor|patch] --no-git-tag-version # Update CHANGELOG.md with release notes # - New features # - Bug fixes # - Breaking changes # - Deprecations
Run comprehensive tests:
# Unit tests npm run test:unit # E2E tests (requires IoTDB) export IOTDB_HOST=localhost export IOTDB_PORT=6667 npm run test:e2e # Linting npm run lint # Build verification npm run build
Create and push version tag:
# Commit version bump git add package.json CHANGELOG.md git commit -m "chore: bump version to X.Y.Z" # Create tag git tag -a vX.Y.Z -m "Release vX.Y.Z" # Push to remote git push origin main git push origin vX.Y.Z
Build and publish:
# Build production assets npm run build # Publish to npm (requires npm account) npm publish # For beta/RC releases npm publish --tag beta
Create GitHub release:
v X.Y.Z - Release NameFor testing before stable release:
# Create beta version npm version 1.2.0-beta.1 --no-git-tag-version # Publish with beta tag npm publish --tag beta # Install beta version npm install iotdb-client-nodejs@beta
For critical production issues:
Apache License 2.0
Copyright © 2024 Apache IoTDB
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.