benchmark/README.md - iotdb-client-nodejs - Git at Google

 # IoTDB Client Benchmark Tools

 Comprehensive performance testing tools for Apache IoTDB Node.js client, inspired by the [thulab/iot-benchmark](https://github.com/thulab/iot-benchmark) project.

 ## Overview

 This benchmark suite provides specialized tools for testing IoTDB write performance:

 - **Tree Model Benchmark** (`benchmark-tree.js`) - Tests timeseries data model
 - **Table Model Benchmark** (`benchmark-table.js`) - Tests relational data model
 - **Multi-Process Cluster** (`benchmark-table-cluster.js`) - **NEW** Multi-process mode for maximum throughput
 - **API Comparison** (`benchmark-comparison.js`, `benchmark-table-comparison.js`) - Compare insertion methods

 ### Performance Summary

 | Mode | Configuration | Throughput | Notes |
 |------|---------------|------------|-------|
 | Single Process | 20 clients, 200 devices | 4.28M pts/s | Best single-process |
 | **Multi-Process** | **8 workers × 10 clients** | **5.42M pts/s** | **Recommended** |
 | Java iot-benchmark | Similar config | ~60M pts/s | Reference |

 ### Key Features

 ✅ **Multi-Process Cluster Mode** - Overcomes Node.js single-thread limitation
 ✅ **Pre-generated Test Data** - Eliminates data generation overhead
 ✅ **Memory-Optimized** - Streaming batch processing for large-scale tests
 ✅ **Flexible Configuration** - Extensive parameters for customizing scenarios
 ✅ **Detailed Metrics** - Comprehensive statistics including latency percentiles

 ## Quick Start

 ### Prerequisites

 1. Node.js >= 14.0.0
 2. Running IoTDB instance (v1.0+)
 3. Built IoTDB client library

 **Note:** The benchmark tools require a working IoTDB instance. If you encounter connection issues, please ensure:
 - IoTDB is fully started and accepting connections
 - The host and port are correctly configured
 - Network connectivity is available
 - The client library's SessionPool is properly initialized

 ### Testing the Benchmark Infrastructure

 To verify the benchmark tools are correctly installed and configured:

 ```bash
 node benchmark/test-benchmark.js
 ```

 This will test the benchmark infrastructure without requiring IoTDB connection, validating:
 - Configuration management
 - Data generation
 - Metrics collection
 - Performance reporting

 ### API Comparison Benchmarks (NEW)

 #### Tree Model Comparison

 Compare the performance of different insertion methods for tree model:

 ```bash
 # Run with default settings
 node benchmark/benchmark-comparison.js

 # Customize parameters
 TABLET_COUNT=200 CONCURRENCY=20 node benchmark/benchmark-comparison.js
 ```

 This benchmark compares:
 1. **Sequential insertTablet** - Baseline (one tablet at a time)
 2. **insertTablets** - Batch insert (single RPC for multiple tablets)
 3. **insertTabletsParallel** - Concurrent insertion with pool

 #### Table Model Comparison

 Compare the performance of different insertion methods for table model:

 ```bash
 # Run with default settings
 node benchmark/benchmark-table-comparison.js

 # Customize parameters
 TABLET_COUNT=200 CONCURRENCY=20 node benchmark/benchmark-table-comparison.js
 ```

 This benchmark compares:
 1. **Sequential insertTablet** - Baseline (one tablet at a time)
 2. **insertTabletsParallel** - Concurrent insertion with pool
 3. **executeParallel** - Generic parallel execution

 ### Build the Client

 ```bash
 npm install
 npm run build
 ```

 ### Run Tree Model Benchmark

 ```bash
 # Using default settings
 node benchmark/benchmark-tree.js

 # With custom parameters
 DEVICE_NUMBER=50 CLIENT_NUMBER=5 node benchmark/benchmark-tree.js
 ```

 ### Run Table Model Benchmark

 ```bash
 # Using default settings
 node benchmark/benchmark-table.js

 # With custom parameters
 DEVICE_NUMBER=50 CLIENT_NUMBER=5 node benchmark/benchmark-table.js
 ```

 ### Run Multi-Process Cluster Benchmark (Recommended for Maximum Throughput)

 ```bash
 # Best configuration (5.42M pts/s)
 IOTDB_HOST=localhost \
 WORKER_COUNT=8 \
 CLIENT_NUMBER=10 \
 DEVICE_NUMBER=1000 \
 SENSOR_NUMBER=50 \
 LOOP=100 \
 BATCH_SIZE_PER_WRITE=500 \
 POOL_MAX_SIZE=10 \
 node benchmark/benchmark-table-cluster.js
 ```

 **Multi-Process Parameters:**

 | Variable | Default | Description |
 |----------|---------|-------------|
 | `WORKER_COUNT` | CPU cores | Number of worker processes |
 | `CLIENT_NUMBER` | `10` | Concurrent clients per worker |
 | `DEVICE_NUMBER` | `1000` | Total devices (distributed across workers) |
 | `POOL_MAX_SIZE` | `10` | Connection pool size per worker |

 **Performance Tips:**
 - 8 workers is optimal for most servers; more workers may cause saturation
 - Tablet size ~25K points (500 rows × 50 sensors) gives best latency/throughput balance
 - Each worker runs independent SessionPool for true parallel execution

 ## Configuration

 All benchmarks support configuration through environment variables. Default values are used if not specified.

 ### Connection Settings

 | Variable | Default | Description |
 |----------|---------|-------------|
 | `IOTDB_HOST` | `localhost` | IoTDB server host |
 | `IOTDB_PORT` | `6667` | IoTDB server port |
 | `IOTDB_USER` | `root` | Username for authentication |
 | `IOTDB_PASSWORD` | `root` | Password for authentication |
 | `NODE_URLS` | - | Multi-node URLs (e.g., `"host1:6667,host2:6668"`) |

 ### Test Parameters

 | Variable | Default | Description |
 |----------|---------|-------------|
 | `CLIENT_NUMBER` | `10` | Number of concurrent clients |
 | `DEVICE_NUMBER` | `100` | Number of devices to simulate |
 | `SENSOR_NUMBER` | `10` | Number of sensors per device |
 | `BATCH_SIZE_PER_WRITE` | `100` | Data rows per write operation |
 | `LOOP` | - | Total execution loops (alternative to TOTAL_DATA_POINTS) |
 | `TOTAL_DATA_POINTS` | `100000` | Total data points (used when LOOP not set) |

 **Note on LOOP mode**: When `LOOP` is set, total data points = DEVICE_NUMBER × BATCH_SIZE × SENSOR_NUMBER × LOOP. Each loop writes one complete batch for all devices (one tablet per device).

 ### Data Generation Settings

 | Variable | Default | Description |
 |----------|---------|-------------|
 | `POINT_STEP` | `1000` | Time interval between points (ms) |
 | `TIMESTAMP_PRECISION` | `ms` | Timestamp precision (`ms`, `us`, `ns`) |
 | `STRING_LENGTH` | `16` | Length of TEXT/STRING values |
 | `REGENERATE_DATA` | `false` | Force regenerate test data |
 | `DATA_FILE_PATH` | `./benchmark/benchmark_data.json` | Path to pre-generated data |

 ### Data Type Distribution

 Configure the proportion of different sensor types. Must sum to 1.0.

 Default distribution:
 - FLOAT: 30%
 - DOUBLE: 20%
 - INT32: 20%
 - INT64: 10%
 - TEXT: 10%
 - BOOLEAN: 10%

 ### Test Execution Settings

 | Variable | Default | Description |
 |----------|---------|-------------|
 | `WARMUP_ROUNDS` | `0` | Number of warmup iterations |
 | `TEST_ROUNDS` | `1` | Number of test iterations |
 | `REPORT_INTERVAL` | `5000` | Progress report interval (ms) |
 | `ENABLE_DETAILED_METRICS` | `true` | Enable percentile calculations |

 ### Connection Pool Settings

 | Variable | Default | Description |
 |----------|---------|-------------|
 | `POOL_MAX_SIZE` | `20` | Maximum connections in pool |
 | `POOL_MIN_SIZE` | `5` | Minimum connections in pool |
 | `POOL_MAX_IDLE_TIME` | `60000` | Maximum idle time (ms) |
 | `POOL_WAIT_TIMEOUT` | `60000` | Wait timeout for connection (ms) |
 | `ENABLE_DEVICE_SESSION_BINDING` | `false` | Bind devices to sessions (requires DEVICE_NUMBER % POOL_MAX_SIZE == 0) |

 **Device-Session Binding**: When enabled, each session is bound to a specific set of devices, avoiding connection redirects and improving performance. Only enable this when DEVICE_NUMBER is evenly divisible by POOL_MAX_SIZE.

 ## Usage Examples

 ### Loop-Based Execution (Recommended)

 ```bash
 # Execute 1000 loops, each writing one batch for all devices
 LOOP=1000 \
 DEVICE_NUMBER=100 \
 SENSOR_NUMBER=10 \
 BATCH_SIZE_PER_WRITE=100 \
 CLIENT_NUMBER=10 \
 node benchmark/benchmark-tree.js

 # Total data points = 100 devices × 100 rows × 10 sensors × 1000 loops = 100,000,000
 ```

 ### Device-Session Binding for Optimal Performance

 ```bash
 # Bind 100 devices to 10 sessions (10 devices per session)
 LOOP=1000 \
 DEVICE_NUMBER=100 \
 SENSOR_NUMBER=10 \
 BATCH_SIZE_PER_WRITE=100 \
 POOL_MAX_SIZE=10 \
 ENABLE_DEVICE_SESSION_BINDING=true \
 node benchmark/benchmark-tree.js
 ```

 ### Basic Test with Moderate Load (Legacy Mode)

 ```bash
 CLIENT_NUMBER=5 \
 DEVICE_NUMBER=50 \
 SENSOR_NUMBER=5 \
 BATCH_SIZE_PER_WRITE=100 \
 TOTAL_DATA_POINTS=50000 \
 node benchmark/benchmark-tree.js
 ```

 ### High Concurrency Test

 ```bash
 CLIENT_NUMBER=50 \
 DEVICE_NUMBER=1000 \
 SENSOR_NUMBER=10 \
 BATCH_SIZE_PER_WRITE=1000 \
 TOTAL_DATA_POINTS=1000000 \
 node benchmark/benchmark-tree.js
 ```

 ### Multi-Node Cluster Test

 ```bash
 NODE_URLS="node1:6667,node2:6667,node3:6667" \
 CLIENT_NUMBER=20 \
 DEVICE_NUMBER=500 \
 node benchmark/benchmark-tree.js
 ```

 ### Test with Custom Data Types

 Create a custom configuration file:

 ```javascript
 // custom-config.js
 const { createConfig } = require('./benchmark/config');

 const config = createConfig({
   CLIENT_NUMBER: 10,
   DEVICE_NUMBER: 100,
   SENSOR_NUMBER: 8,
   INSERT_DATATYPE_PROPORTION: {
     3: 0.5,  // 50% FLOAT
     4: 0.3,  // 30% DOUBLE
     1: 0.2,  // 20% INT32
   },
 });

 module.exports = config;
 ```

 ### Test with Warmup

 ```bash
 WARMUP_ROUNDS=3 \
 TEST_ROUNDS=1 \
 CLIENT_NUMBER=10 \
 node benchmark/benchmark-tree.js
 ```

 ### Test with Progress Monitoring

 ```bash
 REPORT_INTERVAL=2000 \
 ENABLE_DETAILED_METRICS=true \
 CLIENT_NUMBER=10 \
 TOTAL_DATA_POINTS=500000 \
 node benchmark/benchmark-tree.js
 ```

 ## Understanding Results

 ### Performance Metrics

 The benchmark reports the following metrics:

 #### Execution Time
 - **Duration** - Total test duration in seconds/milliseconds

 #### Operations
 - **Total Operations** - Number of write operations executed
 - **Successful** - Number of successful operations
 - **Failed** - Number of failed operations
 - **Success Rate** - Percentage of successful operations

 #### Data Points
 - **Total Points Written** - Total number of data points inserted

 #### Throughput
 - **Operations/sec** - Write operations per second
 - **Points/sec** - Data points inserted per second

 #### Latency (milliseconds)
 - **Min** - Minimum operation latency
 - **Max** - Maximum operation latency
 - **Average** - Mean operation latency
 - **P50 (Median)** - 50th percentile latency
 - **P90** - 90th percentile latency
 - **P95** - 95th percentile latency
 - **P99** - 99th percentile latency

 ### Sample Output

 ```
 ================================================================================
 BENCHMARK RESULTS
 ================================================================================

 [Execution Time]
   Duration:              45.23s (45234ms)

 [Operations]
   Total Operations:      1000
   Successful:            998
   Failed:                2
   Success Rate:          99.80%

 [Data Points]
   Total Points Written:  100,000

 [Throughput]
   Operations/sec:        22.11
   Points/sec:            2,210

 [Latency (ms)]
   Min:                   15.23ms
   Max:                   1250.45ms
   Average:               45.23ms
   P50 (Median):          42.15ms
   P90:                   78.45ms
   P95:                   95.23ms
   P99:                   125.67ms

 ================================================================================
 ```

 ## Benchmark Workflow

 ### 1. Data Preparation Phase
 - Checks if pre-generated data exists
 - Generates new data if needed or if `REGENERATE_DATA=true`
 - Saves generated data to file for reuse
 - **Memory Optimization**: Uses shared batch templates across all devices
   - ONE set of timestamps and values generated
   - ALL devices reference the same template data
   - Only device metadata (ID, measurements, types) stored per device
   - Supports 100K+ devices without OOM (~1.5 KB per device)

 **Memory Comparison (100K devices, 10 sensors, 100 rows):**
 - Old approach: ~15+ GB (each device has own data copy)
 - New approach: ~150 MB (shared batch templates)
 - **Reduction**: 99% memory savings

 ### 2. Schema Registration Phase
 - Creates storage groups/databases
 - Creates timeseries/tables with appropriate data types
 - Pre-registers all metadata to avoid creation overhead during testing

 ### 3. Warmup Phase (Optional)
 - Runs limited operations to warm up connections
 - Not included in final metrics
 - Helps stabilize performance before actual testing

 ### 4. Main Test Phase
 - Spawns concurrent worker clients
 - Each worker reads from shared batch templates
 - Updates timestamps to current time
 - Executes write operations
 - Records latency and success/failure
 - Reports progress at configured intervals

 ### 5. Results Analysis Phase
 - Calculates throughput metrics
 - Computes latency statistics
 - Generates percentile distributions
 - Prints comprehensive report

 ## Architecture

 ### Core Components

 ```
 benchmark/
 ├── config.js              # Configuration management
 ├── data-generator.js      # Pre-generate test data
 ├── schema-manager.js      # Metadata registration
 ├── benchmark-core.js      # Core benchmark engine
 ├── benchmark-tree.js      # Tree model entry point
 ├── benchmark-table.js     # Table model entry point
 └── README.md             # This file
 ```

 ### Component Responsibilities

 - **config.js** - Centralized configuration with validation
 - **data-generator.js** - Generates and caches test data
 - **schema-manager.js** - Creates and manages database schema
 - **benchmark-core.js** - Metrics collection, concurrency control, statistics
 - **benchmark-tree.js** - Tree model specific implementation
 - **benchmark-table.js** - Table model specific implementation

 ## Performance Tuning Tips

 ### 1. Optimize Batch Size
 - Larger batches = fewer operations but more data per operation
 - Sweet spot typically between 100-1000 rows per batch
 - Test different values for your workload

 ### 2. Adjust Concurrency
 - More clients = higher throughput (up to a point)
 - Too many clients may saturate server or network
 - Start with 10-20 clients and adjust based on results

 ### 3. Use Connection Pooling
 - Set `POOL_MIN_SIZE` to warm up connections
 - Set `POOL_MAX_SIZE` based on expected peak concurrency
 - Larger pools help with bursty workloads

 ### 4. Pre-generate Data
 - Always use pre-generated data for accurate results
 - Set `REGENERATE_DATA=false` after first run
 - Cached data eliminates generation overhead

 ### 5. Monitor System Resources
 - Use system monitoring tools (htop, iostat, vmstat)
 - Watch CPU, memory, disk I/O, and network
 - Identify bottlenecks in client or server

 ### 6. Network Considerations
 - Test with server on same network
 - Consider network latency in results
 - Use multi-node testing for cluster performance

 ## Troubleshooting

 ### Common Issues

 #### "Configuration validation failed"
 - Check that all proportions sum to 1.0
 - Verify numeric parameters are positive
 - Ensure required connection parameters are provided

 #### "Connection refused" or "Cannot connect"
 - Verify IoTDB is running
 - Check host and port configuration
 - Ensure firewall allows connections

 #### "Out of memory" errors
 **Note**: The benchmark now uses memory-optimized shared batch templates and can handle 100K+ devices with minimal memory.

 If you still encounter OOM:
 - Reduce `BATCH_SIZE_PER_WRITE`
 - Reduce `CLIENT_NUMBER`
 - For data generation: Delete old data file and regenerate (uses new optimized format)
 - Increase Node.js heap size: `NODE_OPTIONS=--max-old-space-size=4096`

 **Memory usage reference** (with optimization):
 - 10K devices: ~16 MB
 - 100K devices: ~150 MB
 - 1M devices: ~1.5 GB

 #### "Schema already exists" warnings
 - Normal if rerunning tests
 - Clean up with: `DELETE DATABASE root.benchmark.*` (tree model)
 - Or: `DROP DATABASE benchmark_db` (table model)

 #### Poor performance
 - Check server load and resources
 - Verify network connectivity
 - Try fewer concurrent clients
 - Increase batch size
 - Enable warmup rounds

 ## Best Practices

 1. **Run Multiple Tests** - Execute several runs and average results
 2. **Use Warmup** - Set `WARMUP_ROUNDS=3` for stable results
 3. **Monitor Server** - Watch server metrics during tests
 4. **Clean Between Tests** - Drop and recreate schema between major tests
 5. **Document Configuration** - Save test configurations for reproducibility
 6. **Baseline Tests** - Establish baseline before making changes
 7. **Isolate Variables** - Change one parameter at a time
 8. **Test Realistic Scenarios** - Match production data patterns

 ## Contributing

 Contributions are welcome! Please:

 1. Follow existing code style
 2. Add comments for complex logic
 3. Test thoroughly before submitting
 4. Update documentation as needed

 ## License

 Apache License 2.0

 ## References

 - [Apache IoTDB Documentation](https://iotdb.apache.org/)
 - [thulab/iot-benchmark](https://github.com/thulab/iot-benchmark)
 - [IoTDB Client Documentation](../README.md)
	# IoTDB Client Benchmark Tools

	Comprehensive performance testing tools for Apache IoTDB Node.js client, inspired by the [thulab/iot-benchmark](https://github.com/thulab/iot-benchmark) project.

	## Overview

	This benchmark suite provides specialized tools for testing IoTDB write performance:

	- Tree Model Benchmark (`benchmark-tree.js`) - Tests timeseries data model
	- Table Model Benchmark (`benchmark-table.js`) - Tests relational data model
	- Multi-Process Cluster (`benchmark-table-cluster.js`) - NEW Multi-process mode for maximum throughput
	- API Comparison (`benchmark-comparison.js`, `benchmark-table-comparison.js`) - Compare insertion methods

	### Performance Summary

	\| Mode \| Configuration \| Throughput \| Notes \|
	\|------\|---------------\|------------\|-------\|
	\| Single Process \| 20 clients, 200 devices \| 4.28M pts/s \| Best single-process \|
	\| Multi-Process \| 8 workers × 10 clients \| 5.42M pts/s \| Recommended \|
	\| Java iot-benchmark \| Similar config \| ~60M pts/s \| Reference \|

	### Key Features

	✅ Multi-Process Cluster Mode - Overcomes Node.js single-thread limitation
	✅ Pre-generated Test Data - Eliminates data generation overhead
	✅ Memory-Optimized - Streaming batch processing for large-scale tests
	✅ Flexible Configuration - Extensive parameters for customizing scenarios
	✅ Detailed Metrics - Comprehensive statistics including latency percentiles

	## Quick Start

	### Prerequisites

	1. Node.js >= 14.0.0
	2. Running IoTDB instance (v1.0+)
	3. Built IoTDB client library

	Note: The benchmark tools require a working IoTDB instance. If you encounter connection issues, please ensure:
	- IoTDB is fully started and accepting connections
	- The host and port are correctly configured
	- Network connectivity is available
	- The client library's SessionPool is properly initialized

	### Testing the Benchmark Infrastructure

	To verify the benchmark tools are correctly installed and configured:

	```bash
	node benchmark/test-benchmark.js
	```

	This will test the benchmark infrastructure without requiring IoTDB connection, validating:
	- Configuration management
	- Data generation
	- Metrics collection
	- Performance reporting

	### API Comparison Benchmarks (NEW)

	#### Tree Model Comparison

	Compare the performance of different insertion methods for tree model:

	```bash
	# Run with default settings
	node benchmark/benchmark-comparison.js

	# Customize parameters
	TABLET_COUNT=200 CONCURRENCY=20 node benchmark/benchmark-comparison.js
	```

	This benchmark compares:
	1. Sequential insertTablet - Baseline (one tablet at a time)
	2. insertTablets - Batch insert (single RPC for multiple tablets)
	3. insertTabletsParallel - Concurrent insertion with pool

	#### Table Model Comparison

	Compare the performance of different insertion methods for table model:

	```bash
	# Run with default settings
	node benchmark/benchmark-table-comparison.js

	# Customize parameters
	TABLET_COUNT=200 CONCURRENCY=20 node benchmark/benchmark-table-comparison.js
	```

	This benchmark compares:
	1. Sequential insertTablet - Baseline (one tablet at a time)
	2. insertTabletsParallel - Concurrent insertion with pool
	3. executeParallel - Generic parallel execution

	### Build the Client

	```bash
	npm install
	npm run build
	```

	### Run Tree Model Benchmark

	```bash
	# Using default settings
	node benchmark/benchmark-tree.js

	# With custom parameters
	DEVICE_NUMBER=50 CLIENT_NUMBER=5 node benchmark/benchmark-tree.js
	```

	### Run Table Model Benchmark

	```bash
	# Using default settings
	node benchmark/benchmark-table.js

	# With custom parameters
	DEVICE_NUMBER=50 CLIENT_NUMBER=5 node benchmark/benchmark-table.js
	```

	### Run Multi-Process Cluster Benchmark (Recommended for Maximum Throughput)

	```bash
	# Best configuration (5.42M pts/s)
	IOTDB_HOST=localhost \
	WORKER_COUNT=8 \
	CLIENT_NUMBER=10 \
	DEVICE_NUMBER=1000 \
	SENSOR_NUMBER=50 \
	LOOP=100 \
	BATCH_SIZE_PER_WRITE=500 \
	POOL_MAX_SIZE=10 \
	node benchmark/benchmark-table-cluster.js
	```

	Multi-Process Parameters:

	\| Variable \| Default \| Description \|
	\|----------\|---------\|-------------\|
	\| `WORKER_COUNT` \| CPU cores \| Number of worker processes \|
	\| `CLIENT_NUMBER` \| `10` \| Concurrent clients per worker \|
	\| `DEVICE_NUMBER` \| `1000` \| Total devices (distributed across workers) \|
	\| `POOL_MAX_SIZE` \| `10` \| Connection pool size per worker \|

	Performance Tips:
	- 8 workers is optimal for most servers; more workers may cause saturation
	- Tablet size ~25K points (500 rows × 50 sensors) gives best latency/throughput balance
	- Each worker runs independent SessionPool for true parallel execution

	## Configuration

	All benchmarks support configuration through environment variables. Default values are used if not specified.

	### Connection Settings

	\| Variable \| Default \| Description \|
	\|----------\|---------\|-------------\|
	\| `IOTDB_HOST` \| `localhost` \| IoTDB server host \|
	\| `IOTDB_PORT` \| `6667` \| IoTDB server port \|
	\| `IOTDB_USER` \| `root` \| Username for authentication \|
	\| `IOTDB_PASSWORD` \| `root` \| Password for authentication \|
	\| `NODE_URLS` \| - \| Multi-node URLs (e.g., `"host1:6667,host2:6668"`) \|

	### Test Parameters

	\| Variable \| Default \| Description \|
	\|----------\|---------\|-------------\|
	\| `CLIENT_NUMBER` \| `10` \| Number of concurrent clients \|
	\| `DEVICE_NUMBER` \| `100` \| Number of devices to simulate \|
	\| `SENSOR_NUMBER` \| `10` \| Number of sensors per device \|
	\| `BATCH_SIZE_PER_WRITE` \| `100` \| Data rows per write operation \|
	\| `LOOP` \| - \| Total execution loops (alternative to TOTAL_DATA_POINTS) \|
	\| `TOTAL_DATA_POINTS` \| `100000` \| Total data points (used when LOOP not set) \|

	Note on LOOP mode: When `LOOP` is set, total data points = DEVICE_NUMBER × BATCH_SIZE × SENSOR_NUMBER × LOOP. Each loop writes one complete batch for all devices (one tablet per device).

	### Data Generation Settings

	\| Variable \| Default \| Description \|
	\|----------\|---------\|-------------\|
	\| `POINT_STEP` \| `1000` \| Time interval between points (ms) \|
	\| `TIMESTAMP_PRECISION` \| `ms` \| Timestamp precision (`ms`, `us`, `ns`) \|
	\| `STRING_LENGTH` \| `16` \| Length of TEXT/STRING values \|
	\| `REGENERATE_DATA` \| `false` \| Force regenerate test data \|
	\| `DATA_FILE_PATH` \| `./benchmark/benchmark_data.json` \| Path to pre-generated data \|

	### Data Type Distribution

	Configure the proportion of different sensor types. Must sum to 1.0.

	Default distribution:
	- FLOAT: 30%
	- DOUBLE: 20%
	- INT32: 20%
	- INT64: 10%
	- TEXT: 10%
	- BOOLEAN: 10%

	### Test Execution Settings

	\| Variable \| Default \| Description \|
	\|----------\|---------\|-------------\|
	\| `WARMUP_ROUNDS` \| `0` \| Number of warmup iterations \|
	\| `TEST_ROUNDS` \| `1` \| Number of test iterations \|
	\| `REPORT_INTERVAL` \| `5000` \| Progress report interval (ms) \|
	\| `ENABLE_DETAILED_METRICS` \| `true` \| Enable percentile calculations \|

	### Connection Pool Settings

	\| Variable \| Default \| Description \|
	\|----------\|---------\|-------------\|
	\| `POOL_MAX_SIZE` \| `20` \| Maximum connections in pool \|
	\| `POOL_MIN_SIZE` \| `5` \| Minimum connections in pool \|
	\| `POOL_MAX_IDLE_TIME` \| `60000` \| Maximum idle time (ms) \|
	\| `POOL_WAIT_TIMEOUT` \| `60000` \| Wait timeout for connection (ms) \|
	\| `ENABLE_DEVICE_SESSION_BINDING` \| `false` \| Bind devices to sessions (requires DEVICE_NUMBER % POOL_MAX_SIZE == 0) \|

	Device-Session Binding: When enabled, each session is bound to a specific set of devices, avoiding connection redirects and improving performance. Only enable this when DEVICE_NUMBER is evenly divisible by POOL_MAX_SIZE.

	## Usage Examples

	### Loop-Based Execution (Recommended)

	```bash
	# Execute 1000 loops, each writing one batch for all devices
	LOOP=1000 \
	DEVICE_NUMBER=100 \
	SENSOR_NUMBER=10 \
	BATCH_SIZE_PER_WRITE=100 \
	CLIENT_NUMBER=10 \
	node benchmark/benchmark-tree.js

	# Total data points = 100 devices × 100 rows × 10 sensors × 1000 loops = 100,000,000
	```

	### Device-Session Binding for Optimal Performance

	```bash
	# Bind 100 devices to 10 sessions (10 devices per session)
	LOOP=1000 \
	DEVICE_NUMBER=100 \
	SENSOR_NUMBER=10 \
	BATCH_SIZE_PER_WRITE=100 \
	POOL_MAX_SIZE=10 \
	ENABLE_DEVICE_SESSION_BINDING=true \
	node benchmark/benchmark-tree.js
	```

	### Basic Test with Moderate Load (Legacy Mode)

	```bash
	CLIENT_NUMBER=5 \
	DEVICE_NUMBER=50 \
	SENSOR_NUMBER=5 \
	BATCH_SIZE_PER_WRITE=100 \
	TOTAL_DATA_POINTS=50000 \
	node benchmark/benchmark-tree.js
	```

	### High Concurrency Test

	```bash
	CLIENT_NUMBER=50 \
	DEVICE_NUMBER=1000 \
	SENSOR_NUMBER=10 \
	BATCH_SIZE_PER_WRITE=1000 \
	TOTAL_DATA_POINTS=1000000 \
	node benchmark/benchmark-tree.js
	```

	### Multi-Node Cluster Test

	```bash
	NODE_URLS="node1:6667,node2:6667,node3:6667" \
	CLIENT_NUMBER=20 \
	DEVICE_NUMBER=500 \
	node benchmark/benchmark-tree.js
	```

	### Test with Custom Data Types

	Create a custom configuration file:

	```javascript
	// custom-config.js
	const { createConfig } = require('./benchmark/config');

	const config = createConfig({
	CLIENT_NUMBER: 10,
	DEVICE_NUMBER: 100,
	SENSOR_NUMBER: 8,
	INSERT_DATATYPE_PROPORTION: {
	3: 0.5, // 50% FLOAT
	4: 0.3, // 30% DOUBLE
	1: 0.2, // 20% INT32
	},
	});

	module.exports = config;
	```

	### Test with Warmup

	```bash
	WARMUP_ROUNDS=3 \
	TEST_ROUNDS=1 \
	CLIENT_NUMBER=10 \
	node benchmark/benchmark-tree.js
	```

	### Test with Progress Monitoring

	```bash
	REPORT_INTERVAL=2000 \
	ENABLE_DETAILED_METRICS=true \
	CLIENT_NUMBER=10 \
	TOTAL_DATA_POINTS=500000 \
	node benchmark/benchmark-tree.js
	```

	## Understanding Results

	### Performance Metrics

	The benchmark reports the following metrics:

	#### Execution Time
	- Duration - Total test duration in seconds/milliseconds

	#### Operations
	- Total Operations - Number of write operations executed
	- Successful - Number of successful operations
	- Failed - Number of failed operations
	- Success Rate - Percentage of successful operations

	#### Data Points
	- Total Points Written - Total number of data points inserted

	#### Throughput
	- Operations/sec - Write operations per second
	- Points/sec - Data points inserted per second

	#### Latency (milliseconds)
	- Min - Minimum operation latency
	- Max - Maximum operation latency
	- Average - Mean operation latency
	- P50 (Median) - 50th percentile latency
	- P90 - 90th percentile latency
	- P95 - 95th percentile latency
	- P99 - 99th percentile latency

	### Sample Output

	```
	================================================================================
	BENCHMARK RESULTS
	================================================================================

	[Execution Time]
	Duration: 45.23s (45234ms)

	[Operations]
	Total Operations: 1000
	Successful: 998
	Failed: 2
	Success Rate: 99.80%

	[Data Points]
	Total Points Written: 100,000

	[Throughput]
	Operations/sec: 22.11
	Points/sec: 2,210

	[Latency (ms)]
	Min: 15.23ms
	Max: 1250.45ms
	Average: 45.23ms
	P50 (Median): 42.15ms
	P90: 78.45ms
	P95: 95.23ms
	P99: 125.67ms

	================================================================================
	```

	## Benchmark Workflow

	### 1. Data Preparation Phase
	- Checks if pre-generated data exists
	- Generates new data if needed or if `REGENERATE_DATA=true`
	- Saves generated data to file for reuse
	- Memory Optimization: Uses shared batch templates across all devices
	- ONE set of timestamps and values generated
	- ALL devices reference the same template data
	- Only device metadata (ID, measurements, types) stored per device
	- Supports 100K+ devices without OOM (~1.5 KB per device)

	Memory Comparison (100K devices, 10 sensors, 100 rows):
	- Old approach: ~15+ GB (each device has own data copy)
	- New approach: ~150 MB (shared batch templates)
	- Reduction: 99% memory savings

	### 2. Schema Registration Phase
	- Creates storage groups/databases
	- Creates timeseries/tables with appropriate data types
	- Pre-registers all metadata to avoid creation overhead during testing

	### 3. Warmup Phase (Optional)
	- Runs limited operations to warm up connections
	- Not included in final metrics
	- Helps stabilize performance before actual testing

	### 4. Main Test Phase
	- Spawns concurrent worker clients
	- Each worker reads from shared batch templates
	- Updates timestamps to current time
	- Executes write operations
	- Records latency and success/failure
	- Reports progress at configured intervals

	### 5. Results Analysis Phase
	- Calculates throughput metrics
	- Computes latency statistics
	- Generates percentile distributions
	- Prints comprehensive report

	## Architecture

	### Core Components

	```
	benchmark/
	├── config.js # Configuration management
	├── data-generator.js # Pre-generate test data
	├── schema-manager.js # Metadata registration
	├── benchmark-core.js # Core benchmark engine
	├── benchmark-tree.js # Tree model entry point
	├── benchmark-table.js # Table model entry point
	└── README.md # This file
	```

	### Component Responsibilities

	- config.js - Centralized configuration with validation
	- data-generator.js - Generates and caches test data
	- schema-manager.js - Creates and manages database schema
	- benchmark-core.js - Metrics collection, concurrency control, statistics
	- benchmark-tree.js - Tree model specific implementation
	- benchmark-table.js - Table model specific implementation

	## Performance Tuning Tips

	### 1. Optimize Batch Size
	- Larger batches = fewer operations but more data per operation
	- Sweet spot typically between 100-1000 rows per batch
	- Test different values for your workload

	### 2. Adjust Concurrency
	- More clients = higher throughput (up to a point)
	- Too many clients may saturate server or network
	- Start with 10-20 clients and adjust based on results

	### 3. Use Connection Pooling
	- Set `POOL_MIN_SIZE` to warm up connections
	- Set `POOL_MAX_SIZE` based on expected peak concurrency
	- Larger pools help with bursty workloads

	### 4. Pre-generate Data
	- Always use pre-generated data for accurate results
	- Set `REGENERATE_DATA=false` after first run
	- Cached data eliminates generation overhead

	### 5. Monitor System Resources
	- Use system monitoring tools (htop, iostat, vmstat)
	- Watch CPU, memory, disk I/O, and network
	- Identify bottlenecks in client or server

	### 6. Network Considerations
	- Test with server on same network
	- Consider network latency in results
	- Use multi-node testing for cluster performance

	## Troubleshooting

	### Common Issues

	#### "Configuration validation failed"
	- Check that all proportions sum to 1.0
	- Verify numeric parameters are positive
	- Ensure required connection parameters are provided

	#### "Connection refused" or "Cannot connect"
	- Verify IoTDB is running
	- Check host and port configuration
	- Ensure firewall allows connections

	#### "Out of memory" errors
	Note: The benchmark now uses memory-optimized shared batch templates and can handle 100K+ devices with minimal memory.

	If you still encounter OOM:
	- Reduce `BATCH_SIZE_PER_WRITE`
	- Reduce `CLIENT_NUMBER`
	- For data generation: Delete old data file and regenerate (uses new optimized format)
	- Increase Node.js heap size: `NODE_OPTIONS=--max-old-space-size=4096`

	Memory usage reference (with optimization):
	- 10K devices: ~16 MB
	- 100K devices: ~150 MB
	- 1M devices: ~1.5 GB

	#### "Schema already exists" warnings
	- Normal if rerunning tests
	- Clean up with: `DELETE DATABASE root.benchmark.*` (tree model)
	- Or: `DROP DATABASE benchmark_db` (table model)

	#### Poor performance
	- Check server load and resources
	- Verify network connectivity
	- Try fewer concurrent clients
	- Increase batch size
	- Enable warmup rounds

	## Best Practices

	1. Run Multiple Tests - Execute several runs and average results
	2. Use Warmup - Set `WARMUP_ROUNDS=3` for stable results
	3. Monitor Server - Watch server metrics during tests
	4. Clean Between Tests - Drop and recreate schema between major tests
	5. Document Configuration - Save test configurations for reproducibility
	6. Baseline Tests - Establish baseline before making changes
	7. Isolate Variables - Change one parameter at a time
	8. Test Realistic Scenarios - Match production data patterns

	## Contributing

	Contributions are welcome! Please:

	1. Follow existing code style
	2. Add comments for complex logic
	3. Test thoroughly before submitting
	4. Update documentation as needed

	## License

	Apache License 2.0

	## References

	- [Apache IoTDB Documentation](https://iotdb.apache.org/)
	- [thulab/iot-benchmark](https://github.com/thulab/iot-benchmark)
	- [IoTDB Client Documentation](../README.md)