This short guide will walk you through the basic process of using IoTDB. For a more-complete guide, please visit our website's User Guide.
To use IoTDB, you need to have:
IoTDB provides you three installation methods, you can refer to the following suggestions, choose one of them:
You can download the binary file from: Download Page
Configuration files are under “conf” folder
datanode-env.bat
, datanode-env.sh
),iotdb-datanode.properties
)logback.xml
).For more, see Config in detail.
You can go through the following step to test the installation, if there is no error after execution, the installation is completed.
IoTDB is a database based on distributed system. To launch IoTDB, you can first start standalone mode (i.e. 1 ConfigNode and 1 DataNode) to check.
Users can start IoTDB standalone mode by the start-standalone script under the sbin folder.
# Unix/OS X > bash sbin/start-standalone.sh
# Windows > sbin\start-standalone.bat
This article uses a local environment as an example to illustrate how to start, expand, and shrink an IoTDB Cluster.
Notice: This document is a tutorial for deploying in a pseudo-cluster environment using different local ports, and is for exercise only. In real deployment scenarios, you only need to configure the IPV4 address or domain name of the server, and do not need to change the Node ports.
Unzip the apache-iotdb-1.0.0-all-bin.zip file to cluster0 folder.
Start the Cluster version with one ConfigNode and one DataNode(1C1D), and the default number of replicas is one.
./cluster0/sbin/start-confignode.sh ./cluster0/sbin/start-datanode.sh
./cluster0/sbin/start-cli.sh
show cluster details
command on the Cli. The result is shown below:IoTDB> show cluster details +------+----------+-------+---------------+------------+-------------------+----------+-------+--------+-------------------+-----------------+ |NodeID| NodeType| Status|InternalAddress|InternalPort|ConfigConsensusPort|RpcAddress|RpcPort|MppPort |SchemaConsensusPort|DataConsensusPort| +------+----------+-------+---------------+------------+-------------------+----------+-------+--------+-------------------+-----------------+ | 0|ConfigNode|Running| 127.0.0.1| 10710| 10720| | | | | | | 1| DataNode|Running| 127.0.0.1| 10730| | 127.0.0.1| 6667| 10740| 10750| 10760| +------+----------+-------+---------------+------------+-------------------+----------+-------+--------+-------------------+-----------------+ Total line number = 2 It costs 0.242s
Unzip the apache-iotdb-1.0.0-all-bin.zip file to cluster1 and cluster2 folder.
For folder cluster1:
configuration item | value |
---|---|
cn_internal_address | 127.0.0.1 |
cn_internal_port | 10711 |
cn_consensus_port | 10721 |
cn_target_config_node_list | 127.0.0.1:10710 |
configuration item | value |
---|---|
dn_rpc_address | 127.0.0.1 |
dn_rpc_port | 6668 |
dn_internal_address | 127.0.0.1 |
dn_internal_port | 10731 |
dn_mpp_data_exchange_port | 10741 |
dn_schema_region_consensus_port | 10751 |
dn_data_region_consensus_port | 10761 |
dn_target_config_node_list | 127.0.0.1:10710 |
For folder cluster2:
configuration item | value |
---|---|
cn_internal_address | 127.0.0.1 |
cn_internal_port | 10712 |
cn_consensus_port | 10722 |
cn_target_config_node_list | 127.0.0.1:10710 |
configuration item | value |
---|---|
dn_rpc_address | 127.0.0.1 |
dn_rpc_port | 6669 |
dn_internal_address | 127.0.0.1 |
dn_internal_port | 10732 |
dn_mpp_data_exchange_port | 10742 |
dn_schema_region_consensus_port | 10752 |
dn_data_region_consensus_port | 10762 |
dn_target_config_node_list | 127.0.0.1:10710 |
Expanding the Cluster to three ConfigNode and three DataNode(3C3D). The following commands can be executed in arbitrary order.
./cluster1/sbin/start-confignode.sh ./cluster1/sbin/start-datanode.sh ./cluster2/sbin/start-confignode.sh ./cluster2/sbin/start-datanode.sh
Execute the show cluster details
command, then the result is shown below:
IoTDB> show cluster details +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ |NodeID| NodeType| Status|InternalAddress|InternalPort|ConfigConsensusPort|RpcAddress|RpcPort|MppPort|SchemaConsensusPort|DataConsensusPort| +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ | 0|ConfigNode|Running| 127.0.0.1| 10710| 10720| | | | | | | 2|ConfigNode|Running| 127.0.0.1| 10711| 10721| | | | | | | 3|ConfigNode|Running| 127.0.0.1| 10712| 10722| | | | | | | 1| DataNode|Running| 127.0.0.1| 10730| | 127.0.0.1| 6667| 10740| 10750| 10760| | 4| DataNode|Running| 127.0.0.1| 10731| | 127.0.0.1| 6668| 10741| 10751| 10761| | 5| DataNode|Running| 127.0.0.1| 10732| | 127.0.0.1| 6669| 10742| 10752| 10762| +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ Total line number = 6 It costs 0.012s
# Removing by ip:port ./cluster0/sbin/remove-confignode.sh 127.0.0.1:10711 # Removing by Node index ./cluster0/sbin/remove-confignode.sh 2
# Removing by ip:port ./cluster0/sbin/remove-datanode.sh 127.0.0.1:6668 # Removing by Node index ./cluster0/sbin/remove-confignode.sh 4
Execute the show cluster details
command, then the result is shown below:
IoTDB> show cluster details +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ |NodeID| NodeType| Status|InternalAddress|InternalPort|ConfigConsensusPort|RpcAddress|RpcPort|MppPort|SchemaConsensusPort|DataConsensusPort| +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ | 0|ConfigNode|Running| 127.0.0.1| 10710| 10720| | | | | | | 3|ConfigNode|Running| 127.0.0.1| 10712| 10722| | | | | | | 1| DataNode|Running| 127.0.0.1| 10730| | 127.0.0.1| 6667| 10740| 10750| 10760| | 5| DataNode|Running| 127.0.0.1| 10732| | 127.0.0.1| 6669| 10742| 10752| 10762| +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ Total line number = 4 It costs 0.005s
You can either download the binary release files (see Chap 3.1) or compile with source code (see Chap 3.2).
Git
git clone https://github.com/apache/iotdb.git git checkout v1.3.x
Website
Under the source root folder:
mvn clean package -pl distribution -am -DskipTests
Then you will get the binary distribution under distribution/target/apache-iotdb-1.3.x-SNAPSHOT-all-bin/apache-iotdb-1.3.x-SNAPSHOT-all-bin.
Folder | Description |
---|---|
conf | Configuration files folder, contains configuration files of ConfigNode, DataNode, JMX and logback |
data | Data files folder, contains data files of ConfigNode and DataNode |
lib | Jar files folder |
licenses | Licenses files folder |
logs | Logs files folder, contains logs files of ConfigNode and DataNode |
sbin | Shell files folder, contains start/stop/remove shell of ConfigNode and DataNode, cli shell |
tools | System tools |
apache-iotdb-1.0.0-SNAPSHOT-all-bin
contains both the ConfigNode and the DataNode. Please deploy the files to all servers of your target cluster. A best practice is deploying the files into the same directory in all servers.
If you want to try the cluster mode on one server, please read Cluster Quick Start.
We need to modify the configurations on each server. Therefore, login each server and switch the working directory to apache-iotdb-1.0.0-SNAPSHOT-all-bin
. The configuration files are stored in the ./conf
directory.
For all ConfigNode servers, we need to modify the common configuration (see Chap 5.2.1) and ConfigNode configuration (see Chap 5.2.2).
For all DataNode servers, we need to modify the common configuration (see Chap 5.2.1) and DataNode configuration (see Chap 5.2.3).
Open the common configuration file ./conf/iotdb-common.properties, and set the following parameters base on the Deployment Recommendation:
Configuration | Description | Default |
---|---|---|
cluster_name | Cluster name for which the Node to join in | defaultCluster |
config_node_consensus_protocol_class | Consensus protocol of ConfigNode | org.apache.iotdb.consensus.ratis.RatisConsensus |
schema_replication_factor | Schema replication factor, no more than DataNode number | 1 |
schema_region_consensus_protocol_class | Consensus protocol of schema replicas | org.apache.iotdb.consensus.ratis.RatisConsensus |
data_replication_factor | Data replication factor, no more than DataNode number | 1 |
data_region_consensus_protocol_class | Consensus protocol of data replicas. Note that RatisConsensus currently does not support multiple data directories | org.apache.iotdb.consensus.iot.IoTConsensus |
Notice: The preceding configuration parameters cannot be changed after the cluster is started. Ensure that the common configurations of all Nodes are the same. Otherwise, the Nodes cannot be started.
Open the ConfigNode configuration file ./conf/iotdb-confignode.properties, and set the following parameters based on the IP address and available port of the server or VM:
Configuration | Description | Default | Usage |
---|---|---|---|
cn_internal_address | Internal rpc service address of ConfigNode | 127.0.0.1 | Set to the IPV4 address or domain name of the server |
cn_internal_port | Internal rpc service port of ConfigNode | 10710 | Set to any unoccupied port |
cn_consensus_port | ConfigNode replication consensus protocol communication port | 10720 | Set to any unoccupied port |
cn_target_config_node_list | ConfigNode address to which the node is connected when it is registered to the cluster. Note that Only one ConfigNode can be configured. | 127.0.0.1:10710 | For Seed-ConfigNode, set to its own cn_internal_address:cn_internal_port; For other ConfigNodes, set to other one running ConfigNode's cn_internal_address:cn_internal_port |
Notice: The preceding configuration parameters cannot be changed after the node is started. Ensure that all ports are not occupied. Otherwise, the Node cannot be started.
Open the DataNode configuration file ./conf/iotdb-datanode.properties, and set the following parameters based on the IP address and available port of the server or VM:
Configuration | Description | Default | Usage |
---|---|---|---|
dn_rpc_address | Client RPC Service address | 127.0.0.1 | Set to the IPV4 address or domain name of the server |
dn_rpc_port | Client RPC Service port | 6667 | Set to any unoccupied port |
dn_internal_address | Control flow address of DataNode inside cluster | 127.0.0.1 | Set to the IPV4 address or domain name of the server |
dn_internal_port | Control flow port of DataNode inside cluster | 10730 | Set to any unoccupied port |
dn_mpp_data_exchange_port | Data flow port of DataNode inside cluster | 10740 | Set to any unoccupied port |
dn_data_region_consensus_port | Data replicas communication port for consensus | 10750 | Set to any unoccupied port |
dn_schema_region_consensus_port | Schema replicas communication port for consensus | 10760 | Set to any unoccupied port |
dn_target_config_node_list | Running ConfigNode of the Cluster | 127.0.0.1:10710 | Set to any running ConfigNode's cn_internal_address:cn_internal_port. You can set multiple values, separate them with commas(“,”) |
Notice: The preceding configuration parameters cannot be changed after the node is started. Ensure that all ports are not occupied. Otherwise, the Node cannot be started.
This section describes how to start a cluster that includes several ConfigNodes and DataNodes. The cluster can provide services only by starting at least one ConfigNode and no less than the number of data/schema_replication_factor DataNodes.
The total process are three steps:
The first Node started in the cluster must be ConfigNode. The first started ConfigNode must follow the tutorial in this section.
The first ConfigNode to start is the Seed-ConfigNode, which marks the creation of the new cluster. Before start the Seed-ConfigNode, please open the common configuration file ./conf/iotdb-common.properties and check the following parameters:
Configuration | Check |
---|---|
cluster_name | Is set to the expected name |
config_node_consensus_protocol_class | Is set to the expected consensus protocol |
schema_replication_factor | Is set to the expected schema replication count |
schema_region_consensus_protocol_class | Is set to the expected consensus protocol |
data_replication_factor | Is set to the expected data replication count |
data_region_consensus_protocol_class | Is set to the expected consensus protocol |
Notice: Please set these parameters carefully based on the Deployment Recommendation. These parameters are not modifiable after the Node first startup.
Then open its configuration file ./conf/iotdb-confignode.properties and check the following parameters:
Configuration | Check |
---|---|
cn_internal_address | Is set to the IPV4 address or domain name of the server |
cn_internal_port | The port isn't occupied |
cn_consensus_port | The port isn't occupied |
cn_target_config_node_list | Is set to its own internal communication address, which is cn_internal_address:cn_internal_port |
After checking, you can run the startup script on the server:
# Linux foreground bash ./sbin/start-confignode.sh # Linux background nohup bash ./sbin/start-confignode.sh >/dev/null 2>&1 & # Windows .\sbin\start-confignode.bat
For more details about other configuration parameters of ConfigNode, see the ConfigNode Configurations.
The ConfigNode who isn't the first one started must follow the tutorial in this section.
You can add more ConfigNodes to the cluster to ensure high availability of ConfigNodes. A common configuration is to add extra two ConfigNodes to make the cluster has three ConfigNodes.
Ensure that all configuration parameters in the ./conf/iotdb-common.properites are the same as those in the Seed-ConfigNode; otherwise, it may fail to start or generate runtime errors. Therefore, please check the following parameters in common configuration file:
Configuration | Check |
---|---|
cluster_name | Is consistent with the Seed-ConfigNode |
config_node_consensus_protocol_class | Is consistent with the Seed-ConfigNode |
schema_replication_factor | Is consistent with the Seed-ConfigNode |
schema_region_consensus_protocol_class | Is consistent with the Seed-ConfigNode |
data_replication_factor | Is consistent with the Seed-ConfigNode |
data_region_consensus_protocol_class | Is consistent with the Seed-ConfigNode |
Then, please open its configuration file ./conf/iotdb-confignode.properties and check the following parameters:
Configuration | Check |
---|---|
cn_internal_address | Is set to the IPV4 address or domain name of the server |
cn_internal_port | The port isn't occupied |
cn_consensus_port | The port isn't occupied |
cn_target_config_node_list | Is set to the internal communication address of an other running ConfigNode. The internal communication address of the seed ConfigNode is recommended. |
After checking, you can run the startup script on the server:
# Linux foreground bash ./sbin/start-confignode.sh # Linux background nohup bash ./sbin/start-confignode.sh >/dev/null 2>&1 & # Windows .\sbin\start-confignode.bat
For more details about other configuration parameters of ConfigNode, see the ConfigNode Configurations.
Before adding DataNodes, ensure that there exists at least one ConfigNode is running in the cluster.
You can add any number of DataNodes to the cluster. Before adding a new DataNode,
please open its common configuration file ./conf/iotdb-common.properties and check the following parameters:
Configuration | Check |
---|---|
cluster_name | Is consistent with the Seed-ConfigNode |
Then open its configuration file ./conf/iotdb-datanode.properties and check the following parameters:
Configuration | Check |
---|---|
dn_rpc_address | Is set to the IPV4 address or domain name of the server |
dn_rpc_port | The port isn't occupied |
dn_internal_address | Is set to the IPV4 address or domain name of the server |
dn_internal_port | The port isn't occupied |
dn_mpp_data_exchange_port | The port isn't occupied |
dn_data_region_consensus_port | The port isn't occupied |
dn_schema_region_consensus_port | The port isn't occupied |
dn_target_config_node_list | Is set to the internal communication address of other running ConfigNodes. The internal communication address of the seed ConfigNode is recommended. |
After checking, you can run the startup script on the server:
# Linux foreground bash ./sbin/start-datanode.sh # Linux background nohup bash ./sbin/start-datanode.sh >/dev/null 2>&1 & # Windows .\sbin\start-datanode.bat
For more details about other configuration parameters of DataNode, see the DataNode Configurations.
Notice: The cluster can provide services only if the number of its DataNodes is no less than the number of replicas(max{schema_replication_factor, data_replication_factor}).
If the cluster is in local environment, you can directly run the Cli startup script in the ./sbin directory:
# Linux ./sbin/start-cli.sh # Windows .\sbin\start-cli.bat
If you want to use the Cli to connect to a cluster in the production environment, Please read the Cli manual.
Use a 3C3D(3 ConfigNodes and 3 DataNodes) as an example. Assumed that the IP addresses of the 3 ConfigNodes are 192.168.1.10, 192.168.1.11 and 192.168.1.12, and the default ports 10710 and 10720 are used. Assumed that the IP addresses of the 3 DataNodes are 192.168.1.20, 192.168.1.21 and 192.168.1.22, and the default ports 6667, 10730, 10740, 10750 and 10760 are used.
After starting the cluster successfully according to chapter 6.1, you can run the show cluster details
command on the Cli, and you will see the following results:
IoTDB> show cluster details +------+----------+-------+---------------+------------+-------------------+------------+-------+-------+-------------------+-----------------+ |NodeID| NodeType| Status|InternalAddress|InternalPort|ConfigConsensusPort| RpcAddress|RpcPort|MppPort|SchemaConsensusPort|DataConsensusPort| +------+----------+-------+---------------+------------+-------------------+------------+-------+-------+-------------------+-----------------+ | 0|ConfigNode|Running| 192.168.1.10| 10710| 10720| | | | | | | 2|ConfigNode|Running| 192.168.1.11| 10710| 10720| | | | | | | 3|ConfigNode|Running| 192.168.1.12| 10710| 10720| | | | | | | 1| DataNode|Running| 192.168.1.20| 10730| |192.168.1.20| 6667| 10740| 10750| 10760| | 4| DataNode|Running| 192.168.1.21| 10730| |192.168.1.21| 6667| 10740| 10750| 10760| | 5| DataNode|Running| 192.168.1.22| 10730| |192.168.1.22| 6667| 10740| 10750| 10760| +------+----------+-------+---------------+------------+-------------------+------------+-------+-------+-------------------+-----------------+ Total line number = 6 It costs 0.012s
If the status of all Nodes is Running, the cluster deployment is successful. Otherwise, read the run logs of the Node that fails to start and check the corresponding configuration parameters.
This section describes how to manually shut down the ConfigNode or DataNode process of the IoTDB.
Run the stop ConfigNode script:
# Linux ./sbin/stop-confignode.sh # Windows .\sbin\stop-confignode.bat
Run the stop DataNode script:
# Linux ./sbin/stop-datanode.sh # Windows .\sbin\stop-datanode.bat
Get the process number of the Node:
jps # or ps aux | grep iotdb
Kill the process:
kill -9 <pid>
Notice Some ports require root access, in which case use sudo
This section describes how to remove ConfigNode or DataNode from the cluster.
Before removing a ConfigNode, ensure that there is at least one active ConfigNode in the cluster after the removal. Run the remove-confignode script on an active ConfigNode:
# Linux # Remove the ConfigNode with confignode_id ./sbin/remove-confignode.sh <confignode_id> # Remove the ConfigNode with address:port ./sbin/remove-confignode.sh <cn_internal_address>:<cn_internal_port> # Windows # Remove the ConfigNode with confignode_id .\sbin\remove-confignode.bat <confignode_id> # Remove the ConfigNode with address:port .\sbin\remove-confignode.bat <cn_internal_address>:<cn_internal_portcn_internal_port>
Before removing a DataNode, ensure that the cluster has at least the number of data/schema replicas DataNodes. Run the remove-datanode script on an active DataNode:
# Linux # Remove the DataNode with datanode_id ./sbin/remove-datanode.sh <datanode_id> # Remove the DataNode with rpc address:port ./sbin/remove-datanode.sh <dn_rpc_address>:<dn_rpc_port> # Windows # Remove the DataNode with datanode_id .\sbin\remove-datanode.bat <datanode_id> # Remove the DataNode with rpc address:port .\sbin\remove-datanode.bat <dn_rpc_address>:<dn_rpc_port>
See FAQ.