以本地环境为例,演示 IoTDB 集群的启动、扩容与缩容。
注意:本文档为使用本地不同端口,进行伪分布式环境部署的教程,仅用于练习。在真实环境部署时,一般不需要修改节点端口,仅需配置节点 IPV4 地址或域名即可。
解压 apache-iotdb-1.0.0-all-bin.zip 至 cluster0 目录。
在 Linux 环境中,部署 1 个 ConfigNode 和 1 个 DataNode(1C1D)集群版,默认 1 副本:
./cluster0/sbin/start-confignode.sh ./cluster0/sbin/start-datanode.sh
./cluster0/sbin/start-cli.sh
IoTDB> show cluster details +------+----------+-------+---------------+------------+-------------------+----------+-------+--------+-------------------+-----------------+ |NodeID| NodeType| Status|InternalAddress|InternalPort|ConfigConsensusPort|RpcAddress|RpcPort|MppPort |SchemaConsensusPort|DataConsensusPort| +------+----------+-------+---------------+------------+-------------------+----------+-------+--------+-------------------+-----------------+ | 0|ConfigNode|Running| 127.0.0.1| 10710| 10720| | | | | | | 1| DataNode|Running| 127.0.0.1| 10730| | 127.0.0.1| 6667| 10740| 10750| 10760| +------+----------+-------+---------------+------------+-------------------+----------+-------+--------+-------------------+-----------------+ Total line number = 2 It costs 0.242s
解压 apache-iotdb-1.0.0-all-bin.zip 至 cluster1 目录和 cluster2 目录
对于 cluster1 目录:
| 配置项 | 值 |
|---|---|
| cn_internal_address | 127.0.0.1 |
| cn_internal_port | 10711 |
| cn_consensus_port | 10721 |
| cn_target_config_node_list | 127.0.0.1:10710 |
| 配置项 | 值 |
|---|---|
| dn_rpc_address | 127.0.0.1 |
| dn_rpc_port | 6668 |
| dn_internal_address | 127.0.0.1 |
| dn_internal_port | 10731 |
| dn_mpp_data_exchange_port | 10741 |
| dn_schema_region_consensus_port | 10751 |
| dn_data_region_consensus_port | 10761 |
| dn_target_config_node_list | 127.0.0.1:10710 |
对于 cluster2 目录:
| 配置项 | 值 |
|---|---|
| cn_internal_address | 127.0.0.1 |
| cn_internal_port | 10712 |
| cn_consensus_port | 10722 |
| cn_target_config_node_list | 127.0.0.1:10710 |
| 配置项 | 值 |
|---|---|
| dn_rpc_address | 127.0.0.1 |
| dn_rpc_port | 6669 |
| dn_internal_address | 127.0.0.1 |
| dn_internal_port | 10732 |
| dn_mpp_data_exchange_port | 10742 |
| dn_schema_region_consensus_port | 10752 |
| dn_data_region_consensus_port | 10762 |
| dn_target_config_node_list | 127.0.0.1:10710 |
将集群扩容至 3 个 ConfigNode 和 3 个 DataNode(3C3D)集群版, 指令执行顺序为先启动 ConfigNode,再启动 DataNode:
./cluster1/sbin/start-confignode.sh ./cluster2/sbin/start-confignode.sh ./cluster1/sbin/start-datanode.sh ./cluster2/sbin/start-datanode.sh
在 Cli 执行 show cluster details,结果如下:
IoTDB> show cluster details +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ |NodeID| NodeType| Status|InternalAddress|InternalPort|ConfigConsensusPort|RpcAddress|RpcPort|MppPort|SchemaConsensusPort|DataConsensusPort| +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ | 0|ConfigNode|Running| 127.0.0.1| 10710| 10720| | | | | | | 2|ConfigNode|Running| 127.0.0.1| 10711| 10721| | | | | | | 3|ConfigNode|Running| 127.0.0.1| 10712| 10722| | | | | | | 1| DataNode|Running| 127.0.0.1| 10730| | 127.0.0.1| 6667| 10740| 10750| 10760| | 4| DataNode|Running| 127.0.0.1| 10731| | 127.0.0.1| 6668| 10741| 10751| 10761| | 5| DataNode|Running| 127.0.0.1| 10732| | 127.0.0.1| 6669| 10742| 10752| 10762| +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ Total line number = 6 It costs 0.012s
# 使用 ip:port 移除 ./cluster0/sbin/remove-confignode.sh 127.0.0.1:10711 # 使用节点编号移除 ./cluster0/sbin/remove-confignode.sh 2
# 使用 ip:port 移除 ./cluster0/sbin/remove-datanode.sh 127.0.0.1:6668 # 使用节点编号移除 ./cluster0/sbin/remove-confignode.sh 4
在 Cli 执行 show cluster details,结果如下:
IoTDB> show cluster details +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ |NodeID| NodeType| Status|InternalAddress|InternalPort|ConfigConsensusPort|RpcAddress|RpcPort|MppPort|SchemaConsensusPort|DataConsensusPort| +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ | 0|ConfigNode|Running| 127.0.0.1| 10710| 10720| | | | | | | 3|ConfigNode|Running| 127.0.0.1| 10712| 10722| | | | | | | 1| DataNode|Running| 127.0.0.1| 10730| | 127.0.0.1| 6667| 10740| 10750| 10760| | 5| DataNode|Running| 127.0.0.1| 10732| | 127.0.0.1| 6669| 10742| 10752| 10762| +------+----------+-------+---------------+------------+-------------------+----------+-------+-------+-------------------+-----------------+ Total line number = 4 It costs 0.005s
IoTDB数据文件夹概览工具用于打印出数据文件夹的结构概览信息,工具位置为 tools/tsfile/print-iotdb-data-dir。
.\print-iotdb-data-dir.bat <IoTDB数据文件夹路径,如果是多个文件夹用逗号分隔> (<输出结果的存储路径>)
./print-iotdb-data-dir.sh <IoTDB数据文件夹路径,如果是多个文件夹用逗号分隔> (<输出结果的存储路径>)
注意:如果没有设置输出结果的存储路径, 将使用相对路径“IoTDB_data_dir_overview.txt”作为默认值。
以Windows系统为例:
.\print-iotdb-data-dir.bat D:\github\master\iotdb\data\datanode\data ```````````````````````` Starting Printing the IoTDB Data Directory Overview ```````````````````````` output save path:IoTDB_data_dir_overview.txt data dir num:1 143 [main] WARN o.a.i.t.c.conf.TSFileDescriptor - not found iotdb-common.properties, use the default configs. |============================================================== |D:\github\master\iotdb\data\datanode\data |--sequence | |--root.redirect0 | | |--1 | | | |--0 | |--root.redirect1 | | |--2 | | | |--0 | |--root.redirect2 | | |--3 | | | |--0 | |--root.redirect3 | | |--4 | | | |--0 | |--root.redirect4 | | |--5 | | | |--0 | |--root.redirect5 | | |--6 | | | |--0 | |--root.sg1 | | |--0 | | | |--0 | | | |--2760 |--unsequence |==============================================================
TsFile概览工具用于以概要模式打印出一个TsFile的内容,工具位置为 tools/tsfile/print-tsfile。
.\print-tsfile-sketch.bat <TsFile文件路径> (<输出结果的存储路径>)
./print-tsfile-sketch.sh <TsFile文件路径> (<输出结果的存储路径>)
注意:如果没有设置输出结果的存储路径, 将使用相对路径“TsFile_sketch_view.txt”作为默认值。
以Windows系统为例:
.\print-tsfile.bat D:\github\master\1669359533965-1-0-0.tsfile D:\github\master\sketch.txt ```````````````````````` Starting Printing the TsFile Sketch ```````````````````````` TsFile path:D:\github\master\1669359533965-1-0-0.tsfile Sketch save path:D:\github\master\sketch.txt 148 [main] WARN o.a.i.t.c.conf.TSFileDescriptor - not found iotdb-common.properties, use the default configs. -------------------------------- TsFile Sketch -------------------------------- file path: D:\github\master\1669359533965-1-0-0.tsfile file length: 2974 POSITION| CONTENT -------- ------- 0| [magic head] TsFile 6| [version number] 3 ||||||||||||||||||||| [Chunk Group] of root.sg1.d1, num of Chunks:3 7| [Chunk Group Header] | [marker] 0 | [deviceID] root.sg1.d1 20| [Chunk] of root.sg1.d1.s1, startTime: 1669359533948 endTime: 1669359534047 count: 100 [minValue:-9032452783138882770,maxValue:9117677033041335123,firstValue:7068645577795875906,lastValue:-5833792328174747265,sumValue:5.795959009889246E19] | [chunk header] marker=5, measurementID=s1, dataSize=864, dataType=INT64, compressionType=SNAPPY, encodingType=RLE | [page] UncompressedSize:862, CompressedSize:860 893| [Chunk] of root.sg1.d1.s2, startTime: 1669359533948 endTime: 1669359534047 count: 100 [minValue:-8806861312244965718,maxValue:9192550740609853234,firstValue:1150295375739457693,lastValue:-2839553973758938646,sumValue:8.2822564314572677E18] | [chunk header] marker=5, measurementID=s2, dataSize=864, dataType=INT64, compressionType=SNAPPY, encodingType=RLE | [page] UncompressedSize:862, CompressedSize:860 1766| [Chunk] of root.sg1.d1.s3, startTime: 1669359533948 endTime: 1669359534047 count: 100 [minValue:-9076669333460323191,maxValue:9175278522960949594,firstValue:2537897870994797700,lastValue:7194625271253769397,sumValue:-2.126008424849926E19] | [chunk header] marker=5, measurementID=s3, dataSize=864, dataType=INT64, compressionType=SNAPPY, encodingType=RLE | [page] UncompressedSize:862, CompressedSize:860 ||||||||||||||||||||| [Chunk Group] of root.sg1.d1 ends 2656| [marker] 2 2657| [TimeseriesIndex] of root.sg1.d1.s1, tsDataType:INT64, startTime: 1669359533948 endTime: 1669359534047 count: 100 [minValue:-9032452783138882770,maxValue:9117677033041335123,firstValue:7068645577795875906,lastValue:-5833792328174747265,sumValue:5.795959009889246E19] | [ChunkIndex] offset=20 2728| [TimeseriesIndex] of root.sg1.d1.s2, tsDataType:INT64, startTime: 1669359533948 endTime: 1669359534047 count: 100 [minValue:-8806861312244965718,maxValue:9192550740609853234,firstValue:1150295375739457693,lastValue:-2839553973758938646,sumValue:8.2822564314572677E18] | [ChunkIndex] offset=893 2799| [TimeseriesIndex] of root.sg1.d1.s3, tsDataType:INT64, startTime: 1669359533948 endTime: 1669359534047 count: 100 [minValue:-9076669333460323191,maxValue:9175278522960949594,firstValue:2537897870994797700,lastValue:7194625271253769397,sumValue:-2.126008424849926E19] | [ChunkIndex] offset=1766 2870| [IndexOfTimerseriesIndex Node] type=LEAF_MEASUREMENT | <s1, 2657> | <endOffset, 2870> ||||||||||||||||||||| [TsFileMetadata] begins 2891| [IndexOfTimerseriesIndex Node] type=LEAF_DEVICE | <root.sg1.d1, 2870> | <endOffset, 2891> | [meta offset] 2656 | [bloom filter] bit vector byte array length=31, filterSize=256, hashFunctionSize=5 ||||||||||||||||||||| [TsFileMetadata] ends 2964| [TsFileMetadataSize] 73 2968| [magic tail] TsFile 2974| END of TsFile ---------------------------- IndexOfTimerseriesIndex Tree ----------------------------- [MetadataIndex:LEAF_DEVICE] └──────[root.sg1.d1,2870] [MetadataIndex:LEAF_MEASUREMENT] └──────[s1,2657] ---------------------------------- TsFile Sketch End ----------------------------------
解释:
TsFile resource概览工具用于打印出TsFile resource文件的内容,工具位置为 tools/tsfile/print-tsfile-resource-files。
.\print-tsfile-resource-files.bat <TsFile resource文件所在的文件夹路径,或者单个TsFile resource文件路径>
./print-tsfile-resource-files.sh <TsFile resource文件所在的文件夹路径,或者单个TsFile resource文件路径>
以Windows系统为例:
.\print-tsfile-resource-files.bat D:\github\master\iotdb\data\datanode\data\sequence\root.sg1\0\0 ```````````````````````` Starting Printing the TsFileResources ```````````````````````` 147 [main] WARN o.a.i.t.c.conf.TSFileDescriptor - not found iotdb-common.properties, use the default configs. 230 [main] WARN o.a.iotdb.db.conf.IoTDBDescriptor - Cannot find IOTDB_HOME or IOTDB_CONF environment variable when loading config file iotdb-common.properties, use default configuration 231 [main] WARN o.a.iotdb.db.conf.IoTDBDescriptor - Couldn't load the configuration iotdb-common.properties from any of the known sources. 233 [main] WARN o.a.iotdb.db.conf.IoTDBDescriptor - Cannot find IOTDB_HOME or IOTDB_CONF environment variable when loading config file iotdb-datanode.properties, use default configuration 237 [main] WARN o.a.iotdb.db.conf.IoTDBDescriptor - Couldn't load the configuration iotdb-datanode.properties from any of the known sources. Analyzing D:\github\master\iotdb\data\datanode\data\sequence\root.sg1\0\0\1669359533489-1-0-0.tsfile ... Resource plan index range [9223372036854775807, -9223372036854775808] device root.sg1.d1, start time 0 (1970-01-01T08:00+08:00[GMT+08:00]), end time 99 (1970-01-01T08:00:00.099+08:00[GMT+08:00]) Analyzing the resource file folder D:\github\master\iotdb\data\datanode\data\sequence\root.sg1\0\0 finished.
.\print-tsfile-resource-files.bat D:\github\master\iotdb\data\datanode\data\sequence\root.sg1\0\0\1669359533489-1-0-0.tsfile.resource ```````````````````````` Starting Printing the TsFileResources ```````````````````````` 178 [main] WARN o.a.iotdb.db.conf.IoTDBDescriptor - Cannot find IOTDB_HOME or IOTDB_CONF environment variable when loading config file iotdb-common.properties, use default configuration 186 [main] WARN o.a.i.t.c.conf.TSFileDescriptor - not found iotdb-common.properties, use the default configs. 187 [main] WARN o.a.iotdb.db.conf.IoTDBDescriptor - Couldn't load the configuration iotdb-common.properties from any of the known sources. 188 [main] WARN o.a.iotdb.db.conf.IoTDBDescriptor - Cannot find IOTDB_HOME or IOTDB_CONF environment variable when loading config file iotdb-datanode.properties, use default configuration 192 [main] WARN o.a.iotdb.db.conf.IoTDBDescriptor - Couldn't load the configuration iotdb-datanode.properties from any of the known sources. Analyzing D:\github\master\iotdb\data\datanode\data\sequence\root.sg1\0\0\1669359533489-1-0-0.tsfile ... Resource plan index range [9223372036854775807, -9223372036854775808] device root.sg1.d1, start time 0 (1970-01-01T08:00+08:00[GMT+08:00]), end time 99 (1970-01-01T08:00:00.099+08:00[GMT+08:00]) Analyzing the resource file D:\github\master\iotdb\data\datanode\data\sequence\root.sg1\0\0\1669359533489-1-0-0.tsfile.resource finished.