SeaTunnel支持对每个实例添加tag
, 然后在提交任务时可以在配置文件中使用tag_filter
来选择任务将要运行的节点.
更新hazelcast.yaml
文件
hazelcast: cluster-name: seatunnel network: rest-api: enabled: true endpoint-groups: CLUSTER_WRITE: enabled: true DATA: enabled: true join: tcp-ip: enabled: true member-list: - localhost port: auto-increment: false port: 5801 properties: hazelcast.invocation.max.retry.count: 20 hazelcast.tcp.join.port.try.count: 30 hazelcast.logging.type: log4j2 hazelcast.operation.generic.thread.count: 50 member-attributes: group: type: string value: platform team: type: string value: team1
在这个配置中, 我们通过member-attributes
设置了group=platform, team=team1
这样两个tag
在任务的配置中添加tag_filter
来选择你需要运行该任务的节点
env { parallelism = 1 job.mode = "BATCH" tag_filter { group = "platform" team = "team1" } } source { FakeSource { plugin_output = "fake" parallelism = 1 schema = { fields { name = "string" } } } } transform { } sink { console { plugin_input="fake" } }
**注意:**
tag_filter
时, 会从所有节点中随机选择节点来运行任务.tag_filter
中存在多个过滤条件时, 会根据key存在以及value相等的全部匹配的节点, 当没有找到匹配的节点时, 会抛出 NoEnoughResourceException
异常.
更新运行中node的tags (可选)
获取具体的使用信息,请参考 更新运行节点的tags