SeaTunnel already has separate pages for source architecture, sink architecture, catalog metadata, and translation. What is still missing is one page that explains how those APIs fit together as a single design.
This page gives that bridge.
SeaTunnel's core API design has one primary goal:
connector developers should express data integration logic once, while engines handle execution differences underneath.
To make that work, the API layer has to do three things at the same time:
At a high level, SeaTunnel's API layer is organized like this:
User Config | v Option / OptionRule / ReadonlyConfig | v Factory Layer - TableSourceFactory - TableSinkFactory - Transform factory | v Runtime Contracts - SeaTunnelSource - SeaTunnelSink - SeaTunnelTransform | v Metadata Contracts - CatalogTable - TableSchema - SeaTunnelDataType - SchemaChangeEvent | v Translation / Engine Runtime
The important point is that connector logic sits in the middle: high enough to avoid engine coupling, but rich enough to describe real-world data pipelines.
Before a connector can run, SeaTunnel needs a stable way to describe user-facing options and validate them.
The configuration contract is centered on:
OptionOptionRuleReadonlyConfigThis part of the API matters because it connects:
Related docs:
The source side is responsible for turning an external system into a stream of SeaTunnelRow records, plus schema and state metadata when needed.
Core interfaces:
SeaTunnelSourceSourceSplitEnumeratorSourceReaderSourceSplitThe design separates coordination from execution:
This makes parallelism, failover, and checkpoint recovery possible without forcing each connector to invent its own execution model.
Related docs:
The sink side turns processed rows into externally visible side effects.
Core interfaces:
SeaTunnelSinkSinkWriterSinkCommitterSinkAggregatedCommitterThe design goal is not only “write data out”, but to let a sink clearly define:
Related docs:
Transforms are the middle layer between source and sink. They operate on SeaTunnel's row and table model rather than on engine-native records.
This gives SeaTunnel a consistent contract for:
The transform contract is what allows a job to remain declarative even when the physical runtime changes underneath.
Related docs:
Row processing alone is not enough for a serious integration system. SeaTunnel also needs a portable metadata model that can describe:
That is the role of:
CatalogTableTableSchemaColumnSeaTunnelDataTypeSchemaChangeEventThis metadata layer is essential for:
Related docs:
In a typical SeaTunnel job, the API contracts interact in this order:
CatalogTable metadataThis separation is why SeaTunnel can keep connector logic reusable across multiple engines without forcing connector authors to depend on engine-specific APIs.
User configuration changes more slowly than internal runtime details. By separating Option and OptionRule from reader and writer logic, SeaTunnel can keep user-facing configuration stable while evolving execution internals.
Rows, schema, and schema changes have different lifecycles. A connector may read rows continuously, while metadata changes only occasionally. Keeping those contracts distinct makes the system easier to reason about and extend.
If connector implementations were written directly against Flink or Spark APIs, every connector would need multiple engine-specific versions. The SeaTunnel API layer avoids that duplication.
Related docs:
When adding or reviewing an API-related change, check these questions first:
OptionRule, runtime APIs, or metadata APIs?These questions matter because API drift is harder to unwind than implementation drift.
Not really. Without CatalogTable, SeaTunnelDataType, and schema change events, connectors would have no engine-independent way to express table metadata and schema evolution.
Not always. In SeaTunnel, transform logic may also need to preserve or reshape metadata, especially in multi-table and CDC pipelines.
It is an adapter layer, but it is also a design boundary. It keeps connector authors from depending on engine internals and limits how much engine-specific behavior leaks into connector code.