blob: 85b5c74099f8093683cdfe74a191e654ca4d34bf [file] [view]
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# TypeScript Coordinator Runtime Example
This example shows the coordinator-mode shape for TypeScript task handlers:
- `dags/typescript_example.py` and `dags/typescript_taskflow_example.py` declare two Airflow Dags and their stub tasks.
- `src/main.ts` and `src/taskflow.ts` register a `TaskHandler` per stub task and start the coordinator runtime.
One bundle provides for both Dags, and both declare a task called `build_message`.
A handler binds the `(dag_id, task_id)` pair, so the two are different tasks with different bodies.
- `dist/bundle.min.mjs` is the generated Node.js bundle that Airflow launches.
The build uses the SDK's `airflow-ts-pack` tool, which bundles the entrypoint
with esbuild and embeds the Airflow metadata generated from the bundle's
registered tasks, producing a single deployable file.
## Build
Build the SDK first so the example can import the local package:
```bash
cd ts-sdk
pnpm install
pnpm run build
```
Build the example bundle and its metadata:
```bash
cd ts-sdk/example
pnpm install
pnpm run build
```
The coordinator expects this layout:
```text
ts-sdk/example/dist/
bundle.min.mjs
```
## Airflow Configuration
Register the example bundle directory as a Dag bundle, and configure Airflow to
route the `typescript` queue to the Node coordinator that reads it:
```bash
export AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST='[
{"name": "dags-folder", "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle", "kwargs": {}},
{
"name": "ts-task-handlers",
"classpath": "airflow.dag_processing.bundles.local.LocalDagBundle",
"kwargs": {"path": "/absolute/path/to/airflow/ts-sdk/example/dist"}
}
]'
export AIRFLOW__SDK__COORDINATORS='{
"ts": {
"classpath": "airflow.sdk.coordinators.node.NodeCoordinator",
"kwargs": {"task_handler_bundle_name": "ts-task-handlers"}
}
}'
export AIRFLOW__SDK__QUEUE_TO_COORDINATOR='{"typescript": "ts"}'
```
Copy both files in `dags/` into your Airflow Dags folder.
The example also reads one Variable and one Connection:
```bash
airflow variables set typescript_example_greeting "hello from Airflow"
airflow connections add typescript_example_http \
--conn-type http \
--conn-host example.com \
--conn-login user \
--conn-password pass
```
`write_and_delete_variable` writes the Variables it needs: it records the run id in
`typescript_example_last_run` and deletes the `typescript_example_scratch` Variable it has just written.
Then start Airflow and trigger the Dag:
```bash
airflow dags trigger typescript_example
airflow dags trigger typescript_taskflow_example
```