tree: 922b380d343d7529c35137ad19d1263a533e2077 [search]
  1. adr/
  2. api-docs/
  3. docs/
  4. example/
  5. schema/
  6. scripts/
  7. src/
  8. tests/
  9. .pre-commit-config.yaml
  10. .prettierignore
  11. .prettierrc.json
  12. capabilities.yaml
  13. DEVELOPMENT.md
  14. eslint.config.js
  15. LICENSE
  16. NOTICE
  17. package.json
  18. pnpm-lock.yaml
  19. pnpm-workspace.yaml
  20. README.md
  21. tsconfig.build.json
  22. tsconfig.json
  23. vitest.config.ts
ts-sdk/README.md

Airflow TypeScript SDK

Write Apache Airflow Dags and tasks in TypeScript.

Status: 0.1.0-beta1 · API may change · Node 22+ · ESM-only

There are two ways to use it:

  • Dag definition: declare the whole Dag in TypeScript, with its schedule, tasks, options and dependencies.
  • Python Dag with stub TaskHandler: a TypeScript TaskHandler implements each stub task of a Python Dag.

Both use the same task API, and one bundle can serve both.

Installation

npm install apache-airflow-ts-sdk@0.1.0-beta1
npm install --save-dev esbuild

Declaring a Dag

import { Bundle, Dag, getClient } from "apache-airflow-ts-sdk";

const dag = new Dag("sales_pipeline", { schedule: "@daily", queue: "typescript" });

const extract = dag.task("extract", async (): Promise<number> => {
  return Number((await getClient().getVariable("daily_row_count")) ?? "0");
});
const transform = dag.task(
  "transform",
  async ({ rows, region }: { rows: number; region: string }) => ({ rows, region }),
);

transform({ rows: extract(), region: "us" });

await new Bundle(dag).serve();

A handler takes one object of named arguments, and calling a task names its inputs. An input is either the reference another task's call returned, which makes this task wait for it and receive its value, or a literal.

A Dag also has order-only dependencies (before and after), task groups, branching with dag.if and dag.switch, and triggerDagRun. See the guide for all of them.

Implementing stub tasks of a Python Dag

The Python Dag declares the tasks and their dependencies:

from airflow.sdk import dag, task


@dag
def sales_pipeline():
    @task.stub(queue="typescript")
    def extract(): ...

    @task.stub(queue="typescript")
    def transform(extracted): ...

    transform(extract())


sales_pipeline()

TypeScript implements them, with a TaskHandler per stub task:

import { Bundle, getClient, TaskHandler } from "apache-airflow-ts-sdk";

export async function extract() {
  const rowCount = Number((await getClient().getVariable("daily_row_count")) ?? "0");
  return { rowCount };
}

export async function transform({ extracted }: { extracted: { rowCount: number } }) {
  return { transformedRows: extracted.rowCount };
}

await new Bundle(
  new TaskHandler("sales_pipeline", "extract", extract),
  new TaskHandler("sales_pipeline", "transform", transform),
).serve();

A TaskHandler names the dag_id and task_id it implements, so the same task_id under two Dags is two different handlers. Nothing connects to Airflow until bundle.serve(), so a unit test can build a bundle and call a handler through bundle.getTaskHandler(dagId, taskId).

TaskFlow arguments

A Python Dag that calls a stub task TaskFlow-style passes those arguments to the handler, which destructures them by name:

@task.stub(queue="typescript")
def transform(region_code: str, threshold: float, dry_run: bool = False): ...


transform("uk", 0.75)
export async function transform({ regionCode, threshold, dryRun }: TransformArgs) {
  // ...
}

Names match ignoring case and underscores, so region_code reaches regionCode with nothing declared on either side. An argument filled from another task, as in transform(extract(), "uk"), arrives as that task's value. A dependency drawn with >> orders the tasks and passes nothing; read such a value with getClient().getXCom.

withArgNames maps a handler's name to a different Python name, when the two cannot match on their own:

const report = withArgNames({ label: "run_label" }, async ({ summary, label }: ReportArgs) => {
  // `label` is the call's `run_label`.
});

Writing tasks

A handler is a plain, usually async, function. getContext() and getClient() give it the task‘s context and Airflow access while it runs, so it takes no SDK-supplied argument. A value it returns becomes the task’s return_value XCom, and an uncaught error fails the task.

Building and deploying

airflow-ts-pack builds the entry module and everything it imports into one file, dist/bundle.min.mjs:

npx airflow-ts-pack src/main.ts --outdir dist
  • --outdir <dir>: output directory (default dist)
  • --outfile <path>: exact output path, whose name must end in .min.mjs

A bundle that declares Dags goes into a Dag bundle, such as the default dags-folder bundle. Route its queue to the Node.js coordinator:

[sdk]
coordinators = {"ts": {"classpath": "airflow.sdk.coordinators.node.NodeCoordinator"}}
queue_to_coordinator = {"typescript": "ts"}

A bundle that only implements stub tasks goes next to the Python Dag that declares them, or into the Dag bundle the coordinator's task_handler_bundle_name names. See the guide for all coordinator options.

See example/ for a working project that serves both a Dag declared in TypeScript and the tasks of two Python Dags from one bundle.

Human-in-the-loop tasks

A Dag declared in TypeScript can wait for a person, as HITLOperator and ApprovalOperator do in Python. approval(...) and hitl(...) come from the apache-airflow-ts-sdk/hitl subpath and are passed to dag.task in place of a handler. Each is named after its Python operator without “Operator”, and each option is the Python keyword argument in camelCase:

import { Bundle, Dag } from "apache-airflow-ts-sdk";
import { approval } from "apache-airflow-ts-sdk/hitl";

const dag = new Dag("release", { queue: "typescript" });
const report = dag.task("build_report", async () => ({ version: "1.4" }))();

const signOff = dag.task(
  "sign_off",
  approval({
    subject: ({ report }: { report: { version: string } }) => `Ship ${report.version}?`,
    responseTimeout: 4 * 60 * 60, // seconds
    defaults: "Reject", // the answer given when the timeout passes
  }),
)({ report });
signOff.before(dag.task("publish", async () => {})());

await new Bundle(dag).serve();

The task's first run writes the request and parks the task in awaiting_input; its Node process exits, so no worker slot is held while it waits. Once someone responds on the Required Actions page, or the timeout passes, the task resumes in a new process. It does not write the request again or run subject or body again.

  • subject and body (Markdown) are a string, or a function of the task's inputs, run once. They may read different inputs, and the task takes both.
  • responseTimeout is in whole seconds. With defaults, the defaults are applied when it passes; without, the task fails, and its retries apply. A HITL task does not enforce executionTimeout, so dag.task rejects it and asks for responseTimeout instead.
  • assignedUsers: [{ id, name }] limits who can respond, by the auth manager's user id.
  • The task returns the response, which downstream tasks receive and XCom stores under return_value: { chosenOptions, paramsInput, respondedAt, respondedByUser, timedout }. These are the keys of Python's HITLOperator result in camelCase, plus timedout; respondedAt is an ISO-8601 string, and respondedByUser is null when the timeout defaults were applied.

On “Reject”, an approval succeeds and skips the tasks directly downstream by default, as ApprovalOperator does. The run then shows as successful, with the guarded tasks skipped. Set failOnReject: true to fail the approval instead, or ignoreDownstreamTriggerRules: true to skip every task downstream. hitl(...) offers any options, with multiple for several, and never skips or fails on what is chosen, even an option named “Reject”.

A failed resume retries, and the retry writes a fresh request, so approval({ failOnReject: true }) with retries asks the reviewer again on each attempt.

When the task resumes, it reads its options, its params and its reject options from the Dag it runs from, as HITLOperator does. A versioned Dag bundle resumes it from the same version it started on; with an unversioned bundle, a bundle deployed while the request waits is what the response is read against. Clearing the task starts a new attempt, which writes a fresh request.

Forms

params adds form fields to the request, as HITLOperator's params does, for approval(...) and hitl(...) alike. Each field is a value that pre-fills it, an optional description, and an optional JSON Schema that the form renders:

import { approval } from "apache-airflow-ts-sdk/hitl";

const rollout = dag.task(
  "size_rollout",
  approval({
    subject: "Approve the rollout?",
    defaults: "Reject",
    params: {
      replicas: {
        value: 3,
        description: "How many replicas to start",
        schema: { type: "integer", minimum: 1, maximum: 10 },
      },
    },
  }),
)();

The answers arrive as paramsInput, keyed by the param names. When the timeout applies defaults, each param's value is recorded as its answer. An answer that fills in some params but not exactly the declared ones fails the task, as HITLOperator does. The values are not yet checked against their schema when the task resumes, so an answer given through the REST API can fall outside it.

A hitl(...) whose options are exactly “Approve” and “Reject” gets the approval buttons in the UI but never skips anything; use approval(...) when Reject should stop the tasks after it.

Branching on the response and notifiers are not supported yet.

TaskClient

getClient() returns a TaskClient for task-time Airflow data access, for as long as a handler is running:

MethodDescription
getVariable(key) / getVariableOrThrowAirflow Variables
setVariable(key, value, description?) / deleteVariable(key)Variable write / delete
getXCom(opts) / setXCom(opts)XCom read/write
getConnection(connId) / getConnectionOrThrowAirflow Connections
taskStateStore.get / set / delete / clearPer-task-instance key/value state store

Locator fields such as dagId, runId, and taskId default to the current task context when omitted.

Cancellation

getContext().signal is an AbortSignal controlled by the active runtime. Pass it to fetch(), timers, database clients, child processes, or any other API that accepts an abort signal, so tasks can clean up cooperatively when Airflow terminates the task subprocess.

Compatibility matrix

Which Airflow TaskInstance states and capabilities this SDK supports. This table is generated from capabilities.yaml; the conformance dimensions are defined in the Language SDK conformance spec. Do not edit the table by hand. Update the manifest and run the update-ts-sdk-readme-matrix prek hook.

Min. Airflow version: 3.4 · supervisor schema: 2026-10-30

DimensionTierSupportedSinceNotes
TaskInstance states
state: successMUST✓3.4
state: failedMUST✓3.4
state: up_for_retryMUST✓3.4RetryTask
state: skippedSHOULD✗–runtime does not emit TaskState skipped yet
state: deferredMAY✗–runtime does not emit DeferTask yet
state: up_for_rescheduleMAY✗–runtime does not emit RescheduleTask yet
state: awaiting_inputMAY✓3.4approval() / hitl() in a native Dag
state: removedMAY✓3.4
Runtime capabilities
capability: mixed-lang-stub-targetMUST✓3.4@task.stub
capability: taskflow-bindingMUST✗–bind @task.stub literal/XCom args to the native handler
capability: task-loggingMUST✓3.4structured records over the log socket
capability: xcom-read-writeMUST✓3.4getXCom / setXCom
capability: connection-readMUST✓3.4getConnection
capability: variable-read-writeMUST✓3.4getVariable / setVariable / deleteVariable
capability: self-contained-bundleMUST✓3.4Airflow metadata embedded in the bundle
capability: retry-policyMAY✗–no task-facing retry-policy API yet
capability: task-state-storeMAY✓3.4taskStateStore.get / set / delete / clear
capability: asset-state-storeMAY✗–no task-facing state-store API yet
capability: asset-event-emitMAY✗–runtime does not emit asset events yet
capability: asset-event-readMAY✗–no task-facing asset-event API yet
Native-Dag authoring
capability: native-dag-authoringSHOULD✗–native Dag authoring not implemented yet
capability: task-argsMUST †n/a–
capability: dag-paramsMUST †n/a–
capability: taskflow-dependenciesMUST †n/a–
capability: branchingSHOULD †n/a–
capability: dag-testSHOULD †n/a–
capability: task-groupMAY †n/a–
capability: dynamic-task-mappingMAY †n/a–
capability: asset-inlets-outletsMAY †n/a–
capability: asset-schedulingMAY †n/a–
capability: object-storeMAY †n/a–no object-storage API yet

Marks: ✓ supported · ✗ not supported · n/a not applicable. A tier marked † applies only when native-dag-authoring is supported.

Links