Welcome to the Apache Hamilton Ecosystem page! This page showcases the integrations, plugins, and external resources available for Apache Hamilton users.
Learn Apache Hamilton concepts through interactive, browser-based tutorials.
Apache Hamilton provides first-class support for many popular data science and engineering tools through built-in plugins and adapters. These integrations are maintained by the Apache Hamilton community and included in the core project.
Apache Hamilton integrates seamlessly with popular data manipulation libraries:
| Integration | Description | Documentation |
|---|---|---|
| pandas | DataFrame operations and transformations | Examples | ResultBuilder |
| Polars | High-performance DataFrame library | Examples | ResultBuilder |
| PySpark | Distributed data processing with Spark | Examples | GraphAdapter |
| Dask | Parallel computing and distributed arrays | Examples | GraphAdapter |
| Ray | Distributed computing framework | Examples | GraphAdapter |
| Ibis | Portable DataFrame API across backends | Integration Guide |
| Vaex | Out-of-core DataFrame library | Examples |
| Narwhals | DataFrame-agnostic interface | Examples | Lifecycle Hook |
| NumPy | Numerical computing arrays | ResultBuilder |
| PyArrow | Columnar in-memory data | ResultBuilder |
Build and deploy ML workflows with Apache Hamilton:
| Integration | Description | Documentation |
|---|---|---|
| MLflow | Experiment tracking and model registry | Examples | Lifecycle Hook |
| scikit-learn | Machine learning algorithms | Examples |
| XGBoost | Gradient boosting framework | IO Adapters |
| LightGBM | Gradient boosting framework | IO Adapters |
| Hugging Face | Transformers and NLP models | IO Adapters |
| Pandera | DataFrame validation | Examples |
| Pydantic | Data validation and settings | Decorator |
Use Apache Hamilton within your existing orchestration infrastructure:
| Integration | Description | Documentation |
|---|---|---|
| Airflow | Workflow orchestration platform | Examples |
| Dagster | Data orchestrator | Examples |
| Prefect | Workflow orchestration | Examples |
| Kedro | Data science pipelines | Examples |
| Metaflow | ML infrastructure | Integration |
| dbt | Data transformation tool | Integration Guide |
Tools for building robust data pipelines:
| Integration | Description | Documentation |
|---|---|---|
| dlt | Data loading and transformation | Integration Guide |
| Feast | Feature store | Examples |
| FastAPI | Web service framework | Integration Guide |
| Streamlit | Interactive web applications | Integration Guide |
Track and monitor your Apache Hamilton dataflows:
| Integration | Description | Documentation |
|---|---|---|
| Datadog | Monitoring and analytics | Lifecycle Hook |
| OpenTelemetry | Observability framework | Examples |
| OpenLineage | Data lineage tracking | Examples | Lifecycle Hook |
| Hamilton UI | Built-in execution tracking | UI Guide |
| Experiment Manager | Lightweight experiment tracking | Examples |
Create visualizations from your dataflows:
| Integration | Description | Documentation |
|---|---|---|
| Plotly | Interactive plotting | Examples |
| Matplotlib | Static plotting | IO Adapters |
| Rich | Terminal formatting and progress | Lifecycle Hook |
Improve your development workflow:
| Integration | Description | Documentation |
|---|---|---|
| Jupyter | Notebook magic commands | Examples |
| VS Code | Language server and extension | VS Code Guide |
| Claude Code | AI assistant plugin for Hamilton development | Plugin Guide |
| MCP Server | LLM tool server for interactive DAG development | MCP Guide |
| tqdm | Progress bars | Lifecycle Hook |
Deploy Apache Hamilton to the cloud:
| Integration | Description | Documentation |
|---|---|---|
| AWS | Amazon Web Services | Examples |
| Google Cloud | Google Cloud Platform | Scale-up Guide |
| Modal | Serverless cloud functions | Scale-up Guide |
Persist and cache your data:
| Integration | Description | Documentation |
|---|---|---|
| DiskCache | Disk-based caching | Examples |
| File-based caching | Local file caching | Caching Guide |
| Integration | Description | Documentation |
|---|---|---|
| Slack | Notifications and integrations | Examples | Lifecycle Hook |
| GeoPandas | Geospatial data analysis | Type extension for GeoDataFrame support |
| YAML | Configuration management | IO Adapters |
| Neo4j | Knowledge graph RAG | Examples |
The following resources and services are provided by third parties and the broader Apache Hamilton community.
⚠️ Important Notice:
These resources and services are not maintained, nor endorsed by the Apache Hamilton Community and Apache Hamilton project (maintained by the Committers and the Apache Hamilton PMC). Use them at your sole discretion. The community does not verify the licenses nor validity of these tools, so it's your responsibility to verify them.
A repository of reusable Apache Hamilton dataflows contributed by the community. Browse and download pre-built dataflows for common use cases.
Note: It's WIP to move the domain to be under Apache. DAGWorks Inc., which donated Hamilton, is not an operating entity anymore.
Articles covering Apache Hamilton use cases, design patterns, reference architectures, and best practices.
Note: It's WIP to move the contents to be under Apache. DAGWorks Inc., which donated Hamilton, is not an operating entity anymore.
Video tutorials, talks, and meetup recordings about Apache Hamilton.
Note: It's WIP to move the contents to be under Apache. DAGWorks Inc., which donated Hamilton, is not an operating entity anymore.
If you‘ve created a plugin or integration for Apache Hamilton, we’d love to include it in our ecosystem!
For Built-in Integrations (maintained by the Apache Hamilton project):
hamilton/plugins/ directoryFor External Resources (maintained by third parties):
:hidden: claude-code-plugin mcp-server