tree: ceb485248590894bd267e8b2d22a1432944e06f1 [search]
  1. all_functions.png
  2. dataflow.png
  3. dataflow.py
  4. mock_api.py
  5. notebook.ipynb
  6. README.md
  7. run.py
examples/dagster/hamilton_code/README.md

Apache Hamilton

Apache Hamilton dataflow materialization plan

File structure

The Apache Hamilton refactor achieves the same data transformations. The project fits in essentially 2 files:

  • dataflow.py contains regular Python functions to define the dataflow (equivalent to Dagster tutorial/assets.py).
  • run.py contains the “driver code” to load and execute the dataflow (equivalent to Dagster tutorial/__init__.py).
  • notebook.ipynb contains code to execute the dataflow (similar code to run.py), but allows for an interactive exploration of results.

The directory also include mock_api.py which simulates a resource in the Dagster tutorial, and dataflow.png which is an artifact generated by the Apache Hamilton Driver.

Instructions

  1. Run the code
    python run.py
    

Going further