tree: 863c795e8d6a9f890f48c1352facf01d9b70528a [path history] [tgz]
  1. all_functions.png
  2. dataflow.png
  3. dataflow.py
  4. mock_api.py
  5. notebook.ipynb
  6. README.md
  7. run.py
examples/dagster/hamilton_code/README.md

Hamilton

Hamilton dataflow materialization plan

File structure

The Hamilton refactor achieves the same data transformations. The project fits in essentially 2 files:

  • dataflow.py contains regular Python functions to define the dataflow (equivalent to Dagster tutorial/assets.py).
  • run.py contains the “driver code” to load and execute the dataflow (equivalent to Dagster tutorial/__init__.py).
  • notebook.ipynb contains code to execute the dataflow (similar code to run.py), but allows for an interactive exploration of results.

The directory also include mock_api.py which simulates a ressource in the Dagster tutorial, and dataflow.png which is an artifact generated by the Hamilton Driver.

Instructions

  1. Run the code
    python run.py
    

Going further