Apache Hamilton dataflow
The Apache Hamilton refactor is composed of a few files:
data_processing.py and data_science.py contains regular Python functions to define the Apache Hamilton dataflow. This is equivalent to Kedro's pipeline.py and nodes.py files.run.py contains the “driver code” to load and execute the dataflow. There's no direct equivalent in the Kedro tutorial since it prefers using the CLI for execution.notebooks/interactive.ipynb contains the “driver code”, similar to run.py, but uses Apache Hamilton Jupyter Magics to define the dataflow interactively in a notebook.tests/test_dataflow.py includes tests equivalent to tests/pipelines/data_science/test_pipeline.py in the Kedro code.Create a virtual environment and activate it
python -m venv venv && . venv/bin/active
Install requirements for the Apache Hamilton code
pip install -r requirements.txt
Install the current hamilton-code project
pip install -e .
Run the dataflow
python run.py
Run the tests
pytest tests/
Concepts/ documentation section