blob: 8b96cc9d1b7c61d71479b638b21ca0642023c088 [file] [log] [blame]
========================
Using Data Adapters
========================
This is an index of all the available data adapters, both savers and loaders.
Note that some savers and loaders are the same (certain classes can handle both),
but some are different. You will want to reference this when calling out to any of the following:
1. Using :doc:`/reference/decorators/save_to/`.
2. Using :doc:`/reference/decorators/load_from/`.
3. Using :doc:`materializers </reference/drivers/Driver/>`
To read these tables, you want to first look at the key to determine which format you want --
these should be human-readable and familiar to you. Then you'll want to look at the `types` field
to figure out which is the best for your case (the object you want to load from or save to).
Finally, look up the adapter params to see what parameters you can pass to the data adapters.
The optional params come with their default value specified.
If you want more information, click on the `module`, it will send you to the code that implements
it to see how the parameters are used.
As an example, say we wanted to save a pandas dataframe to a CSV file. We would first find the
key `csv`, which would inform us that we want to call `save_to.csv` (or `to.csv` in the case
of `materialize`). Then, we would look at the `types` field, finding that there is a pandas
dataframe adapter. Finally, we would look at the `params` field, finding that we can pass
`path`, and (optionally) `sep` (which we'd realize defaults to `,` when looking at the code).
All together, we'd end up with:
.. code-block:: python
import pandas as pd
from hamilton.function_modifiers import value, save_to
@save_to.csv(path=value("my_file.csv"))
def my_data(...) -> pd.DataFrame:
...
And we're good to go!
If you want to extend these, see :doc:`/reference/io/available-data-adapters` for documentation,
and `the example <https://github.com/DAGWorks-Inc/hamilton/blob/main/examples/materialization/README.md>`_
in the repository for an example of how to do so.
=============
Data Loaders
=============
.. data_adapter_table:: loader
=============
Data Savers
=============
.. data_adapter_table:: saver