A suite of YAML pipeline examples is currently located under the directory sdks/python/apache_beam/yaml/examples.
These examples leverage the built-in Combine
transform for performing simple aggregations including sum, mean, count, etc.
These examples leverage DF or other existing templates and convert them to yaml blueprints.
These examples leverage the built-in mapping transforms including MapToFields
, Filter
and Explode
.
These examples leverage the built-in IO transforms to read from and write to various sources and sinks, including Iceberg, Kafka and Spanner.
These examples use Jinja templatization to build off of different contexts and/or with different configurations.
These examples include built-in ML-specific transforms such as RunInference
, MLTransform
and Enrichment
.