Google Git
Sign in
apache / beam / master / . / examples / notebooks / beam-ml
tree: 8e6a344424e46071accc38202dc14c1f25fb4697 [path history] [tgz]
  1. anomaly_detection/
  2. data_preprocessing/
  3. rag_usecase/
  4. alloydb_product_catalog_embeddings.ipynb
  5. automatic_model_refresh.ipynb
  6. bigquery_enrichment_transform.ipynb
  7. bigquery_vector_ingestion_and_search.ipynb
  8. bigtable_enrichment_transform.ipynb
  9. cloudsql_mysql_product_catalog_embeddings.ipynb
  10. cloudsql_postgres_product_catalog_embeddings.ipynb
  11. custom_remote_inference.ipynb
  12. dataflow_tpu_examples.ipynb
  13. dataframe_api_preprocessing.ipynb
  14. gemma_2_sentiment_and_summarization.ipynb
  15. image_processing_tensorflow.ipynb
  16. mltransform_basic.ipynb
  17. nlp_tensorflow_streaming.ipynb
  18. per_key_models.ipynb
  19. README.md
  20. run_custom_inference.ipynb
  21. run_inference_gemma.ipynb
  22. run_inference_generative_ai.ipynb
  23. run_inference_huggingface.ipynb
  24. run_inference_multi_model.ipynb
  25. run_inference_pytorch.ipynb
  26. run_inference_pytorch_tensorflow_sklearn.ipynb
  27. run_inference_sklearn.ipynb
  28. run_inference_tensorflow.ipynb
  29. run_inference_tensorflow_with_tfx.ipynb
  30. run_inference_vertex_ai.ipynb
  31. run_inference_vllm.ipynb
  32. run_inference_windowing.ipynb
  33. run_inference_with_tensorflow_hub.ipynb
  34. run_inference_xgboost.ipynb
  35. speech_emotion_tensorflow.ipynb
  36. tfma_beam.ipynb
  37. vertex_ai_feature_store_enrichment.ipynb
examples/notebooks/beam-ml/README.md

ML sample notebooks

Starting with the Apache Beam SDK version 2.40, users have access to a RunInference transform.

This transform allows you to make predictions and inference on data with machine learning (ML) models. The model handler abstracts the user from the configuration needed for specific frameworks, such as Tensorflow, PyTorch, and others. For a full list of supported frameworks, see the About Beam ML page.

Use the notebooks

These notebooks illustrate ways to use Apache Beam's RunInference transforms, as well as different use cases for ModelHandler implementations. Beam comes with multiple ModelHandler implementations.

Load the notebooks

  1. To get started quickly with notebooks, use Colab.
  2. In Colab, open the notebook from GitHub using the notebook URL, for example:
https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_tensorflow.ipynb
  1. To run most notebooks, you need to change the Google Cloud project and bucket to your project and bucket.

Notebooks

This section contains the following example notebooks.

Data processing

Data enrichment

Prediction and inference with pretrained models

Custom inference

Machine Learning Use Cases

Automatic Model Refresh

Multi-model pipelines

Model Evaluation