title: “Apache Beam 2.44.0” date: 2023-01-17 09:00:00 -0700 categories:

  • blog
  • release authors:
  • klk

We are happy to present the new 2.44.0 release of Beam. This release includes both improvements and new functionality. See the download page for this release.

For more information on changes in 2.44.0, check out the detailed release notes.

I/Os

  • Support for Bigtable sink (Write and WriteBatch) added (Go) (#23324).
  • S3 implementation of the Beam filesystem (Go) (#23991).
  • Support for SingleStoreDB source and sink added (Java) (#22617).
  • Added support for DefaultAzureCredential authentication in Azure Filesystem (Python) (#24210).
  • Added new CdapIO for CDAP Batch and Streaming Source/Sinks (Java) (#24961).
  • Added new SparkReceiverIO for Spark Receivers 2.4.* (Java) (#24960).

New Features / Improvements

  • Beam now provides a portable “runner” that can render pipeline graphs with graphviz. See python -m apache_beam.runners.render --help for more details.
  • Local packages can now be used as dependencies in the requirements.txt file, rather than requiring them to be passed separately via the --extra_package option (Python) (#23684).
  • Pipeline Resource Hints now supported via --resource_hints flag (Go) (#23990).
  • Make Python SDK containers reusable on portable runners by installing dependencies to temporary venvs (BEAM-12792).
  • RunInference model handlers now support the specification of a custom inference function in Python (#22572)
  • Support for map_windows urn added to Go SDK (#24307).

Breaking Changes

  • ParquetIO.withSplit was removed since splittable reading has been the default behavior since 2.35.0. The effect of this change is to drop support for non-splittable reading (Java)(#23832).
  • beam-sdks-java-extensions-google-cloud-platform-core is no longer a dependency of the Java SDK Harness. Some users of a portable runner (such as Dataflow Runner v2) may have an undeclared dependency on this package (for example using GCS with TextIO) and will now need to declare the dependency.
  • beam-sdks-java-core is no longer a dependency of the Java SDK Harness. Users of a portable runner (such as Dataflow Runner v2) will need to provide this package and its dependencies.
  • Slices now use the Beam Iterable Coder. This enables cross language use, but breaks pipeline updates if a Slice type is used as a PCollection element or State API element. (Go)#24339

Bugfixes

  • Fixed JmsIO acknowledgment issue (Java) (#20814)
  • Fixed Beam SQL CalciteUtils (Java) and Cross-language JdbcIO (Python) did not support JDBC CHAR/VARCHAR, BINARY/VARBINARY logical types (#23747, #23526).
  • Ensure iterated and emitted types are used with the generic register package are registered with the type and schema registries.(Go) (#23889)

List of Contributors

According to git shortlog, the following people contributed to the 2.44.0 release. Thank you to all contributors!

Ahmed Abualsaud

Ahmet Altay

Alex Merose

Alexey Inkin

Alexey Romanenko

Anand Inguva

Andrei Gurau

Andrej Galad

Andrew Pilloud

Ayush Sharma

Benjamin Gonzalez

Bjorn Pedersen

Brian Hulette

Bruno Volpato

Bulat Safiullin

Chamikara Jayalath

Chris Gavin

Damon Douglas

Danielle Syse

Danny McCormick

Darkhan Nausharipov

David Cavazos

Dmitry Repin

Doug Judd

Elias Segundo Antonio

Evan Galpin

Evgeny Antyshev

Heejong Lee

Henrik Heggelund-Berg

Israel Herraiz

Jack McCluskey

Jan Lukavský

Janek Bevendorff

Johanna Öjeling

John J. Casey

Jozef Vilcek

Kanishk Karanawat

Kenneth Knowles

Kiley Sok

Laksh

Liam Miller-Cushon

Luke Cwik

MakarkinSAkvelon

Minbo Bae

Moritz Mack

Nancy Xu

Ning Kang

Nivaldo Tokuda

Oleh Borysevych

Pablo Estrada

Philippe Moussalli

Pranav Bhandari

Rebecca Szper

Reuven Lax

Rick Smit

Ritesh Ghorse

Robert Bradshaw

Robert Burke

Ryan Thompson

Sam Whittle

Sanil Jain

Scott Strong

Shubham Krishna

Steven van Rossum

Svetak Sundhar

Thiago Nunes

Tianyang Hu

Trevor Gevers

Valentyn Tymofieiev

Vitaly Terentyev

Vladislav Chunikhin

Xinyu Liu

Yi Hu

Yichi Zhang

AdalbertMemSQL

agvdndor

andremissaglia

arne-alex

bullet03

camphillips22

capthiron

creste

fab-jul

illoise

kn1kn1

nancyxu123

peridotml

shinannegans

smeet07