apache_beam.io.gcp.datastore.v1
has been removed as the client it uses is out of date and does not support Python 3 (BEAM-9529). Please migrate your code to use apache_beam.io.gcp.datastore.v1new. See the updated datastore_wordcount for example usage.Python SDK will now use Python 3 type annotations as pipeline type hints. (#10717)
If you suspect that this feature is causing your pipeline to fail, calling apache_beam.typehints.disable_type_annotations()
before pipeline creation will disable is completely, and decorating specific functions (such as process()
) with @apache_beam.typehints.no_annotations
will disable it for that function.
More details will be in Ensuring Python Type Safety and an upcoming blog post.
Java SDK: Introducing the concept of options in Beam Schema’s. These options add extra context to fields and schemas. This replaces the current Beam metadata that is present in a FieldType only, options are available in fields and row schemas. Schema options are fully typed and can contain complex rows. Remark: Schema aware is still experimental. (BEAM-9035)
Java SDK: The protobuf extension is fully schema aware and also includes protobuf option conversion to beam schema options. Remark: Schema aware is still experimental. (BEAM-9044)
--region
option to be set, unless a default value is set in the environment (BEAM-9199). See here for more details.2.23.0
. (BEAM-9704)SpannerConfig.connectToSpanner
has been moved to SpannerAccessor.create
. (BEAM-9310).force_generated_pcollection_output_ids
experiment.