This closes #4091: Merge branch 'master' upto commit 269bf89 into mr-runner
mr-runner: re-enable sdks/python module.
mr-runner: update to 2.3.0-SNAPSHOT.
Check that bigtableWriter is non-null before calling close().
Added a preprocessing step to the Cloud Spanner sink.
Fix Repackaging Configuration in the the DirectRunner
Migrate shared Fn Execution code to Java7
update dataflow.version
[BEAM-3114] Generate text proto config properly in container boot code
[BEAM-3113] Disable stack trace optimization in java container
Fix Go package comment for syscallx
Add all portability protos to Go
[BEAM-2728] Extension for sketch-based statistics : HyperLogLog
Add a runners/java-fn-execution module
Add sdks/java/fn-execution
[BEAM-3135] Adding futures dependency to python SDK
Updates BigQueryTableSource to consider data in streaming buffer when determining estimated size.
Getting AutoValue 1.5.1 working in Beam.
Add License Header to SqlTypeUtils
[BEAM-2203] Implement TIMESTAMPADD
Fix working dir in website precommits
Do not relocate generated Model Classes
Updates Python datastore wordcount example to take a dataset parameter.
Remove obsolete extra parameter
[BEAM-2482] - CodedValueMutationDetector should use the coders structural value
Changed the mutation detector to be based on structural value only
Allows to set a Cloud Spanner host. https://batch-spanner.googleapis.com/ is set as a default host name.
[BEAM-2468] Reading Kinesis records in the background
[BEAM-3054] Uses locale-insensitive number formatting in ESIO and tests
Added VoidSerializer for KafkaIO. Modified KafkaIO.Write.values() to auto add the VoidSerializer for the key.serializer config for kafka producer
[BEAM-1542] SpannerIO: mutation encoding and size estimation improvements
[BEAM-2979] Fix a race condition in getWatermark() in KafkaIO.
[BEAM-3112] Improve error messages in ElasticsearchIO test utils
[BEAM-3111] Upgrade Elasticsearch to 5.6.3 and clean pom
Temporarily disable Dataflow pipeline_url metadata
NonNull by default in sdk/transforms/splittabledofn
NonNull by default in sdk/transforms/join
NonNull by default in sdk/transforms/display
NonNull by default in sdk/transforms/reflect
NonNull by default in sdk/transforms/windowing
NonNull by default in sdk/transforms
Reading spanner schema transform
Suppress AutoValue warnings in TextIO
Remove extraneous type arguments in Latest.java
Remove extraneous type arguments in PAssert
Remove coveralls invocations from all jobs
Adds ParseResult.failure()
Many improvements to TikaIO
Add missing @RunWith to test.
[BEAM-2566] Decouple SDK harness from Dataflow runner by elevating experiments and SDK harness configuration to java-sdk-core.
Add python_requires to setup.py
CR: [BEAM-3005] Set JVM max heap size in java container
Update rat exclusion for python and Go protos
Declare .go and Dockerfile as text in gitattributes
[BEAM-3005] Set JVM max heap size in java container
[BEAM-3005] Add resource limits to provision proto
Stage the pipeline in Python DataflowRunner
Add zip to the list of accepted extra package file types.
Fix remaining nullability errors
Increase Java postcommit timeout to 240
Manage RAT plugin more centrally; only toggle skipping
Fix RAT exclusions
Rearrange .gitignore slightly
Fix typo in seed job
Rename seed job so it is first in glob used by prior seed job
Fix typo in seed jobs
Increase seed job(s) timeout to 60 minutes
Increase seed job(s) timeout to 30 minutes
Make the main seed job standalone
Increase job_beam_PreCommit_Java_MavenInstall timeout from 2.5 to 4 hours.
Implement FnApi side inputs in Python.
Unit test for label pipeline option
NonNull by default in metrics
Ignore findbugs in AutoValue generated classes
NonNull by default for sdk/testing
NonNull by default in sdk/state
NonNull by default in sdk/runners
NonNull by default in sdk/annotations
NonNull by default in sdk/coders
Make Java core SDK root dir NonNull by default
Add dep on Apache-licensed findbugs-annotations implementation
[BEAM-2682] Deletes AvroIOTransformTest
Clone source to a distinguished subdirectory of Jenkins workspace
Adding lull tracking for python sampler
Avoids generating proto files for Windows if grpcio-tools is not installed.
Do not crash when RawPTransform has null spec
Unit test to repro NPE in PTransformTranslation
Created Java snippets file
Add standalone version of seed job
Pin runner harness also for official BEAM releases.
Remove duplicate mocking in DataflowRunnerTest
Add assertion that valid jobs must have staged pipeline
Stage the pipeline without using a temp file
Stage the portable pipeline in Dataflow
Add ability to stage explicit file list
Improve GcsFileSystem errors messages slightly
[BEAM-2720] Update kafka client version to 0.11.0.1
Update PipelineTest.testReplacedNames
DirectRunner: Replace use of RawPTransform with NotSerializable.forUrn translators
Wordcount on fnapi pipeline and IT test.
Clearer getOrDefault style in RehydratedComponents
...