tree 6b4735cdf171c3cb5134d135e70b0aed1b82199c
parent 5fa0b14d20fab007d9e2d954eb4a34155a6f199f
parent 202a2cb242df8ce8b11ff19c7d8307991d61af3a
author Kenneth Knowles <kenn@apache.org> 1510073611 -0800
committer Kenneth Knowles <kenn@apache.org> 1510073611 -0800

This closes #4091: Merge branch 'master' upto commit 269bf89 into mr-runner

  mr-runner: re-enable sdks/python module.
  mr-runner: update to 2.3.0-SNAPSHOT.
  Check that bigtableWriter is non-null before calling close().
  Added a preprocessing step to the Cloud Spanner sink.
  Fix Repackaging Configuration in the the DirectRunner
  Migrate shared Fn Execution code to Java7
  update dataflow.version
  [BEAM-3114] Generate text proto config properly in container boot code
  [BEAM-3113] Disable stack trace optimization in java container
  Fix Go package comment for syscallx
  Add all portability protos to Go
  [BEAM-2728] Extension for sketch-based statistics : HyperLogLog
  Add a runners/java-fn-execution module
  Add sdks/java/fn-execution
  [BEAM-3135] Adding futures dependency to python SDK
  Updates BigQueryTableSource to consider data in streaming buffer when determining estimated size.
  Getting AutoValue 1.5.1 working in Beam.
  Add License Header to SqlTypeUtils
  [BEAM-2203] Implement TIMESTAMPADD
  Fix working dir in website precommits
  Do not relocate generated Model Classes
  Updates Python datastore wordcount example to take a dataset parameter.
  Remove obsolete extra parameter
  [BEAM-2482] - CodedValueMutationDetector should use the coders structural value
  Changed the mutation detector to be based on structural value only
  Allows to set a Cloud Spanner host. https://batch-spanner.googleapis.com/ is set as a default host name.
  [BEAM-2468] Reading Kinesis records in the background
  [BEAM-3054] Uses locale-insensitive number formatting in ESIO and tests
  Added VoidSerializer for KafkaIO. Modified KafkaIO.Write.values() to auto add the VoidSerializer for the key.serializer config for kafka producer
  [BEAM-1542] SpannerIO: mutation encoding and size estimation improvements
  [BEAM-2979] Fix a race condition in getWatermark() in KafkaIO.
  [BEAM-3112] Improve error messages in ElasticsearchIO test utils
  [BEAM-3111] Upgrade Elasticsearch to 5.6.3 and clean pom
  Temporarily disable Dataflow pipeline_url metadata
  NonNull by default in sdk/transforms/splittabledofn
  NonNull by default in sdk/transforms/join
  NonNull by default in sdk/transforms/display
  NonNull by default in sdk/transforms/reflect
  NonNull by default in sdk/transforms/windowing
  NonNull by default in sdk/transforms
  Reading spanner schema transform
  Suppress AutoValue warnings in TextIO
  Remove extraneous type arguments in Latest.java
  Remove extraneous type arguments in PAssert
  Remove coveralls invocations from all jobs
  Adds ParseResult.failure()
  Many improvements to TikaIO
  Add missing @RunWith to test.
  [BEAM-2566] Decouple SDK harness from Dataflow runner by elevating experiments and SDK harness configuration to java-sdk-core.
  Add python_requires to setup.py
  CR: [BEAM-3005] Set JVM max heap size in java container
  Update rat exclusion for python and Go protos
  Declare .go and Dockerfile as text in gitattributes
  [BEAM-3005] Set JVM max heap size in java container
  [BEAM-3005] Add resource limits to provision proto
  Stage the pipeline in Python DataflowRunner
  Add zip to the list of accepted extra package file types.
  Fix remaining nullability errors
  Increase Java postcommit timeout to 240
  Manage RAT plugin more centrally; only toggle skipping
  Fix RAT exclusions
  Rearrange .gitignore slightly
  Fix typo in seed job
  Rename seed job so it is first in glob used by prior seed job
  Fix typo in seed jobs
  Increase seed job(s) timeout to 60 minutes
  Increase seed job(s) timeout to 30 minutes
  Make the main seed job standalone
  Increase job_beam_PreCommit_Java_MavenInstall timeout from 2.5 to 4 hours.
  Implement FnApi side inputs in Python.
  Unit test for label pipeline option
  NonNull by default in metrics
  Ignore findbugs in AutoValue generated classes
  NonNull by default for sdk/testing
  NonNull by default in sdk/state
  NonNull by default in sdk/runners
  NonNull by default in sdk/annotations
  NonNull by default in sdk/coders
  Make Java core SDK root dir NonNull by default
  Add dep on Apache-licensed findbugs-annotations implementation
  [BEAM-2682] Deletes AvroIOTransformTest
  Clone source to a distinguished subdirectory of Jenkins workspace
  Adding lull tracking for python sampler
  Avoids generating proto files for Windows if grpcio-tools is not installed.
  Do not crash when RawPTransform has null spec
  Unit test to repro NPE in PTransformTranslation
  Created Java snippets file
  Add standalone version of seed job
  Pin runner harness also for official BEAM releases.
  Remove duplicate mocking in DataflowRunnerTest
  Add assertion that valid jobs must have staged pipeline
  Stage the pipeline without using a temp file
  Stage the portable pipeline in Dataflow
  Add ability to stage explicit file list
  Improve GcsFileSystem errors messages slightly
  [BEAM-2720] Update kafka client version to 0.11.0.1
  Update PipelineTest.testReplacedNames
  DirectRunner: Replace use of RawPTransform with NotSerializable.forUrn translators
  Wordcount on fnapi pipeline and IT test.
  Clearer getOrDefault style in RehydratedComponents
  ...