Beam SQL's SET
and RESET
statements allow the user to [configure Pipeline Options]({{ site.baseurl }}/documentation/programming-guide/#configuring-pipeline-options) via the SQL shell. These are the same Pipeline Options passed to other Beam applications on the command line in the --<option>=<value>
format.
SET option = value
The SET command sets a Pipeline Option.
option
: The case sensitive name of the Pipeline Option, specified as an [Identifier]({{ site.baseurl }}/documentation/dsls/sql/lexical/#identifiers).value
: The case sensitive value of the Pipeline Option, specified as an [Identifier]({{ site.baseurl }}/documentation/dsls/sql/lexical/#identifiers). For flag options that have no value on the command line, use true
.RESET option
The RESET command resets a Pipeline Option to its default value.
option
: The case sensitive name of the Pipeline Option, specified as an [Identifier]({{ site.baseurl }}/documentation/dsls/sql/lexical/#identifiers).SET project = `my_gcp_project`
: Sets the default GCP project tomy_gcp_project
.SET runner = DataflowRunner
: Sets the pipeline to run on Dataflow.