| //// |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| http://www.apache.org/licenses/LICENSE-2.0 |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| //// |
| :documentationPath: /pipeline/transforms/ |
| :language: en_US |
| :description: The Workflow Executor transform execute a Hop workflow from within a pipeline. |
| |
| = image:transforms/icons/workflow.svg[Workflow Executor transform Icon, role="image-doc-icon"] Workflow Executor |
| |
| [%noheader,cols="3a,1a", role="table-no-borders" ] |
| |=== |
| | |
| == Description |
| |
| The Workflow Executor transform execute a Hop workflow from within a pipeline. |
| |
| | |
| == Supported Engines |
| [%noheader,cols="2,1a",frame=none, role="table-supported-engines"] |
| !=== |
| !Hop Engine! image:check_mark.svg[Supported, 24] |
| !Spark! image:question_mark.svg[Maybe Supported, 24] |
| !Flink! image:question_mark.svg[Maybe Supported, 24] |
| !Dataflow! image:question_mark.svg[Maybe Supported, 24] |
| !=== |
| |=== |
| |
| == Usage |
| |
| By default, the specified workflow will be executed once for each input row. |
| |
| This row can be used to set parameters and variables and it is passed to the workflow in the form of a result row. |
| |
| You can also allow a group of records to be passed based on the value in a field (when the value changes the workflow is executed) or on time. |
| |
| In these cases, the first row of the group or rows is used to set parameters or variables in the workflow. |
| |
| It is possible to launch multiple copies of this transform to facilitate parallel workflow processing. |
| |
| See also: |
| |
| * The xref:workflow/actions/workflow.adoc[Workflow action] that executes a sub-workflow from a workflow. |
| * The xref:workflow/actions/pipeline.adoc[Pipeline action] that executes a pipeline from a workflow. |
| * The xref:pipeline/transforms/pipelineexecutor.adoc[Pipeline Executor transform] that executes a sub-pipeline from a pipeline. |
| |
| == Options |
| |
| === General |
| |
| [options="header"] |
| |=== |
| |Option|Description |
| |Transform name|Name of the transform. |
| |Workflow|Use this option to specify a workflow stored in a file (.hwf file) |
| |=== |
| |
| === Parameters Tab |
| |
| In this tab you can specify which field to use to set a certain parameter or variable value. |
| If you specify an input field to use, the static input value is not used. |
| If multiple rows are passed to the workflow, the first row is taken to set the parameters or variables. |
| |
| There is a button in the lower right corner of the tab that will insert all the defined parameters of the specified workflow. |
| For information the description of the parameter is inserted into the static input value field. |
| |
| If you enable the "Inherit all variables from the pipeline" option, all the variables defined in the parent pipeline are passed to the workflow. |
| |
| === Row Grouping Tab |
| |
| On this tab you can specify the amount of input rows that are passed to the workflow in the form of result rows. |
| You can use the result rows in a workflow or Pipeline workflow action to loop over or you can get the records themselves in a Get rows from result transform in a pipeline. |
| |
| - The number of rows to send to the workflow: after every X rows the workflow will be executed and these X rows will be passed to the workflow. |
| - Field to group rows on: Rows will be accumulated in a group as long as the field value stays the same. |
| If the value changes the workflow will be executed and the accumulated rows will be passed to the workflow. |
| - The time to wait collecting rows before execution: This is time the transform will spend accumulating rows prior to the execution of the workflow. |
| |
| Please note that you can only specify one method of grouping. |
| |
| === Execution Results Tab |
| |
| You can specify result fields and to which transform to send them. |
| If you don't need a certain result simply leave a blank input field. |
| |
| === Result Rows Tab |
| |
| In the "Result rows" tab you can specify the layout of the expected result rows of this workflow and to which transform to send them after execution. |
| |
| Please note that this transform will verify that the data type of the result row fields are identical to what is specified. |
| If there is a difference an error will be thrown. |
| |
| === Result Files Tab |
| |
| Here you can specify where to send the result files from the workflow execution. |