The type: workflow
can be used to create new step types and save them to the Brooklyn type registry. New step types can be used in the same way as the standard step types. This allows common workflow logic to be saved and re-used as steps in other workflow. The usual properties of steps -- inputs, outputs, error handling -- are all supported, and it supports defining parameters and even custom shorthand templates.
This step type can also be used directly within a workflow to run a “nested workflow”, for instance to structure code and to isolate a sequence of steps or to apply on-error
behavior to a group of steps.
Nested and custom workflows are not permitted to access data from their containing workflow; instead, they accept an input
block like other steps.
This type permits all the common step properties and all the workflow settings properties, plus a few others, target
, concurrency
, parameters
, and shorthand
as described below.
When used in a workflow, the nested workflow should be defined as a list in the steps
key. An input
section can also be defined to pass data from the outer workflow to the nested workflow.
For example:
- log This is about to run a nested workflow - let x=1 - ley y=2 - type: workflow input: x: ${x} steps: - log This is a nested workflow, able to see x=${x} from input but not y from the output workflow. on-error: # error handler which runs if the nested workflow fails (i.e. if any step therein fails and does not correct it)
The workflow
type can also be used to run a sequence of steps on multiple targets. If given a target
value, Brooklyn will run the workflow against that target or targets, as follows:
$brooklyn:entity("some-child")
, the nested workflow will run in the scope of that entity. It will be visible in the UI under that entity, and references to sensors and effectors will be aginst that entity.${target}
available in the sub-workflow to refer to the relevant entrychildren
or members
it will run against each entity in the relevant listM..N
for integers M
and N
it will run for all integers in that range, inclusive (so the string 1..4
is equivalent to the list [1,2,3,4]
)The scratch variables target
and target_index
are available referring to to the specific target and its 0-indexed position. These names can be overridden with the target_var_name
and target_index_var_name
keys.
Where a list is supplied, the result of the step is the list collecting the output of each sub-workflow.
If a condition
is supplied when a list is being used, the workflow
step will always run, and the condition
will be applied to entries in the list. An example of this is included below.
The foreach
type is a simplified variant of workflow
when recursing over a list, taking the same.
- step: foreach x in 1..3 steps: - return ${x}
The above loop will return [1,2,3]
.
Each nested workflow runs in its own scope and does not share workflow variables with the parent, apart from values specified as input
, or with other iterations of a loop. Where it is desired to share variables across iterations, the key reducing
can be supplied, giving a map of variable names to be shared and their initial values.
When reducing
, the output of the workflow is this set of variables with their final values.
- step: foreach x in 1..3 reducing: sum: 0 steps: - let sum = ${sum} + ${x}
The above loop will return 6
.
By default nested workflows with list targets run sequentially over the entries, but this can be varied by setting concurrency
. The following values are allowed:
1
being the default, for no concurrency)unlimited
to allow all to run in parallelmin(...)
or max(...)
, where ...
is a comma separated list of valid valuesThis concisely allows complicated -- but important in the real world -- logic such as max(1, min(50%, -10))
to express running concurrently over up to half if more than twenty, otherwise all but 10, and always allowing 1. This might be used for example to upgrade a cluster in situ, leaving the larger of 10 instances or half the cluster alone, if possible.
If the concurrency expression evaluates to 0, or to a negative number whose absolute value is larger than the number of values, the step will fail before executing, to ensure that if e.g. “-10” is specified when there are fewer than 10 items in the target list, the workflow does not run. (Use “max(1, -10)” to allow it to run 1 at a time if there are 10 or fewer.)
Note: Concurrency cannot be specified when reducing
.
This example invokes an effector on all children which are service.isUp
, running in batches of up to 5 but not more than a third of the children at once:
- type: workflow target: children concurrency: max(1, min(33%, 5)) condition: sensor: service.isUp equals: true steps: - invoke-effector effector-on-children
This type can be used to define new step types and add them as new types in the type registry. The definition must specify the steps
, and may in addition specify:
parameters
: a map of parameters accepted by the workflow, with the key the parameter name, and the value map possibly empty or providing optional type
(default string
), defaultValue
(default none), required
(default false
), description
(default none), and/or constraints
shorthand
: a template, as described belowoutput
: an output map or value to be returned by the step, evaluated in the context of the nested workflowWhen this type is used to define a new workflow step, the newly defined step does not allow the steps
or any of the parameters listed above to be overridden. Instead it accepts the parameters defined in the parameters
key of the definition. It also accepts the standard step keys such as input
, timeout
on on-error
. A user of the defined step type can also supply output
which, as per other steps, is evaluated in the context of the outer workflow, with visibility of the output from the current step.
When supplying a workflow in contexts where a workflow
is already expected, such as in a config key that takes a workflow
(a Java CustomWorkflowStep
), it is not necessary to specify the type: workflow
, and additionally, if the only things being set is steps
, those steps can be provided as a list without the steps
keyword. Internally a list will coerce to a workflow
by interpreting the list as the steps.
A custom workflow step can define a shorthand
template which permits a user to use the workflow step as a string rather than a map, even with parameters. The shorthand template syntax consists of a sequence of the following tokens:
${VAR}
- to set VAR, which should be of the regex [A-Za-z0-9_-]+(.[A-Za-z0-9_-]+)*, with dot separation used to set nested maps${VAR...}
- as ${VAR}
but allowing it to match multiple words"LITERAL"
- to expect the user to supply the exact token LITERAL
; this should include spaces if spaces are required[ <TOKENS> ]
- to indicate that a sequence of <TOKENS>
is optional; parsing is attempted first with this block, then without itA simple example to say hello is as follows:
id: greet type: workflow shorthand: ${name...} [ " with " ${greeting} ] parameters: name: required: true greeting: defaultValue: Hello steps: - log ${greeting} ${name}
With this added as a registered type, workflows can write:
- type: greet input: name: Angela
The result will be the same as log Hello Angela
. The shorthand template spec also allows greet Angela
for the same, or exercising the optional block, greet Zachary Jones with Howdy
to log Howdy Zachary Jones
.
This is a trivial single-step example but shows the power of creating custom workflows, especially with parameters and shorthand templates. The examples and the workflow settings include more realistic illustrations of custom workflow steps.
The most common way to define custom workflow types is as workflow, using the primitives defined here, and delegating to custom containers where the logic is best done in a higher-level programming language. This avoids any language bias and the need to learn Brooklyn interfaces. However it is supported to provide custom workflow step types as Java classes in a bundle.
To write a Java workflow step type, provide a class extending WorkflowStepDefinition
, providing implementations for the following methods:
Object doTaskBody(WorkflowStepInstanceExecutionContext context); void populateFromShorthand(String value); boolean isDefaultIdempotent();
The first of these does the work of the step, resolving inputs and accessing context as needed via context
. The second handles providing a custom shorthand, as described above; it can call to a superclass method populateFromShorthandTemplate(TEMPLATE, value)
with the TEMPLATE
for the class, if shorthand is to be supported. Finally, the third returns whether the step is idempotent, that is if the custom step is interrupted, can Brooklyn safely recover simply by rerunning it with the same inputs. As described here, it is recommended to write the step so that it is idempotent if possible.
Once written, the class should be added to the Brooklyn Catalog, e.g. for a custom java step called com.acme.YourJavaWorkflowStep
with shorthand name your-step
, create a catalog.bom
such as the following, and br catalog add catalog.bom
:
brooklyn.catalog: bundle: your-step-bundle version: "1.0.0-SNAPSHOT" items: - id: your-step format: java-type-name itemType: bean item: type: com.acme.YourJavaWorkflowStep