Google Git
Sign in
apache / nifi / a3f54fa578f86220ff63e73f526e510de3485f41 / . / nifi-docs / src / main / asciidoc / expression-language-guide.adoc
blob: d09fa875d22ce916c2aec3cd5aa65be4a6776726 [file] [log] [blame]
//
// Licensed to the Apache Software Foundation (ASF) under one or more
// contributor license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright ownership.
// The ASF licenses this file to You under the Apache License, Version 2.0
// (the "License"); you may not use this file except in compliance with
// the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
= Apache NiFi Expression Language Guide
Apache NiFi Team <dev@nifi.apache.org>
:homepage: http://nifi.apache.org
:linkattrs:
[[overview]]
== Overview
All data in Apache NiFi is represented by an abstraction called a FlowFile.
A FlowFile is comprised of two major pieces: content and attributes.
The content portion of the FlowFile represents the data on which to operate.
For instance, if a file is picked up from a local file system using the
GetFile Processor, the contents of the file will become the contents of the
FlowFile.
The attributes portion of the FlowFile represents information about the data
itself, or metadata. Attributes are key-value pairs that represent what is
known about the data as well as information that is useful for routing and
processing the data appropriately.
Keeping with the example of a file that is picked up from
a local file system, the FlowFile would have an attribute called `filename` that
reflected the name of the file on the file system. Additionally, the FlowFile will
have a `path` attribute that reflects the directory on the file system that this
file lived in. The FlowFile will also have an attribute named `uuid`, which is a
unique identifier for this FlowFile. For complete listing of the core attributes
check out the FlowFile section of the link:developer-guide.html#flowfile[Developer's Guide].
However, placing these attributes on a FlowFile do not provide much benefit
if the user is unable to make use of them. The NiFi Expression Language provides
the ability to reference these attributes, compare them to other values,
and manipulate their values.
[[structure]]
== Structure of a NiFi Expression
The NiFi Expression Language always begins with the start delimiter `${` and ends
with the end delimiter `}`. Between the start and end delimiters is the text of the
Expression itself. In its most basic form, the Expression can consist of just an
attribute name. For example, `${filename}` will return the value of the `filename`
attribute.
In a slightly more complex example, we can instead return a manipulation of this value.
We can, for example, return an all upper-case version of the filename by calling the
`toUpper` function: `${filename:toUpper()}`. In this case, we reference the `filename`
attribute and then manipulate this value by using the `toUpper` function. A function call
consists of 5 elements. First, there is a function call delimiter `:`. Second is the name
of the function - in this case, `toUpper`. Next is an open parenthesis (`(`), followed
by the function arguments. The arguments necessary are dependent upon which function
is being called. In this example, we are using the `toUpper` function, which does not
have any arguments, so this element is omitted. Finally, the closing parenthesis (`)`)
indicates the end of the function call. There are many different functions that are supported
by the Expression Language to achieve many different goals. Some functions provide String (text)
manipulation, such as the `toUpper` function. Others, such as the `equals` and `matches` functions,
provide comparison functionality. Functions also exist for manipulating dates and times and
for performing mathematical operations. Each of these functions is described below, in the
<<functions>> section, with an explanation of what the function does, the arguments that it
requires, and the type of information that it returns.
When we perform a function call on an attribute, as above, we refer to the attribute as the
_subject_ of the function, as the attribute is the entity on which the function is operating.
We can then chain together multiple function calls, where the return value of the first function
becomes the subject of the second function and its return value becomes the subject of the third
function and so on. Continuing with our example, we can chain together multiple functions by using
the expression `${filename:toUpper():equals('HELLO.TXT')}`. There is no limit to the number of
functions that can be chained together.
Any FlowFile attribute can be referenced using the Expression Language. However, if the attribute
name contains a "special character", the attribute name must be escaped by quoting it. The following
characters are each considered "special characters":
- $ (dollar sign)
- | (pipe)
- { (open brace)
- } (close brace)
- ( (open parenthesis)
- ) (close parenthesis)
- [ (open bracket)
- ] (close bracket)
- , (comma)
- : (colon)
- ; (semicolon)
- / (forward slash)
- * (asterisk)
- ' (single quote)
- (space)
- \t (tab)
- \r (carriage return)
- \n (new-line)
Additionally, a number is considered a "special character" if it is the first character of the attribute name.
If any of these special characters is present in an attribute is quoted by using either single or double quotes.
The Expression Language allows single quotes and double quotes to be used interchangeably. For example, the following
can be used to escape an attribute named `my attribute`: `${"my attribute"}` or `${'my attribute'}`.
In this example, the value to be returned is the value of the "my attribute" attribute, if it exists. The Expression
Language will search through a hierarchy for a matching property. See <<expression-language-hierarchy>>
for a description of the hierarchy.
There also exist some functions that expect to have no subject. These functions are invoked simply
by calling the function at the beginning of the Expression, such as `${hostname()}`. These functions
can then be changed together, as well. For example, `${hostname():toUpper()}`. Attempting to
evaluate the function with subject will result in an error. In the <<functions>>
section below, these functions will clearly indicate in their descriptions that they do not
require a subject.
Often times, we will need to compare the values of two different attributes to each other.
We are able to accomplish this by using embedded Expressions. We can, for example, check if
the `filename` attribute is the same as the `uuid` attribute: `${filename:equals( ${uuid} )}`.
Notice here, also, that we have a space between the opening parenthesis for the `equals` method and
the embedded Expression. This is not necessary and does not affect how the Expression is evaluated
in any way. Rather, it is intended to make the Expression easier to read. White space is ignored by
the Expression Language between delimiters. Therefore, we can use the Expression
`${ filename : equals(${ uuid}) }` or `${filename:equals(${uuid})}` and both Expressions
mean the same thing. We cannot, however, use `${file name:equals(${uuid})}`, because this results
in `file` and `name` being interpreted as different tokens, rather than a single token, `filename`.
[[expression-language-hierarchy]]
=== Expression Language Hierarchy
When using Expression Language to reference a property by name there is a defined hierarchy within which NiFi
will search for the value.
The current hierarchy in NiFi is as follows:
1. Search FlowFile for attribute/key
2. Search Process Group Variables for attribute/key
3. Search File Registry file for attribute/key
4. Search NiFi JVM Properties for attribute/key
5. Search System Environment Variables for attribute/key
NiFi will search for, and return, the first occurrence of a matching property. If no matching property is found, `null` is returned.
[[usage]]
== Expression Language in the Application
The Expression Language is used heavily throughout the NiFi application for configuring Processor
properties. Not all Processor properties support the Expression Language, however. Whether or not
a Property supports the Expression Language is determined by the developer of the Processor when
the Processor is written. However, the application strives to clearly illustrate for each Property
whether or not the Expression Language is supported.
In the application, when configuring a component property, the User Interface provides an Information
icon (
image:iconInfo.png["Info"]
) next to the name of the Property. Hovering over this icon with the mouse will provide a tooltip that
provides helpful information about the Property. This information includes a description of the Property,
the default value (if any), historically configured values (if any), and the evaluation scope of this
property for expression language. There are three values and the evaluation scope of the expression
language is hierarchical: NONE -> VARIABLE_REGISTRY -> FLOWFILE_ATTRIBUTES.
* NONE - expression language is not supported for this property
* VARIABLE_REGISTRY is hierarchically constructed as below:
** Variables defined at process group level and then, recursively, up to the higher process group until
the root process group.
** Variables defined in custom properties files through the nifi.variable.registry.properties property
in nifi.properties file.
** Environment variables defined at JVM level and system properties.
* FLOWFILE_ATTRIBUTES - will use attributes of each individual flow file, as well as those variables defined
by the Variable Registry, as described above.
[[escaping]]
=== Escaping Expression Language
:extra-dollar-sign: Hello $${UserName}
:literal-value: Hello $$User$$Name
:four-dollar-signs: $$$${abc}
:five-dollar-signs: $$$$${abc}
There may be times when a property supports Expression Language, but the user wishes to use a literal value
that follows the same syntax as the Expression Language. For example, a user may want to configure a property
value to be the literal text `Hello ${UserName}`. In such a case, this can be accomplished by using an extra
`$` (dollar sign symbol) just before the expression to escape it (i.e., `{extra-dollar-sign}`). Unless the `$`
character is being used to escape an Expression, it should not be escaped. For example, the value `{literal-value}`
should not escape the `$` characters, so the literal value that will be used is `{literal-value}`.
If more than two `$` characters are encountered sequentially before a `{`, then each pair of `$` characters will
be considered an escaping of the `$` character. The escaping will be performed from left-to-right.
To help illustrate this, consider that the variable `abc` contains the value `xyz`. Then, consider the following
table of Expressions and their corresponding evaluated values:
.Escaping EL Examples
|========================================================================================
| Expression | Value | Notes
| `${abc}` | `xyz` |
| `$${abc}` | `${abc}` |
| `$$${abc}` | `$xyz` |
| `{four-dollar-signs}` | `$${abc}` |
| `{five-dollar-signs}` | `$$xyz` |
| `I owe you $5` | `I owe you $5` | No actual Expression is present here.
| `You owe me $$5 too` | `You owe me $$5 too` | The $ character is not escaped because it does not immediately precede an Expression.
| `Unescaped $$${5 because no closing brace` | `Unescaped $$${5 because no closing brace` | Because there is no closing brace here, there is no actual Expression and hence the $ characters are not
escaped.
| `Unescaped $$${5} because no closing brace` | <Error> | This expression is not valid because it equates to an escaped $, followed by `${5}` and the `${5}` is not a valid Expression. The number
must be escaped.
| `Unescaped $$${'5'} because no closing brace` | `Unescaped $ because no closing brace` | There is no attribute named `5` so the Expression evaluates to an empty string. The `$$` evaluates to a
single (escaped) `$` because it immediately precedes an Expression.
|========================================================================================
[[editor]]
=== Expression Language Editor
When configuring the value of a Processor property, the NiFi User Interface provides help with the
Expression Language using the Expression Language editor. Once an Expression is begin by typing `${`,
the editor begins to highlight parentheses and braces so that the user is easily able to tell which
opening parenthesis or brace matches which closing parenthesis or brace.
The editor also supplies context-sensitive help by providing a list of all functions that can be used
at the current cursor position. To activate this feature, press Ctrl+Space on the keyboard. The user
is also able to type part of a function name and then press Ctrl+Space to see all functions that can
be used that start with the same prefix. For example, if we type into the editor `${filename:to`
and then press Ctrl+Space, we are provided a pop-up that lists six different functions: `toDate`,
`toLower`, `toNumber`, `toRadix`, `toString`, and `toUpper`. We can then continue typing to narrow
which functions are shown, or we can select one of the functions from the list by double-clicking
it with the mouse or using the arrow keys to highlight the desired function and pressing Enter.
[[functions]]
== Functions
Functions provide a convenient way to manipulate and compare values of attributes. The Expression Language
provides many different functions to meet the needs of a automated dataflow. Each function takes
zero or more arguments and returns a single value. These functions can then be chained together to create
powerful Expressions to evaluate conditions and manipulate values. See <<structure>> for more information
on how to call and chain functions together.
[[types]]
=== Data Types
Each argument to a function and each value returned from a function has a specific data type. The Expression
Language supports four different data types:
- *String*: A String is a sequence of characters that can consist of numbers, letters, white space, and
special characters.
- *Number*: A Number is an whole number comprised of one or more digits (`0` through `9`). When converting to numbers from Date data types, they are represented as
the number of milliseconds since midnight GMT on January 1, 1970.
- *Decimal*: A Decimal is a numeric value that can support decimals and larger values with minimal loss of precision. More precisely it
is a double-precision 64-bit IEEE 754 floating point. Due to this minimal loss of precision this data type should not be used for
very precise values, such as currency. For more documentation on the range of values stored in this data type
refer to this link:https://docs.oracle.com/javase/specs/jls/se7/html/jls-4.html#jls-4.2.3[link^]. The following are some examples of the forms of
literal decimals that are supported in expression language (the "E" can also be lower-case):
* 1.1
* .1E1
* 1.11E-12
- *Date*: A Date is an object that holds a Date and Time. Utilizing the <<dates>> and <<type_cast>> functions this data
type can be converted to/from Strings and numbers. If the whole Expression Language expression is evaluated to be a
date then it will be converted to a String with the format: "<Day of Week> <Month> <Day of Month> <Hour>:<Minute>:<Second> <Time Zone> <Year>".
Also expressed as "E MMM dd HH:mm:ss z yyyy" in Java SimpleDateFormat format. For example: "Wed Dec 31 12:00:04 UTC 2016".
- *Boolean*: A Boolean is one of either `true` or `false`.
After evaluating expression language functions, all attributes are stored as type String.
The Expression Language is generally able to automatically coerce a value of one data type to the appropriate
data type for a function. However, functions do exist to manually coerce a value into a specific data type.
See the <<type_cast>> section for more information.
Hex values are supported for Number and Decimal types but they must be quoted and prepended with "0x" when being
interpreted as literals. For example these two expressions are valid (without the quotes or "0x" the expression would fail to run properly):
- ${literal("0xF"):toNumber()}
- ${literal("0xF.Fp10"):toDecimal()}
[[boolean]]
== Boolean Logic
One of the most powerful features of the Expression Language is the ability to compare an attribute value against
some other value. This is used often, for example, to configure how a Processor should route data. The following
functions are used for performing boolean logic, such as comparing two values.
Each of these functions are designed to work on values of type Boolean.
[.function]
=== isNull
*Description*: [.description]#The `isNull` function returns `true` if the subject is null, `false` otherwise. This is typically used to determine
if an attribute exists.#
*Subject Type*: [.subject]#Any#
*Arguments*: No arguments
*Return Type*: [.returnType]#Boolean#
*Examples*: `${filename:isNull()}` returns `true` if the "filename" attribute does not exist.
It returns `false` if the attribute exists.
[.function]
=== notNull
*Description*: [.description]#The `notNull` function returns the opposite value of the `isNull` function. That is, it will return `true` if the
subject exists and `false` otherwise.#
*Subject Type*: [.subject]#Any#
*Arguments*: No arguments
*Return Type*: [.returnType]#Boolean#
*Examples*: `${filename:notNull()}` returns `true` if the "filename" attribute exists. It returns `false` if the attribute
does not exist.
[.function]
=== isEmpty
*Description*: [.description]#The `isEmpty` function returns `true` if the Subject is null, does not contain any characters
or contains only white-space (new line, carriage return, space, tab), `false` otherwise.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#Boolean#
*Examples*: `${filename:isEmpty()}` returns `true` if the "filename" attribute does not exist or contains only
white space. `${literal(" "):isEmpty()}` returns true as well as `${literal(""):isEmpty()}`.
[.function]
=== equals
[.description]
*Description*: [.description]#The `equals` function is very widely used and determines if its subject is equal to another String value.
Note that the `equals` function performs a direct comparison of two String values. Take care not to confuse this
function with the <<matches>> function, which evaluates its subject against a Regular Expression.#
[.subject]
*Subject Type*: [.subject]#Any#
[.arguments]
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The value to compare the Subject to. Must be same type as the Subject.#
[.returnType]
*Return Type*: [.returnType]#Boolean#
[.examples]
*Examples*:
We can check if the filename of a FlowFile is "hello.txt" by using the expression `${filename:equals('hello.txt')}`,
or we could check if the value of the attribute `hello` is equal to the value of the `filename` attribute:
`${hello:equals( ${filename} )}`.
[.function]
=== equalsIgnoreCase
*Description*: [.description]#Similar to the `equals` function, the `equalsIgnoreCase` function compares its subject against a String value but returns
`true` if the two values differ only by case (upper case vs. lower case).#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The value to compare the Subject to.#
*Return Type*: [.returnType]#Boolean#
*Examples*: `${filename:equalsIgnoreCase('hello.txt')}` will evaluate to `true` if filename is equal to "hello.txt"
or "HELLO.TXT" or "HeLLo.TxT".
[.function]
=== gt
*Description*: [.description]#The `gt` function is used for numeric comparison and returns `true` if the subject is Greater Than
its argument. If either the subject or the argument cannot be coerced into a Number,
this function returns `false`.#
*Subject Type*: [.subject]#Number#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The number to compare the Subject to.#
*Return Type*: [.returnType]#Boolean#
*Examples*: `${fileSize:gt( 1024 )}` will return `true` if the size of the FlowFile's content is more than 1 kilobyte
(1024 bytes). Otherwise, it will return `false`.
[.function]
=== ge
*Description*: [.description]#The `ge` function is used for numeric comparison and returns `true` if the subject is Greater Than
Or Equal To its argument. If either the subject or the argument cannot be coerced into a Number,
this function returns `false`.#
*Subject Type*: [.subject]#Number#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The number to compare the Subject to.#
*Return Type*: [.returnType]#Boolean#
*Examples*: `${fileSize:ge( 1024 )}` will return `true` if the size of the FlowFile's content is at least (
is greater than or equal to) 1 kilobyte (1024 bytes). Otherwise, it will return `false`.
[.function]
=== lt
*Description*: [.description]#The `lt` function is used for numeric comparison and returns `true` if the subject is Less Than
its argument. If either the subject or the argument cannot be coerced into a Number,
this function returns `false`.#
*Subject Type*: [.subject]#Number#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The number to compare the Subject to.#
*Return Type*: [.returnType]#Boolean#
*Examples*: `${fileSize:lt( 1048576 )}` will return `true` if the size of the FlowFile's content is less than
1 megabyte (1048576 bytes). Otherwise, it will return `false`.
[.function]
=== le
*Description*: [.description]#The `le` function is used for numeric comparison and returns `true` if the subject is Less Than
Or Equal To its argument. If either the subject or the argument cannot be coerced into a Number,
this function returns `false`.#
*Subject Type*: [.subject]#Number#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The number to compare the Subject to.#
*Return Type*: [.returnType]#Boolean#
*Examples*: `${fileSize:le( 1048576 )}` will return `true` if the size of the FlowFile's content is at most
(less than or equal to) 1 megabyte (1048576 bytes). Otherwise, it will return `false`.
[.function]
=== and
*Description*: [.description]#The `and` function takes as a single argument a Boolean value and returns `true` if both the Subject
and the argument are `true`. If either the subject or the argument is `false` or cannot be coerced into a Boolean,
the function returns `false`. Typically, this is used with an embedded Expression as the argument.#
*Subject Type*: [.subject]#Boolean#
*Arguments*:
- [.argName]#_condition_# : [.argDesc]#The right-hand-side of the 'and' Expression#
*Return Type*: [.returnType]#Boolean#
*Examples*: We can check if the filename is both all lower-case and has at least 5 characters by using the Expression
-----------------------------------------------
${filename:toLower():equals( ${filename} ):and(
${filename:length():ge(5)}
)}
-----------------------------------------------
[.function]
=== or
*Description*: [.description]#The `or` function takes as a single argument a Boolean value and returns `true` if either the Subject
or the argument is `true`. If both the subject and the argument are `false`, the function returns `false`. If
either the Subject or the argument cannot be coerced into a Boolean value, this function will return `false`.#
*Subject Type*: [.subject]#Boolean#
*Arguments*:
- [.argName]#_condition_# : [.argDesc]#The right-hand-side of the 'and' Expression#
*Return Type*: [.returnType]#Boolean#
*Examples*: The following example will return `true` if either the filename has exactly 5 characters or if
the filename is all lower-case.
----------------------------------------------
${filename:toLower():equals( ${filename} ):or(
${filename:length():equals(5)}
)}
----------------------------------------------
[.function]
=== not
[.description]
*Description*: [.description]#The `not` function returns the negation of the Boolean value of the subject.#
[.subject]
*Subject Type*: [.subject]#Boolean#
[.arguments]
*Arguments*: No arguments
[.returnType]
*Return Type*: [.returnType]#Boolean#
[.examples]
*Examples*: We can invert the value of another function by using the `not` function, as
`${filename:equals('hello.txt'):not()}`. This will return `true` if the filename is NOT equal to
"hello.txt" and will return `false` if the filename is "hello.txt."
[.function]
=== ifElse
*Description*: [.description]#Evaluates the first argument if the Subject evaluates to `true`, or the second argument
if the Subject evaluates to `false`.#
*Subject Type*: [.subject]#Boolean#
*Arguments*:
- [.argName]#_EvaluateIfTrue_# : [.argDesc]#The value to return if the Subject is true#
- [.argName]#_EvaluateIfFalse_# : [.argDesc]#The value to return if the Subject is false#
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt", the "nullFilename" attribute has
the value null, and the "bool" attribute has the value "true", then the following expressions will provide
the following results:
.ifElse Examples
|===================================================================
| Expression | Value
| `${bool:ifElse('a','b')}` | `a`
| `${literal(true):ifElse('a','b')}` | `a`
| `${nullFilename:isNull():ifElse('file does not exist', 'located file')}` | `file does not exist`
| `${nullFilename:ifElse('found', 'not_found')}` | `not_found`
| `${filename:ifElse('found', 'not_found')}` | `not_found`
| `${filename:isNull():not():ifElse('found', 'not_found')}` | `found`
|===================================================================
[[strings]]
== String Manipulation
Each of the following functions manipulates a String in some way.
[.function]
=== toUpper
*Description*: [.description]#This function converts the Subject into an all upper-case String. Said another way, it
replaces any lowercase letter with the uppercase equivalent.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute is "abc123.txt", then the Expression `${filename:toUpper()}`
will return "ABC123.TXT"
[.function]
=== toLower
*Description*: [.description]#This function converts the Subject into an all lower-case String. Said another way,
it replaces any uppercase letter with the lowercase equivalent.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute is "ABC123.TXT", then the Expression `${filename:toLower()}`
will return "abc123.txt"
[.function]
=== trim
*Description*: [.description]#The `trim` function will remove any leading or trailing white space from its subject.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the attribute `attr` has the value " 1 2 3 ", then the Expression `${attr:trim()}` will
return the value "1 2 3".
[.function]
=== substring
*Description*:
[.description]#Returns a portion of the Subject, given a _starting index_ and an optional _ending index_.
If the _ending index_ is not supplied, it will return the portion of the Subject starting at the given
'start index' and ending at the end of the Subject value.#
[.description]#The _starting index_ and _ending index_ are zero-based. That is, the first character is referenced by using
the value `0`, not `1`.#
[.description]#If either the _starting index_ is or the _ending index_ is not a number, this function call will result
in an error.#
[.description]#If the _starting index_ is larger than the _ending index_, this function call will result in an error.#
[.description]#If the _starting index_ or the _ending index_ is greater than the length of the Subject or has a value
less than 0, this function call will return an empty string.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_starting index_# : [.argDesc]#The 0-based index of the first character to capture (inclusive)#
- [.argName]#_ending index_# : [.argDesc]#The 0-based index of the last character to capture (exclusive)#
*Return Type*: [.returnType]#String#
*Examples*:
If we have an attribute named "filename" with the value "a brand new filename.txt",
then the following Expressions will result in the following values:
.Substring Examples
|================================================================
| Expression | Value
| `${filename:substring(0,1)}` | `a`
| `${filename:substring(2)}` | `brand new filename.txt`
| `${filename:substring(12)}` | `filename.txt`
| `${filename:substring( ${filename:length():minus(2)} )}` | `xt`
|================================================================
[.function]
=== substringBefore
*Description*: [.description]#Returns a portion of the Subject, starting with the first character of the Subject
and ending with the character immediately before the first occurrence of the argument. If
the argument is not present in the Subject, the entire Subject will be returned.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The String to search for in the Subject#
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt",
then the following Expressions will result in the following values:
.SubstringBefore Examples
|======================================================================
| Expression | Value
| `${filename:substringBefore('.')}` | `a brand new filename`
| `${filename:substringBefore(' ')}` | `a`
| `${filename:substringBefore(' n')}` | `a brand`
| `${filename:substringBefore('missing')}` | `a brand new filename.txt`
|======================================================================
[.function]
=== substringBeforeLast
*Description*: [.description]#Returns a portion of the Subject, starting with the first character of the Subject
and ending with the character immediately before the last occurrence of the argument. If
the argument is not present in the Subject, the entire Subject will be returned.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The String to search for in the Subject#
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt",
then the following Expressions will result in the following values:
.SubstringBeforeLast Examples
|==========================================================================
| Expression | Value
| `${filename:substringBeforeLast('.')}` | `a brand new filename`
| `${filename:substringBeforeLast(' ')}` | `a brand new`
| `${filename:substringBeforeLast(' n')}` | `a brand`
| `${filename:substringBeforeLast('missing')}` | `a brand new filename.txt`
|==========================================================================
[.function]
=== substringAfter
*Description*: [.description]#Returns a portion of the Subject, starting with the character immediately after
the first occurrence of the argument and extending to the end of the Subject. If
the argument is not present in the Subject, the entire Subject will be returned.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The String to search for in the Subject#
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt",
then the following Expressions will result in the following values:
.SubstringAfter Examples
|======================================================================
| Expression | Value
| `${filename:substringAfter('.')}` | `txt`
| `${filename:substringAfter(' ')}` | `brand new filename.txt`
| `${filename:substringAfter(' n')}` | `ew filename.txt`
| `${filename:substringAfter('missing')}` | `a brand new filename.txt`
|======================================================================
[.function]
=== substringAfterLast
*Description*: [.description]#Returns a portion of the Subject, starting with the character immediately after
the last occurrence of the argument and extending to the end of the Subject. If
the argument is not present in the Subject, the entire Subject will be returned.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The String to search for in the Subject#
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt",
then the following Expressions will result in the following values:
.SubstringAfterLast Examples
|=========================================================================
| Expression | Value
| `${filename:substringAfterLast('.')}` | `txt`
| `${filename:substringAfterLast(' ')}` | `filename.txt`
| `${filename:substringAfterLast(' n')}` | `ew filename.txt`
| `${filename:substringAfterLast('missing')}` | `a brand new filename.txt`
|=========================================================================
[.function]
=== getDelimitedField
*Description*: [.description]#Parses the Subject as a delimited line of text and returns just a single field
from that delimited text.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_index_# : [.argDesc]#The index of the field to return. A value of 1 will return the first field,
a value of 2 will return the second field, and so on.#
- [.argName]#_delimiter_# : [.argDesc]#Optional argument that provides the character to use as a field separator.
If not specified, a comma will be used. This value must be exactly 1 character.#
- [.argName]#_quoteChar_# : [.argDesc]#Optional argument that provides the character that can be used to quote values
so that the delimiter can be used within a single field. If not specified, a double-quote (") will be used. This value
must be exactly 1 character.#
- [.argName]#_escapeChar_# : [.argDesc]#Optional argument that provides the character that can be used to escape the Quote Character
or the Delimiter within a field. If not specified, a backslash (\) is used. This value must be exactly 1 character.#
- [.argName]#_stripChars_# : [.argDesc]#Optional argument that specifies whether or not quote characters and escape characters should
be stripped. For example, if we have a field value "1, 2, 3" and this value is true, we will get the value `1, 2, 3`, but if this
value is false, we will get the value `"1, 2, 3"` with the quotes. The default value is false. This value must be either `true`
or `false`.#
*Return Type*: [.returnType]#String#
*Examples*: If the "line" attribute contains the value _"Jacobson, John", 32, Mr._
and the "altLine" attribute contains the value _Jacobson, John|32|Mr._
then the following Expressions will result in the following values:
.GetDelimitedField Examples
|======================================================================
| Expression | Value
| `${line:getDelimitedField(2)}` | _(space)_32
| `${line:getDelimitedField(2):trim()}` | 32
| `${line:getDelimitedField(1)}` | "Jacobson, John"
| `${line:getDelimitedField(1, ',', '"', '\\', true)}` | Jacobson, John
| `${altLine:getDelimitedField(1, '\|')}` | Jacobson, John
|======================================================================
[.function]
=== append
*Description*: [.description]#The `append` function returns the result of appending the argument to the value of
the Subject. If the Subject is null, returns the argument itself.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The String to append to the end of the Subject#
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt", then the Expression
`${filename:append('.gz')}` will return "a brand new filename.txt.gz".
[.function]
=== prepend
*Description*: [.description]#The `prepend` function returns the result of prepending the argument to the value of
the Subject. If the subject is null, returns the argument itself.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The String to prepend to the beginning of the Subject#
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute has the value "filename.txt", then the Expression
`${filename:prepend('a brand new ')}` will return "a brand new filename.txt".
[.function]
=== replace
*Description*: [.description]#Replaces *all* occurrences of one literal String within the Subject with another String.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_Search String_# : [.argDesc]#The String to find within the Subject#
- [.argName]#_Replacement_# : [.argDesc]#The value to replace _Search String_ with#
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt", then the following
Expressions will provide the following results:
.Replace Examples
|===================================================================
| Expression | Value
| `${filename:replace('.', '_')}` | `a brand new filename_txt`
| `${filename:replace(' ', '.')}` | `a.brand.new.filename.txt`
| `${filename:replace('XYZ', 'ZZZ')}` | `a brand new filename.txt`
| `${filename:replace('filename', 'book')}` | `a brand new book.txt`
|===================================================================
[.function]
=== replaceFirst
*Description*: [.description]#Replaces *the first* occurrence of one literal String or regular expression within the Subject with another String.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_Search String_# : [.argDesc]#The String (literal or regular expression pattern) to find within the Subject#
- [.argName]#_Replacement_# : [.argDesc]#The value to replace _Search String_ with#
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt", then the following
Expressions will provide the following results:
.ReplaceFirst Examples
|===================================================================
| Expression | Value
| `${filename:replaceFirst('a', 'the')}` | `the brand new filename.txt`
| `${filename:replaceFirst('[br]', 'g')}` | `a grand new filename.txt`
| `${filename:replaceFirst('XYZ', 'ZZZ')}` | `a brand new filename.txt`
| `${filename:replaceFirst('\w{8}', 'book')}` | `a brand new book.txt`
|===================================================================
[.function]
=== replaceAll
*Description*: [.description]#The `replaceAll` function takes two String arguments: a literal String or Regular Expression (NiFi uses the Java Pattern
syntax), and a replacement string. The return value is the result of substituting the replacement string for
all patterns within the Subject that match the Regular Expression.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_Regex_# : [.argDesc]#The Regular Expression (in Java syntax) to match in the Subject#
- [.argName]#_Replacement_# : [.argDesc]#The value to use for replacing matches in the Subject. If the _regular expression_
argument uses Capturing Groups, back references are allowed in the _replacement_.#
*Return Type*: [.returnType]#String#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt", then the following
Expressions will provide the following results:
.ReplaceAll Examples
|=======================================================================================
| Expression | Value
| `${filename:replaceAll('\..*', '')}` | `a brand new filename`
| `${filename:replaceAll('a brand (new)', '$1')}` | `new filename.txt`
| `${filename:replaceAll('XYZ', 'ZZZ')}` | `a brand new filename.txt`
| `${filename:replaceAll('brand (new)', 'somewhat $1')}` | `a somewhat new filename.txt`
|=======================================================================================
[.function]
=== padLeft
*Description*: [.description]#The `padLeft` function prepends the given padding string (or `'_'`, if nothing is provided) to the argument `String` until the passed desired length is reached.#
It returns the argument as is if its length is already equal or higher than the desired length, if the padding string is `null`, and if the desired length is either negative or greater than `Integer.MAX_VALUE`.
It returns `null` if the argument string is not a valid attribute.
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_DesiredLength_# : [.argDesc]#The integer value to pad to.#
- [.argName]#_PaddingString_# : [.argDesc]#The optional string to pad with. `"_"` will be used if a `PaddingString` is not provided. If the `PaddingString` is not an exact multiple of the actual pad size, it will be trimmed to fit in `DesiredLength`.#
*Return Type*: [.returnType]#String#
*Examples*: If the "greetings" attribute has the value "hello", then the following
Expressions will provide the following results:
.PadLeft Examples
|=======================================================================================
| Expression | Value
| `${greetings:padLeft(10)}` | `\_____hello`
| `${greetings:padLeft(10, '@')}` | `@@@@@hello`
| `${greetings:padLeft(10, 'xy')}` | `xyxyxhello`
| `${greetings:padLeft(10, 'aVeryLongPaddingString')}` | `aVeryhello`
|=======================================================================================
[.function]
=== padRight
*Description*: [.description]#The `padRight` function appends the given padding string (or `'_'`, if nothing is provided) to the argument `String` until the passed desired length is reached.#
It returns the argument as is if its length is already equal or higher than the desired length, if the padding string is `null`, and if the desired length is either negative or greater than `Integer.MAX_VALUE`.
It returns `null` if the argument string is not a valid attribute.
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_DesiredLength_# : [.argDesc]#The integer value to pad to.#
- [.argName]#_PaddingString_# : [.argDesc]#The optional string to pad with. `"_"` will be used if a `PaddingString` is not provided. If the `PaddingString` is not an exact multiple of the actual pad size, it will be trimmed to fit in `DesiredLength`.#
*Return Type*: [.returnType]#String#
*Examples*: If the "greetings" attribute has the value "hello", then the following
Expressions will provide the following results:
.PadLeft Examples
|=======================================================================================
| Expression | Value
| `${greetings:padRight(10)}` | `hello\_____`
| `${greetings:padRight(10, '@')}` | `hello@@@@@`
| `${greetings:padRight(10, 'xy')}` | `helloxyxyx`
| `${greetings:padLeft(10, 'aVeryLongPaddingString')}` | `helloaVery`
|=======================================================================================
[.function]
=== replaceNull
*Description*: [.description]#The `replaceNull` function returns the argument if the Subject is null. Otherwise,
returns the Subject.#
*Subject Type*: [.subject]#Any#
*Arguments*:
- [.argName]#_Replacement_# : [.argDesc]#The value to return if the Subject is null.#
*Return Type*: [.returnType]#Type of Subject if Subject is not null; else, type of Argument#
*Examples*: If the attribute "filename" has the value "a brand new filename.txt" and the attribute
"hello" does not exist, then the Expression `${filename:replaceNull('abc')}` will return
"a brand new filename.txt", while `${hello:replaceNull('abc')}` will return "abc".
[.function]
=== replaceEmpty
*Description*: [.description]#The `replaceEmpty` function returns the argument if the Subject is null or
if the Subject consists only of white space (new line, carriage return, tab, space). Otherwise,
returns the Subject.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_Replacement_# : [.argDesc]#The value to return if the Subject is null or empty.#
*Return Type*: [.returnType]#String#
*Examples*: If the attribute "filename" has the value "a brand new filename.txt" and the attribute
"hello" has the value " ", then the Expression `${filename:replaceEmpty('abc')}` will return
"a brand new filename.txt", while `${hello:replaceEmpty('abc')}` will return "abc".
[.function]
=== length
*Description*: [.description]#Returns the length of the Subject#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#Number#
*Examples*: If the attribute "filename" has a value of "a brand new filename.txt" and the attribute
"hello" does not exist, then the Expression `${filename:length()}` will return 24. `${hello:length()}`
will return 0.
[.function]
=== evaluateELString
*Description*: [.description]#This function evaluates the Expression Language inside the variable value.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If one of the registry variable named "query" has value "SELECT * FROM TABLE WHERE ID = ${id}"
and the value of the "id" field, we are getting from the flowfile attributes (i.e. id=20)
then the Expression `${query:evaluateELString()}` will return SELECT * FROM TABLE WHERE ID = 20
[.function]
=== repeat
*Description*:
[.description]#Returns a string that is the Subject repeated a random number of times between _min repeats_ and
_max repeats_. If _max repeats_ is not supplied, it will return the Subject repeated exactly _min repeats_ times.#
[.description]#The _min repeats_ and _max repeats_ must be positive numbers, with _max repeats_ greater than or equal
to _min repeats_#
[.description]#If either _min repeats_ or _max repeats_ is not a number, this function call will result
in an error.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_min repeats_# : [.argDesc]#The minimum number (inclusive) of times to repeat the subject, or the exact number
of times to repeat the subject if _max repeats_ is not supplied.#
- [.argName]#_max repeats_# : [.argDesc]#The maximum number (inclusive) of times to repeat the subject.#
*Return Type*: [.returnType]#String#
*Examples*:
If we have an attribute named "str" with the value "abc",
then the following Expressions will result in the following values:
.Repeat Examples
|================================================================
| Expression | Value
| `${str:repeat(1)}` | `abc`
| `${str:repeat(2)}` | `abcabc`
| `${str:repeat(1,2)}` | `abc` or `abcabc` (at random)
| `${str:repeat( ${str:length()} )}` | `abc` or `abcabc` or `abcabcabc` (at random)
|================================================================
[[encode]]
== Encode/Decode Functions
Each of the following functions will encode a string according the rules of the given data format.
[.function]
=== escapeJson
*Description*: [.description]#This function prepares the Subject to be inserted into JSON document by escaping the characters
in the String using Json String rules. The function correctly escapes quotes and control-chars (tab, backslash,
cr, ff, etc.)#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "message" attribute is 'He didn't say, "Stop!"', then the Expression `${message:escapeJson()}`
will return 'He didn't say, \"Stop!\"'
[.function]
=== escapeXml
*Description*: [.description]#This function prepares the Subject to be inserted into XML document by escaping the characters
in a String using XML entities. The function correctly escapes quotes, apostrophe, ampersand, <, > and
control-chars.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "message" attribute is '"bread" & "butter"', then the Expression `${message:escapeXml()}`
will return '&quot;bread&quot; &amp; &quot;butter&quot;'
[.function]
=== escapeCsv
*Description*: [.description]#This function prepares the Subject to be inserted into CSV document by escaping the characters
in a String using the rules in RFC 4180. The function correctly escapes quotes and surround the string in quotes if needed.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "message" attribute is 'But finally, she left', then the Expression `${message:escapeCsv()}`
will return '"But finally, she left"'
[.function]
=== escapeHtml3
*Description*: [.description]#This function prepares the Subject to be inserted into HTML document by escaping the characters
in a String using the HTML entities. Supports only the HTML 3.0 entities.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "message" attribute is '"bread" & "butter"', then the Expression `${message:escapeHtml3()}`
will return '&quot;bread&quot; &amp; &quot;butter&quot;'
[.function]
=== escapeHtml4
*Description*: [.description]#This function prepares the Subject to be inserted into HTML document by escaping the characters
in a String using the HTML entities. Supports all known HTML 4.0 entities.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "message" attribute is '"bread" & "butter"', then the Expression `${message:escapeHtml4()}`
will return '&quot;bread&quot; &amp; &quot;butter&quot;'
[.function]
=== unescapeJson
*Description*: [.description]#This function unescapes any Json literals found in the String.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "message" attribute is 'He didn't say, \"Stop!\"', then the Expression `${message:unescapeJson()}`
will return 'He didn't say, "Stop!"'
[.function]
=== unescapeXml
*Description*: [.description]#This function unescapes a string containing XML entity escapes to a string containing the
actual Unicode characters corresponding to the escapes. Supports only the five basic XML entities (gt, lt,
quot, amp, apos).#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "message" attribute is '&quot;bread&quot; &amp; &quot;butter&quot;', then the Expression `${message:unescapeXml()}`
will return '"bread" & "butter"'
[.function]
=== unescapeCsv
*Description*: [.description]#This function unescapes a String from a CSV document according to the rules of RFC 4180.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "message" attribute is '"But finally, she left"', then the Expression `${message:unescapeCsv()}`
will return 'But finally, she left'
[.function]
=== unescapeHtml3
*Description*: [.description]#This function unescapes a string containing HTML 3 entity to a string containing the
actual Unicode characters corresponding to the escapes. Supports only HTML 3.0 entities.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "message" attribute is '&quot;bread&quot; &amp; &quot;butter&quot;', then the Expression `${message:unescapeHtml3()}`
will return '"bread" & "butter"'
[.function]
=== unescapeHtml4
*Description*: [.description]#This function unescapes a string containing HTML 4 entity to a string containing the
actual Unicode characters corresponding to the escapes. Supports all known HTML 4.0 entities.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If the "message" attribute is '&quot;bread&quot; &amp; &quot;butter&quot;', then the Expression `${message:unescapeHtml4()}`
will return '"bread" & "butter"'
[.function]
=== urlEncode
*Description*: [.description]#Returns a URL-friendly version of the Subject. This is useful, for instance, when using an
attribute value to indicate the URL of a website.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: We can URL-Encode an attribute named "url" by using the Expression `${url:urlEncode()}`. If
the value of the "url" attribute is "https://nifi.apache.org/some value with spaces", this
Expression will then return "https%3A%2F%2Fnifi.apache.org%2Fsome+value+with+spaces".
[.function]
=== urlDecode
*Description*: [.description]#Converts a URL-friendly version of the Subject into a human-readable form.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If we have a URL-Encoded attribute named "url" with the value
"https://nifi.apache.org/some%20value%20with%20spaces" or "https%3A%2F%2Fnifi.apache.org%2Fsome+value+with+spaces", then the Expression
`${url:urlDecode()}` will return "https://nifi.apache.org/some value with spaces".
[.function]
=== base64Encode
*Description*: [.description]#Returns a Base64 encoded string. This is useful for being able to transfer binary data as ascii.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: We can Base64-Encoded an attribute named "payload" by using the Expression
`${payload:base64Encode()}` If the attribute payload had a value of "admin:admin"
then the Expression `${payload:base64Encode()}` will return "YWRtaW46YWRtaW4=".
[.function]
=== base64Decode
*Description*: [.description]#Reverses the Base64 encoding on given string.#
*Subject Type*: [.subject]#String#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: If we have a Base64-Encoded attribute named "payload" with the value
"YWRtaW46YWRtaW4=", then the Expression
`${payload:base64Decode()}` will return "admin:admin".
[.function]
=== UUID3
*Description*: [.description]#Returns a type 3 (MD5 hashed) namespace name-based UUID. The argument must be a valid UUID.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_namespace_# : [.argDesc]#The namespace UUID identifier#
*Return Type*: [.returnType]#String#
*Examples*: If we have an attribute named "attr" with a value of "string value", then the Expression `${attr:UUID3('b9e81de3-7047-4b5e-a822-8fff5b49f808')}` will return "bf0ea246-a177-3300-bd7e-d4c9e973dc6f".
An empty argument or an argument value with an invalid UUID results in an exception bulletin.
*See Also*: [.seeAlso]#<<uuid,UUID()>>#
[.function]
=== UUID5
*Description*: [.description]#Returns a type 5 (SHA-1 hashed) namespace name-based UUID. The argument must be a valid UUID.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_namespace_# : [.argDesc]#The namespace UUID identifier#
*Return Type*: [.returnType]#String#
*Examples*: If we have an attribute named "attr" with a value of "string value", then the Expression `${attr:UUID5('245b55a8-397d-4480-a41e-16603c8cf9ad')}` will return "4d111477-5100-5f2d-ae79-b38bbe15aa78".
An empty argument or an argument value with an invalid UUID results in an exception bulletin.
*See Also*: [.seeAlso]#<<uuid,UUID()>>#
[.function]
=== hash
*Description*: [.description]#Returns a hex encoded string using the hash algorithm provided. This can be used to generate unique keys.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_algorithm_# : [.argDesc]#An algorithm to hash value.
Supports one of [SHA-384, SHA-224, SHA-256, MD2, SHA, SHA-512, MD5]. Warning: MD2, MD5, and SHA (SHA-1) should not be considered cryptographically secure (link:https://csrc.nist.gov/projects/hash-functions/nist-policy-on-hash-functions[https://csrc.nist.gov/projects/hash-functions/nist-policy-on-hash-functions^]).#
*Return Type*: [.returnType]#String#
*Examples*: We can hash an attribute named "payload" by using the Expression
`${payload:hash('SHA-256')}` If the attribute payload had a value of "string value"
then the Expression `${payload:hash('SHA-256')}` will return "9b6a1a9167a5caf3f5948413faa89e0ec0de89e12bef55327442e60dcc0e8c9b".
[[searching]]
== Searching
Each of the following functions is used to search its subject for some value.
[.function]
=== startsWith
*Description*: [.description]#Returns `true` if the Subject starts with the String provided as the argument,
`false` otherwise.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The value to search for#
*Return Type*: [.returnType]#Boolean#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt", then the Expression
`${filename:startsWith('a brand')}` will return `true`. `${filename:startsWith('A BRAND')}` will
return `false`. `${filename:toUpper():startsWith('A BRAND')}` returns `true`.
[.function]
=== endsWith
*Description*: [.description]#Returns `true` if the Subject ends with the String provided as the argument,
`false` otherwise.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The value to search for#
*Return Type*: [.returnType]#Boolean#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt", then the Expression
`${filename:endsWith('txt')}` will return `true`. `${filename:endsWith('TXT')}` will
return `false`. `${filename:toUpper():endsWith('TXT')}` returns `true`.
[.function]
=== contains
*Description*: [.description]#Returns `true` if the Subject contains the value of the argument anywhere in the value.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The value to search for#
*Return Type*: [.returnType]#Boolean#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt", then the Expression
`${filename:contains('new')}` will return `true`. `${filename:contains('NEW')}` will
return `false`. `${filename:toUpper():contains('NEW')}` returns `true`.
[.function]
=== in
*Description*: [.description]#Returns `true` if the Subject is matching one of the provided arguments.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value1_# : [.argDesc]#First possible matching value#
- [.argName]#_valueN_# : [.argDesc]#Nth possible matching value#
*Return Type*: [.returnType]#Boolean#
*Examples*: If the "myEnum" attribute has the value "JOHN", then the Expression
`${myEnum:in("PAUL", "JOHN", "MIKE")}` will return `true`. `${myEnum:in("RED", "GREEN", "BLUE")}` will
return `false`.
[.function]
=== find
*Description*: [.description]#Returns `true` if the Subject contains any sequence of characters that matches the
Regular Expression provided by the argument.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_Regex_# : [.argDesc]#The Regular Expression (in the Java Pattern syntax) to match against the Subject#
*Return Type*: [.returnType]#Boolean#
*Examples*:
If the "filename" attribute has the value "a brand new filename.txt", then the following
Expressions will provide the following results:
.find Examples
|=======================================================================================
| Expression | Value
| `${filename:find('a [Bb]rand [Nn]ew')}` | `true`
| `${filename:find('Brand.*')}` | `false`
| `${filename:find('brand')}` | `true`
|=======================================================================================
[.function]
=== matches
*Description*: [.description]#Returns `true` if the Subject exactly matches the Regular Expression provided by the argument.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_Regex_# : [.argDesc]#The Regular Expression (in the Java Pattern syntax) to match against the Subject#
*Return Type*: [.returnType]#Boolean#
*Examples*:
If the "filename" attribute has the value "a brand new filename.txt", then the following
Expressions will provide the following results:
.matches Examples
|=======================================================================================
| Expression | Value
| `${filename:matches('a.*txt')}` | `true`
| `${filename:matches('brand')}` | `false`
| `${filename:matches('.+brand.+')}` | `true`
|=======================================================================================
[.function]
=== indexOf
*Description*: [.description]#Returns the index of the first character in the Subject that matches the String value provided
as an argument. If the argument is found multiple times within the Subject, the value returned is the
starting index of the *first* occurrence.
If the argument cannot be found in the Subject, returns `-1`. The index is zero-based. This means that if
the search string is found at the beginning of the Subject, the value returned will be `0`, not `1`.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The value to search for in the Subject#
*Return Type*: [.returnType]#Number#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt", then the following
Expressions will provide the following results:
.indexOf Examples
|===============================================
| Expression | Value
| `${filename:indexOf('a.*txt')}` | `-1`
| `${filename:indexOf('.')}` | `20`
| `${filename:indexOf('a')}` | `0`
| `${filename:indexOf(' ')}` | `1`
|===============================================
[.function]
=== lastIndexOf
*Description*: [.description]#Returns the index of the first character in the Subject that matches the String value provided
as an argument. If the argument is found multiple times within the Subject, the value returned is the
starting index of the *last* occurrence.
If the argument cannot be found in the Subject, returns `-1`. The index is zero-based. This means that if
the search string is found at the beginning of the Subject, the value returned will be `0`, not `1`.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The value to search for in the Subject#
*Return Type*: [.returnType]#Number#
*Examples*: If the "filename" attribute has the value "a brand new filename.txt", then the following
Expressions will provide the following results:
.lastIndexOf Examples
|=======================================================================================
| Expression | Value
| `${filename:lastIndexOf('a.*txt')}` | `-1`
| `${filename:lastIndexOf('.')}` | `20`
| `${filename:lastIndexOf('a')}` | `17`
| `${filename:lastIndexOf(' ')}` | `11`
|=======================================================================================
[.function]
=== jsonPath
*Description*: [.description]#The `jsonPath` function generates a string by evaluating the Subject as JSON and applying a JSON
path expression. An empty string is generated if the Subject does not contain valid JSON, the _jsonPath_ is invalid, or the path
does not exist in the Subject. If the evaluation results in a scalar value, the string representation of scalar value is
generated. Otherwise a string representation of the JSON result is generated. A JSON array of length 1 is special cased
when `[0]` is a scalar, the string representation of `[0]` is generated.#
*Subject Type*: [.subject]#String#
*Arguments*:
[.argName]#_jsonPath_# : [.argDesc]#the JSON path expression used to evaluate the Subject.#
*Return Type*: [.returnType]#String#
*Examples*: If the "myJson" attribute is
..........
{
"firstName": "John",
"lastName": "Smith",
"isAlive": true,
"age": 25,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"phoneNumbers": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "office",
"number": "646 555-4567"
}
],
"children": [],
"spouse": null
}
..........
.jsonPath Examples
|===================================================================
| Expression | Value
| `${myJson:jsonPath('$.firstName')}` | `John`
| `${myJson:jsonPath('$.address.postalCode')}` | `10021-3100`
| `${myJson:jsonPath('$.phoneNumbers[?(@.type=="home")].number')}` | `212 555-1234`
| `${myJson:jsonPath('$.phoneNumbers')}` | `[{"type":"home","number":"212 555-1234"},{"type":"office","number":"646 555-4567"}]`
| `${myJson:jsonPath('$.missing-path')}` | _empty_
| `${myJson:jsonPath('$.bad-json-path..')}` | _exception bulletin_
|===================================================================
An empty subject value or a subject value with an invalid JSON document results in an exception bulletin.
[.function]
=== jsonPathDelete
*Description*: [.description]#The `jsonPathDelete` function deletes the specified JsonPath from a Subject JSON and returns string
form of the updated JSON.#
*Subject Type*: [.subject]#String#
*Arguments*:
[.argName]#_jsonPath_# : [.argDesc]#the JSON path expression to delete from the Subject.#
*Return Type*: [.returnType]#String#
*Examples*: If the "myJson" attribute is
..........
{
"firstName": "John",
"lastName": "Smith",
"isAlive": true,
"age": 25,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"phoneNumbers": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "office",
"number": "646 555-4567"
}
],
"children": [],
"spouse": null
}
..........
.jsonPathDelete Examples
|===================================================================
| Expression | Value
| `${myJson:jsonPathDelete('$.firstName')}` | `{"lastName":"Smith","age":25,"address":{"streetAddress":"21 2nd Street","city":"New York","state":"NY","postalCode":"10021-3100"},"phoneNumbers":[{"type":"home","number":"212 555-1234"},{"type":"office","number":"646 555-4567"}]}`
| `${myJson:jsonPathDelete('$.missing-path')}` | Returns original JSON document
|===================================================================
An empty subject value or a subject value with an invalid JSON document results in an exception bulletin.
[.function]
=== jsonPathAdd
*Description*: [.description]#The `jsonPathAdd` function adds a scalar value to an array at the specified JsonPath on
a Subject JSON and returns string form of the updated JSON.# If the expression target element is a non-array,
an empty string is returned and an exception is logged indicating the error. If the expression target element
path is not in the JSON, the operation returns the original JSON without any modifications.
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_jsonPath_# : [.argDesc]#the JSON path expression to set value on the Subject.#
- [.argName]#_value_# : [.argDesc]#the value expression to be added to the array at the specified path on Subject.#
*Return Type*: [.returnType]#String#
*Examples*: If the "myJson" attribute is
..........
{
"firstName": "John",
"lastName": "Smith",
"age": 25,
"voter" : true,
"height" : 6.1,
"address" : {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"phoneNumbers": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "office",
"number": "646 555-4567"
}
],
"nicknames" : []
}
..........
.jsonPathAdd Examples
|===================================================================
| Expression | Value
| `${myJson:jsonPathAdd('$.nicknames', 'Jimmy')}` | `{"firstName":"James", lastName":"Smith", "age":25, "voter":true, "height":6.1, "address":{"streetAddress":"21 2nd Street", "city":"New York", "state":"NY", "postalCode":"10021-3100"}, "phoneNumbers":[{"type":"home", "number":"212 555-1234"}, {"type":"office", "number":"646 555-4567"}],"nicknames":["Jimmy"]}`
| `${myJson:jsonPathAdd('$.missingpath', 'Jimmy')}` | Returns original JSON document with no modifications
| `${myJson:jsonPathAdd('$.firstName', 'Jimmy')}` | _empty_
|===================================================================
[.function]
=== jsonPathSet
*Description*: [.description]#The `jsonPathSet` function sets the value at the specified JsonPath on a Subject JSON and returns string
form of the updated JSON.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_jsonPath_# : [.argDesc]#the JSON path expression to set value on the Subject.#
- [.argName]#_value_# : [.argDesc]#the value expression to be set on the specified path on Subject.#
*Return Type*: [.returnType]#String#
*Examples*: If the "myJson" attribute is
..........
{
"firstName": "John",
"lastName": "Smith",
"age": 25,
"voter" : true,
"height" : 6.1,
"address" : {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"phoneNumbers": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "office",
"number": "646 555-4567"
}
],
"nicknames" : []
}
..........
.jsonPathSet Examples
|===================================================================
| Expression | Value
| `${myJson:jsonPathSet('$.firstName', 'James')}` | `{"firstName":"James", lastName":"Smith", "age":25, "voter":true, "height":6.1, "address":{"streetAddress":"21 2nd Street", "city":"New York", "state":"NY", "postalCode":"10021-3100"}, "phoneNumbers":[{"type":"home", "number":"212 555-1234"}, {"type":"office", "number":"646 555-4567"}]}`
| `${myJson:jsonPathSet('$.missingpath', 'James')}` | Returns original JSON document
|===================================================================
An empty subject value or a subject value with an invalid JSON document results in an exception bulletin.
[.function]
=== jsonPathPut
*Description*: [.description]#The `jsonPathPut` function puts the key and scalar value at the specified JsonPath on a Subject JSON and returns string
form of the updated JSON.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_jsonPath_# : [.argDesc]#the JSON path expression to set value on the Subject.#
- [.argName]#_value_# : [.argDesc]#the value expression to be set on the specified path on Subject.#
- [.argName]#_key_# : [.argDesc]#the key expression with the associated value the specified path on Subject.#
*Return Type*: [.returnType]#String#
*Examples*: If the "myJson" attribute is
..........
{
"firstName": "John",
"lastName": "Smith",
"age": 25,
"voter" : true,
"height" : 6.1,
"address" : {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"phoneNumbers": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "office",
"number": "646 555-4567"
}
],
"nicknames" : []
}
..........
.jsonPathPut Examples
|===================================================================
| Expression | Value
| `${myJson:jsonPathPut('$','middlename','Turon')}` | `{"firstName":"James", lastName":"Smith", "middlename": "Turon", "age":25, "voter":true, "height":6.1, "address":{"streetAddress":"21 2nd Street", "city":"New York", "state":"NY", "postalCode":"10021-3100"}, "phoneNumbers":[{"type":"home", "number":"212 555-1234"}, {"type":"office", "number":"646 555-4567"}]}`
|===================================================================
An empty subject value or a subject value with an invalid JSON document results in an exception bulletin.
[[numbers]]
== Mathematical Operations and Numeric Manipulation
For those functions that support Decimal and Number (whole number) types, the return value type depends on the input types. If either the
subject or argument are a Decimal then the result will be a Decimal. If both values are Numbers then the result will be a Number. This includes
Divide. This is to preserve backwards compatibility and to not force rounding errors.
[.function]
=== plus
*Description*: [.description]#Adds a numeric value to the Subject. If either the argument or the Subject cannot be
coerced into a Number, returns `null`. Does not provide handling for overflow.#
*Subject Type*: [.subject]#Number or Decimal#
*Arguments*:
- [.argName]#_Operand_# : [.argDesc]#The value to add to the Subject#
*Return Type*: [.returnType]#Number or Decimal (depending on input types)#
*Examples*: If the "fileSize" attribute has a value of 100, then the Expression `${fileSize:plus(1000)}`
will return the value `1100`.
[.function]
=== minus
*Description*: [.description]#Subtracts a numeric value from the Subject. Does not provide handling for overflow.#
*Subject Type*: [.subject]#Number or Decimal#
*Arguments*:
- [.argName]#_Operand_# : [.argDesc]#The value to subtract from the Subject#
*Return Type*: [.returnType]#Number or Decimal (depending on input types)#
*Examples*: If the "fileSize" attribute has a value of 100, then the Expression `${fileSize:minus(100)}`
will return the value `0`.
[.function]
=== multiply
*Description*: [.description]#Multiplies a numeric value by the Subject and returns the product.# Does not provide handling for overflow.
*Subject Type*: [.subject]#Number or Decimal#
*Arguments*:
- [.argName]#_Operand_# : [.argDesc]#The value to multiple the Subject by#
*Return Type*: [.returnType]#Number or Decimal (depending on input types)#
*Examples*: If the "fileSize" attribute has a value of 100, then the Expression `${fileSize:multiply(1024)}`
will return the value `102400`.
[.function]
=== divide
*Description*: [.description]#Divides the Subject by a numeric value and returns the result.#
*Subject Type*: [.subject]#Number or Decimal#
*Arguments*:
- [.argName]#_Operand_# : [.argDesc]#The value to divide the Subject by#
*Return Type*: [.returnType]#Number or Decimal (depending on input types)#
*Examples*: If the "fileSize" attribute has a value of 100, then the Expression `${fileSize:divide(12)}`
will return the value `8`.
[.function]
=== mod
*Description*: [.description]#Performs a modular division of the Subject by the argument. That is, this function will divide
the Subject by the value of the argument and return not the quotient but rather the remainder.#
*Subject Type*: [.subject]#Number or Decimal#
*Arguments*:
- [.argName]#_Operand_# : [.argDesc]#The value to divide the Subject by#
*Return Type*: [.returnType]#Number or Decimal (depending on input types)#
*Examples*: If the "fileSize" attribute has a value of 100, then the Expression `${fileSize:mod(12)}`
will return the value `4`.
[.function]
=== toRadix
*Description*: [.description]#Converts the Subject from a Base 10 number to a different Radix (or number base). An optional second argument can be used to indicate the minimum number of characters to be used. If the converted value has fewer than this number of characters, the number will be padded with leading zeroes. If a decimal is passed as the subject, it will first be converted to a whole number and then processed.#
*Subject Type*: [.subject]#Number#
*Arguments*:
- [.argName]#_Desired Base_# : [.argDesc]#A Number between 2 and 36 (inclusive)#
- [.argName]#_Padding_# : [.argDesc]#Optional argument that specifies the minimum number of characters in the converted output#
*Return Type*: [.returnType]#String#
*Examples*: If the "fileSize" attributes has a value of 1024, then the following Expressions will yield
the following results:
.toRadix Examples
|=======================================================================================
| Expression | Value
| `${fileSize:toRadix(10)}` | `1024`
| `${fileSize:toRadix(10, 1)}` | `1024`
| `${fileSize:toRadix(10, 8)}` | `00001024`
| `${fileSize:toRadix(16)}` | `400`
| `${fileSize:toRadix(16, 8)}` | `00000400`
| `${fileSize:toRadix(2)}` | `10000000000`
| `${fileSize:toRadix(2, 16)}` | `0000010000000000`
|=======================================================================================
[.function]
=== fromRadix
*Description*: [.description]#Converts the Subject from a specified Radix (or number base) to a base ten whole number. The subject will converted as is, without interpretation, and all characters must be valid for the base being converted from. For example converting "0xFF" from hex will not work due to "x" being a invalid hex character. If a decimal is passed as the subject, it will first be converted to a whole number and then processed.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_Subject Base_# : [.argDesc]#A Number between 2 and 36 (inclusive)#
*Return Type*: [.returnType]#Number#
*Examples*: If the "fileSize" attributes has a value of 1234A, then the following Expressions will yield
the following results:
.toRadix Examples
|=======================================================================================
| Expression | Value
| `${fileSize:fromRadix(11)}` | `17720`
| `${fileSize:fromRadix(16)}` | `74570`
| `${fileSize:fromRadix(20)}` | `177290`
|=======================================================================================
[.function]
=== random
*Description*: [.description]#Returns a random whole number ( 0 to 2^63 - 1) using an insecure random number generator.#
*Subject Type*: [.subjectless]#No subject#
*Arguments*: No arguments
*Return Type*: [.returnType]#Number#
*Examples*: `${random():mod(10):plus(1)}` returns random number between 1 and 10 inclusive.
[.function]
=== math
*Description*: [.description]#ADVANCED FEATURE. This expression is designed to be used by advanced users only. It utilizes Java Reflection to run arbitrary java.lang.Math static methods. The exact API will depend on the version of Java you are running. The Java 8 API can be found here: link:https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html[https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html^]
+
In order to run the correct method, the parameter types must be correct. The Expression Language "Number" (whole number) type is interpreted as a Java "long". The "Decimal" type is interpreted as a Java "double". Running the desired method may require calling "toNumber()" or "toDecimal()" in order to "cast" the value to the desired type. This also is important to remember when cascading "math()" calls since the return type depends on the method that was run.#
*Subject Type*: [.subject .subjectless]#Subjectless, Number or Decimal (depending on the desired method to run)#
*Arguments*:
- [.argName]#_Method_# : [.argDesc]#The name of the Java Math method to run#
- [.argName]#_Optional Argument_# : [.argDesc]#Optional argument that acts as the second parameter to the method.#
*Return Type*: [.returnType]#Number or Decimal (depending on method run)#
*Examples*:
- ${math("random")} runs Math.random().
- ${literal(2):toDecimal:math("pow", 2.5)} runs Math.pow(2D,2.5D).
- ${literal(64):toDouble():math("cbrt"):toNumber():math("max", 5)} runs Math.max((Double.valueOf(Math.cbrt(64D))).longValue(), 5L). Note that the toDecimal() is needed because "cbrt" takes a "double" as input and the "64" will get interpreted as a long. The "toDecimal()" call is necessary to correctly call the method. that the "toNumber()" call is necessary because "cbrt" returns a double and the "max" method is must have parameters of the same type and "5" is interpreted as a long.
- ${literal(5.4):math("scalb", 2)} runs Math.scalb(5.4, 2). This example is important because NiFi EL treats all whole numbers as "longs" and there is no concept of an "int". "scalb" takes a second parameter of an "int" and it is not overloaded to accept longs so it could not be run without special type handling. In the instance where the Java method cannot be found using parameters of type "double" and "long" the "math()" EL function will attempt to find a Java method with the same name but parameters of "double" and "int".
- ${first:toDecimal():math("pow", ${second:toDecimal()})} where attributes evaluate to "first" = 2.5 and "second" = 2. This example runs Math.pow(2.5D, 2D). The explicit calls to toDecimal() are important because of the dynamic nature of EL. When creating the flow, the user is unaware if the expression language values will be able to be interpreted as a whole number or not. In this example without the explicit calls "toDecimal" the "math" function would attempt to run a Java method "pow" with types "double" and "long" (which doesn't exist).
[[dates]]
== Date Manipulation
[[format]]
[.function]
=== format
*Description*: [.description]#Formats a number as a date/time according to the format specified by the argument. The argument
must be a String that is a valid Java SimpleDateFormat format. The Subject is expected to be a Number that
represents the number of milliseconds since Midnight GMT on January 1, 1970. The number will be evaluated using the local
time zone unless specified in the second optional argument.#
*Subject Type*: [.subject]#Number#
*Arguments*:
- [.argName]#_format_# : [.argDesc]#The format to use in the Java SimpleDateFormat syntax#
- [.argName]#_time zone_# : [.argDesc]#Optional argument that specifies the time zone to use (in the Java TimeZone syntax)#
*Return Type*: [.returnType]#String#
*Examples*: If the attribute "time" has the value "1420058163264", then the following Expressions will yield
the following results:
.format Examples
|============================================================================
| Expression | Value
| `${time:format("yyyy/MM/dd HH:mm:ss.SSS\'Z'", "GMT")}` | `2014/12/31 20:36:03.264Z`
| `${time:format("yyyy/MM/dd HH:mm:ss.SSS\'Z'", "America/Los_Angeles")}` | `2014/12/31 12:36:03.264Z`
| `${time:format("yyyy/MM/dd HH:mm:ss.SSS\'Z'", "Asia/Tokyo")}` | `2015/01/01 05:36:03.264Z`
| `${time:format("yyyy/MM/dd", "GMT")}` | `2014/12/31`
| `${time:format("HH:mm:ss.SSS\'Z'", "GMT")}` | `20:36:03.264Z`
| `${time:format("yyyy", "GMT")}` | `2014`
|============================================================================
[.function]
=== toDate
*Description*: [.description]#Converts a String into a Date data type, based on the format specified by the argument. The argument
must be a String that is a valid Java SimpleDateFormat syntax. The Subject is expected to be a String that is formatted
according the argument. The date will be evaluated using the local time zone unless specified in the second optional argument.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_format_# : [.argDesc]#The current format to use when parsing the Subject, in the Java SimpleDateFormat syntax.#
- [.argName]#_time zone_# : [.argDesc]#Optional argument that specifies the time zone to use when parsing the Subject, in the Java TimeZone syntax.#
*Return Type*: [.returnType]#Date#
*Examples*: If the attribute "year" has the value "2014" and the attribute "time" has the value "2014/12/31 15:36:03.264Z",
then the Expression `${year:toDate('yyyy', 'GMT')}` will return a Date data type with a value representing Midnight GMT on
January 1, 2014. The Expression `${time:toDate("yyyy/MM/dd HH:mm:ss.SSS'Z'", "GMT")}` will result in a Date data type for
15:36:03.264 GMT on December 31, 2014.
Often, this function is used in conjunction with the <<format>> function to change the format of a date/time. For example,
if the attribute "date" has the value "12-24-2014" and we want to change the format to "2014/12/24", we can do so by
chaining together the two functions: `${date:toDate('MM-dd-yyyy'):format('yyyy/MM/dd')}`.
[.function]
=== now
*Description*: [.description]#Returns the current date and time as a Date data type object.#
*Subject Type*: [.subjectless]#No Subject#
*Arguments*: No arguments
*Return Type*: [.returnType]#Date#
*Examples*: We can get the current date and time as a Date data type by using the `now` function: `${now()}`. As an example,
on Wednesday December 31st 2014 at 36 minutes after 3pm and 36.123 seconds EST `${now()}` would be evaluated to be a
Date type representing that time. Since whole Expression Language expressions can only return Strings it would formatted as
`Wed Dec 31 15:36:03 EST 2014` when the expression completes.
Utilizing the <<toNumber>> method, `now` can provide the current date and time as the number of milliseconds since
Midnight GMT on January 1, 1970. For instance, if instead of executing `${now()}` in the previous example `${now():toNumber()}`
was run then it would output `1453843201123`. This method provides millisecond-level precision and provides the ability to
manipulate the value.
.now Examples
|==================================================================================================================
| Expression | Value
| `${now()}` | A Date type representing the current date and time to the nearest millisecond
| `${now():toNumber()}` | The number of milliseconds since midnight GMT Jan 1, 1970 (`1453843201123`, for example)
| `${now():toNumber():minus(86400000)` | A number presenting the time 24 hours ago
| `${now():format('yyyy')}` | The current year
| `${now():toNumber():minus(86400000):format('E')}` | The day of the week that was yesterday,
as a 3-letter abbreviation (For example, `Wed`)
|==================================================================================================================
[[type_cast]]
== Type Coercion
[.function]
=== toString
*Description*: [.description]#Coerces the Subject into a String#
*Subject Type*: [.subject]#Any type#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: The Expression `${fileSize:toNumber():toString()}` converts the value of "fileSize" attribute to a number and
back to a String.
[.function]
=== toNumber
*Description*: [.description]#Coerces the Subject into a Number#
*Subject Type*: [.subject]#String, Decimal, or Date#
*Arguments*: No arguments
*Return Type*: [.returnType]#Number#
*Examples*: The Expression `${fileSize:toNumber()}` converts the attribute value of "fileSize" to a number.
[.function]
=== toDecimal
*Description*: [.description]#Coerces the Subject into a Decimal#
*Subject Type*: [.subject]#String, Whole Number or Date#
*Arguments*: No arguments
*Return Type*: [.returnType]#Decimal#
*Examples*: The Expression `${fileSize:toDecimal()}` converts the attribute value of "fileSize" to a decimal.
[[subjectless]]
== Subjectless Functions
While the majority of functions in the Expression Language are called by using the syntax
`${attributeName:function()}`, there exist a few functions that are not expected to have subjects.
In this case, the attribute name is not present. For example, the IP address of the machine can
be obtained by using the Expression `${ip()}`. All of the functions in this section are to be called
without a subject. Attempting to call a subjectless function and provide it a subject will result in
an error when validating the function.
[.function]
=== ip
*Description*: [.description]#Returns the IP address of the machine.#
*Subject Type*: [.subjectless]#No subject#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: The IP address of the machine can be obtained by using the Expression `${ip()}`.
[.function]
=== hostname
*Description*: [.description]#Returns the Hostname of the machine. An optional argument of type Boolean can be provided
to specify whether or not the Fully Qualified Domain Name should be used. If `false`, or not specified,
the hostname will not be fully qualified. If the argument is `true` but the fully qualified hostname
cannot be resolved, the simple hostname will be returned.#
*Subject Type*: [.subjectless]#No subject#
*Arguments*:
- [.argName]#_Fully Qualified_# : [.argDesc]#Optional parameter that specifies whether or not the hostname should be
fully qualified. If not specified, defaults to false.#
*Return Type*: [.returnType]#String#
*Examples*: The fully qualified hostname of the machine can be obtained by using the Expression `${hostname(true)}`,
while the simple hostname can be obtained by using either `${hostname(false)}` or simply `${hostname()}`.
[.function]
=== UUID
*Description*: [.description]#Returns a randomly generated type 4 UUID.#
*Subject Type*: [.subjectless]#No Subject#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: `${UUID()}` returns a value similar to "de305d54-75b4-431b-adb2-eb6b9e546013"
*See Also*: [.seeAlso]#<<uuid3,UUID3()>>, <<uuid5,UUID5()>>#
[.function]
=== nextInt
*Description*: [.description]#Returns a one-up value (starting at 0) and increasing over the lifetime of the running instance of NiFi.
This value is not persisted across restarts and is not guaranteed to be unique across a cluster.
This value is considered "one-up" in that if called multiple times across the NiFi instance, the values will be sequential.
However, this counter is shared across all NiFi components, so calling this function multiple times from one Processor will
not guarantee sequential values within the context of a particular Processor.#
*Subject Type*: [.subjectless]#No Subject#
*Arguments*: No arguments
*Return Type*: [.returnType]#Number#
*Examples*: If the previous value returned by `nextInt` was `5`, the Expression `${nextInt():divide(2)}` obtains the next available
integer (6) and divides the result by 2, returning a value of `3`.
[.function]
=== literal
*Description*: [.description]#Returns its argument as a literal String value. This is useful in order to treat a string or a number
at the beginning of an Expression as an actual value, rather than treating it as an attribute name. Additionally, it
can be used when the argument is an embedded Expression that we would then like to evaluate additional functions against.#
*Subject Type*: [.subjectless]#No Subject#
*Arguments*:
- [.argName]#_value_# : [.argDesc]#The value to be treated as a literal string, number, or boolean value.#
*Return Type*: [.returnType]#String#
*Examples*: `${literal(2):gt(1)}` returns `true`
`${literal( ${allMatchingAttributes('a.*'):count()} ):gt(3)}` returns true if there are more than 3 attributes whose
names begin with the letter `a`.
[.function]
=== getStateValue
*Description*: [.description]#Access a processor's state values by passing in the String key and getting the value back as a String. This
is a special Expression Language function that only works with processors that explicitly allow EL to query state. Currently only UpdateAttribute
does.#
*Subject Type*: [.subjectless]#No Subject#
*Arguments*:
- [.String]#_Key_# : [.argDesc]#The key to use when accessing the state map.#
*Return Type*: [.returnType]#String#
*Examples*: UpdateAttribute processor has stored the key "count" with value "20" in state. '${getStateValue("count")}` returns `20`.
[.function]
=== thread
*Description*: [.description]#Returns the name of the thread used by the processor when evaluating the Expression. This can be useful
when using a processor with multiple concurrent tasks and where some data uniqueness is required.#
*Subject Type*: [.subjectless]#No Subject#
*Arguments*: No arguments
*Return Type*: [.returnType]#String#
*Examples*: `${thread()}` could return something like `Timer-Driven Process Thread-4`.
[[multi]]
== Evaluating Multiple Attributes
When it becomes necessary to evaluate the same conditions against multiple attributes, this can be accomplished by means of the
`and` and `or` functions. However, this quickly becomes tedious, error-prone, and difficult to maintain. For this reason, NiFi
provides several functions for evaluating the same conditions against groups of attributes at the same time.
[.function]
=== anyAttribute
*Description*: [.description]#Checks to see if any of the given attributes, match the given condition. This function has no subject and takes one or more
arguments that are the names of attributes to which the remainder of the Expression is to be applied. If any of the attributes specified,
when evaluated against the rest of the Expression, returns a value of `true`, then this function will return `true`. Otherwise, this function
will return `false`.#
*Subject Type*: [.subjectless]#No Subject#
*Arguments*:
- [.argName]#_Attribute Names_# : [.argDesc]#One or more attribute names to evaluate#
*Return Type*: [.returnType]#Boolean#
*Examples*: Given that the "abc" attribute contains the value "hello world", "xyz" contains "good bye world",
and "filename" contains "file.txt" consider the following examples:
.anyAttribute Examples
|=======================================================================
| Expression | Value
| `${anyAttribute("abc", "xyz"):contains("bye")}` | `true`
| `${anyAttribute("filename","xyz"):toUpper():contains("e")}` | `false`
|=======================================================================
[.function]
=== allAttributes
*Description*: [.description]#Checks to see if all of the given attributes match the given condition. This function has no subject and takes one or more
arguments that are the names of attributes to which the remainder of the Expression is to be applied. If all of the attributes specified,
when evaluated against the rest of the Expression, returns a value of `true`, then this function will return `true`. Otherwise, this function
will return `false`.#
*Subject Type*: [.subjectless]#No Subject#
*Arguments*:
- [.argName]#_Attribute Names_# : [.argDesc]#One or more attribute names to evaluate#
*Return Type*: [.returnType]#Boolean#
*Examples*: Given that the "abc" attribute contains the value "hello world", "xyz" contains "good bye world",
and "filename" contains "file.txt" consider the following examples:
.allAttributes Example
|=============================================================================
| Expression | Value
| `${allAttributes("abc", "xyz"):contains("world")}` | `true`
| `${allAttributes("abc", "filename","xyz"):toUpper():contains("e")}` | `false`
|=============================================================================
[.function]
=== anyMatchingAttribute
*Description*: [.description]#Checks to see if any of the given attributes, match the given condition. This function has no subject and takes one or more
arguments that are Regular Expressions to match against attribute names. Any attribute whose name matches one of the supplied
Regular Expressions will be evaluated against the rest of the Expression. If any of the attributes specified,
when evaluated against the rest of the Expression, returns a value of `true`, then this function will return `true`. Otherwise, this function
will return `false`.#
*Subject Type*: [.subjectless]#No Subject#
*Arguments*:
- [.argName]#_Regex_# : [.argDesc]#One or more Regular Expressions (in the Java Pattern syntax) to evaluate against attribute names#
*Return Type*: [.returnType]#Boolean#
*Examples*: Given that the "abc" attribute contains the value "hello world", "xyz" contains "good bye world",
and "filename" contains "file.txt" consider the following examples:
.anyMatchingAttribute Example
|==============================================================
| Expression | Value
| `${anyMatchingAttribute("[ax].*"):contains('bye')}` | `true`
| `${anyMatchingAttribute(".*"):isNull()}` | `false`
|==============================================================
[.function]
=== allMatchingAttributes
*Description*: [.description]#Checks to see if any of the given attributes, match the given condition. This function has no subject and takes one or more
arguments that are Regular Expressions to match against attribute names. Any attribute whose name matches one of the supplied
Regular Expressions will be evaluated against the rest of the Expression. If all of the attributes specified,
when evaluated against the rest of the Expression, return a value of `true`, then this function will return `true`. Otherwise, this function
will return `false`.#
*Subject Type*: [.subjectless]#No Subject#
- [.argName]#_Regex_# : [.argDesc]#One or more Regular Expressions (in the Java Pattern syntax) to evaluate against attribute names#
*Return Type*: [.returnType]#Boolean#
*Examples*: Given that the "abc" attribute contains the value "hello world", "xyz" contains "good bye world",
and "filename" contains "file.txt" consider the following examples:
.anyMatchingAttributes Examples
|==============================================================
| Expression | Value
| `${allMatchingAttributes("[ax].*"):contains("world")}` | `true`
| `${allMatchingAttributes(".*"):isNull()}` | `false`
| `${allMatchingAttributes("f.*"):count()}` | `1`
|==============================================================
[.function]
=== anyDelineatedValue
*Description*: [.description]#Splits a String apart according to a delimiter that is provided, and then evaluates each of the values against
the rest of the Expression. If the Expression, when evaluated against any of the individual values, returns `true`, this
function returns `true`. Otherwise, the function returns `false`.#
*Subject Type*: [.subjectless]#No Subject#
*Arguments*:
- [.argName]#_Delineated Value_# : [.argDesc]#The value that is delineated. This is generally an embedded Expression,
though it does not have to be.#
- [.argName]#_Delimiter_# : [.argDesc]#The value to use to split apart the _delineatedValue_ argument.#
*Return Type*: [.returnType]#Boolean#
*Examples*: Given that the "number_list" attribute contains the value "1,2,3,4,5", and the "word_list" attribute contains the value "the,and,or,not",
consider the following examples:
.anyDelineatedValue Examples
|===============================================================================
| Expression | Value
| `${anyDelineatedValue("${number_list}", ","):contains("5")}` | `true`
| `${anyDelineatedValue("this that and", ","):equals("${word_list}")}` | `false`
|===============================================================================
[.function]
=== allDelineatedValues
*Description*: [.description]#Splits a String apart according to a delimiter that is provided, and then evaluates each of the values against
the rest of the Expression. If the Expression, when evaluated against all of the individual values, returns `true` in each
case, then this function returns `true`. Otherwise, the function returns `false`.#
*Subject Type*: [.subjectless]#No Subject#
*Arguments*:
- [.argName]#_Delineated Value_# : [.argDesc]#The value that is delineated. This is generally
an embedded Expression, though it does not have to be.#
- [.argName]#_Delimiter_# : [.argDesc]#The value to use to split apart the _delineatedValue_ argument.#
*Return Type*: [.returnType]#Boolean#
*Examples*: Given that the "number_list" attribute contains the value "1,2,3,4,5", and the "word_list" attribute contains the value "those,known,or,not",
consider the following examples:
.allDelineatedValues Examples
|===============================================================================
| Expression | Value
| `${allDelineatedValues("${word_list}", ","):contains("o")}` | `true`
| `${allDelineatedValues("${number_list}", ","):count()}` | `4`
| `${allDelineatedValues("${number_list}", ","):matches("[0-9]+")}` | `true`
| `${allDelineatedValues("${word_list}", ","):matches('e')}` | `false`
|===============================================================================
[.function]
=== join
*Description*: [.description]#Aggregate function that concatenates multiple values with the specified delimiter. This function
may be used only in conjunction with the `allAttributes`, `allMatchingAttributes`, and `allDelineatedValues`
functions.#
*Subject Type*: [.subject]#String#
*Arguments*:
- [.argName]#_Delimiter_# : [.argDesc]#The String delimiter to use when joining values#
*Return Type*: [.returnType]#String#
*Examples*: Given that the "abc" attribute contains the value "hello world", "xyz" contains "good bye world",
and "filename" contains "file.txt" consider the following examples:
.join Examples
|=======================================================================================
| Expression | Value
| `${allMatchingAttributes("[ax].*"):substringBefore(" "):join("-")}` | `hello-good`
| `${allAttributes("abc", "xyz"):join(" now")}` | `hello world nowgood bye world now`
|=======================================================================================
[.function]
=== count
*Description*: [.description]#Aggregate function that counts the number of non-null, non-false values returned by the
`allAttributes`, `allMatchingAttributes`, and `allDelineatedValues`. This function
may be used only in conjunction with the `allAttributes`, `allMatchingAttributes`, and `allDelineatedValues`
functions.#
*Subject Type*: [.subject]#Any#
*Arguments*: No arguments
*Return Type*: [.returnType]#Number#
*Examples*: Given that the "abc" attribute contains the value "hello world", "xyz" contains "good bye world",
and "number_list" contains "1,2,3,4,5" consider the following examples:
.count Examples
|===========================================================================
| Expression | Value
| `${allMatchingAttributes("[ax].*"):substringBefore(" "):count()}` | `2`
| `${allAttributes("abc", "xyz"):contains("world"):count()}` | `1`
| `${allDelineatedValues(${number_list}, ","):count()}` | `5`
| `${allAttributes("abc", "non-existent-attr", "xyz"):count()}` | `2`
| `${allMatchingAttributes(".*"):length():gt(10):count()}` | `2`
|===========================================================================