This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance.
The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell.
To execute a single command:
./bin/cli.sh <command> <args>
To launch the interactive shell:
./bin/cli.sh
Most commands will require specifying a baseUrl for the NiFi or NiFi registry instance.
An example command to list the buckets in a NiFi Registry instance would be the following:
./bin/cli.sh registry list-buckets -u http://localhost:18080
In order to avoid specifying the URL (and possibly other optional arguments for TLS) on every command, you can define a properties file containing the repetitive arguments.
An example properties file for a local NiFi Registry instance would look like the following:
baseUrl=http://localhost:18080 keystore= keystoreType= keystorePasswd= keyPasswd= truststore= truststoreType= truststorePasswd= proxiedEntity=
This properties file can then be used on a command by specifying -p :
./bin/cli.sh registry list-buckets -p /path/to/local-nifi-registry.properties
You could then maintain a properties file for each environment you plan to interact with, such as dev, qa, prod.
In addition to specifying, a properties file on each command, you can setup a default properties file to be used in the event that no properties file is specified.
The default properties file is specified using the session concept, which persists to the users home directory in a file called .nifi-cli.config.
An example of setting the default property files would be following:
./bin/cli.sh session set nifi.props /path/to/local-nifi.properties ./bin/cli.sh session set nifi.reg.props /path/to/local-nifi-registry.properties
This will write the above properties into the .nifi-cli.config in the user's home directory and will allow commands to be executed without specifying a URL or properties file:
./bin/cli.sh registry list-buckets
The above command will now use the baseUrl from local-nifi-registry.properties.
The order of resolving an argument is the following:
If NiFi and NiFi Registry are secured, then commands executed from the CLI will need to make a TLS connection and authenticate as a user with permissions to perform the desired action.
Currently the CLI supports authenticating with a client certificate and an optional proxied-entity. A common scenario would be running the CLI from one of the nodes where NiFi or NiFi Registry is installed, which allows the CLI to use the same key store and trust store as the NiFi/NiFi Registry instance.
The security configuration can be specified per-command, or in one of the properties files described in the previous section.
The examples below are for NiFi Registry, but the same concept applies for NiFi commands.
Assuming we have a keystore containing the certificate for ‘CN=user1, OU=NIFI’, an example properties file would be the following:
baseUrl=https://localhost:18443 keystore=/path/to/keystore.jks keystoreType=JKS keystorePasswd=changeme keyPasswd=changeme truststore=/path/to/truststore.jks truststoreType=JKS truststorePasswd=changeme
In this example, commands will be executed as ‘CN=user1, OU=NIFI’. This user would need to be a user in NiFi Registry, and commands accessing buckets would be restricted to buckets this user has access to.
Assuming we have access to the keystore of NiFi Registry itself, and that NiFi Registry is also configured to allow Kerberos or LDAP authentication, an example properties file would be the following:
baseUrl=https://localhost:18443 keystore=/path/to/keystore.jks keystoreType=JKS keystorePasswd=changeme keyPasswd=changeme truststore=/path/to/truststore.jks truststoreType=JKS truststorePasswd=changeme proxiedEntity=user1@NIFI.COM
In this example, the certificate in keystore.jks would be for the NiFi Registry server, for example ‘CN=localhost, OU=NIFI’. This identity would need to be defined as a user in NiFi Registry and given permissions to ‘Proxy’.
‘CN=localhost, OU=NIFI’ would be proxying commands to be executed as ‘user1@NIFI.COM’.
In interactive mode the tab key can be used to perform auto-completion.
For example, typing tab at an empty prompt should display possible commands for the first argument:
#> exit help nifi registry session
Typing "nifi " and then a tab will show the sub-commands for NiFi:
#> nifi create-reg-client current-user get-root-id list-reg-clients pg-get-vars pg-import update-reg-client
Arguments that represent a path to a file, such as -p or when setting a properties file in the session, will auto-complete the path being typed:
#> session set nifi.props /tmp/ dir1/ dir2/ dir3/
All commands (except export-flow-version) support the ability to specify an --outputType argument, or -ot for short.
Currently the output type may be simple or json.
The default output type in interactive mode is simple, and the default output type in standalone mode is json.
Example of simple output for list-buckets:
#> registry list-buckets -ot simple My Bucket - 3c7b7467-0012-4d8f-a918-6aa42b6b9d39
Example of json output for list-buckets:
#> registry list-buckets -ot json
[ {
"identifier" : "3c7b7467-0012-4d8f-a918-6aa42b6b9d39",
"name" : "My Bucket",
"createdTimestamp" : 1516718733854,
"permissions" : {
"canRead" : true,
"canWrite" : true,
"canDelete" : true
},
"link" : {
"params" : {
"rel" : "self"
},
"href" : "buckets/3c7b7467-0012-4d8f-a918-6aa42b6b9d39"
}
} ]
When using the interactive CLI, a common scenario will be using an id from a previous result as the input to the next command. Back-referencing provides a shortcut for referencing a result from the previous command via a positional reference.
NOTE: Not every command produces back-references. To determine if a command supports back-referencing, check the usage.
#> registry list-buckets help Lists the buckets that the current user has access to. PRODUCES BACK-REFERENCES
A common scenario for utilizing back-references would be the following:
User starts by exploring the available buckets in a registry instance
#> registry list-buckets # Name Id Description - ------------ ------------------------------------ ----------- 1 My Bucket 3c7b7467-0012-4d8f-a918-6aa42b6b9d39 (empty) 2 Other Bucket 175fb557-43a2-4abb-871f-81a354f47bc2 (empty)
User then views the flows in one of the buckets using a back-reference to the bucket id from the previous result in position 1
#> registry list-flows -b &1 Using a positional back-reference for 'My Bucket' # Name Id Description - ------- ------------------------------------ ---------------- 1 My Flow 06acb207-d2f1-447f-85ed-9b8672fe6d30 This is my flow.
User then views the version of the flow using a back-reference to the flow id from the previous result in position 1
#> registry list-flow-versions -f &1 Using a positional back-reference for 'My Flow' Ver Date Author Message --- -------------------------- ------------------------ ------------------------------------- 1 Tue, Jan 23 2018 09:48 EST anonymous This is the first version of my flow.
User deploys version 1 of the flow using back-references to the bucket and flow id from step 2
#> nifi pg-import -b &1 -f &1 -fv 1 Using a positional back-reference for 'My Bucket' Using a positional back-reference for 'My Flow' 9bd157d4-0161-1000-b946-c1f9b1832efd
The reason step 4 was able to reference the results from step 2, is because the list-flow-versions command in step 3 does not produce back-references, so the results from step 2 are still available.
To add a NiFi command, create a new class that extends AbstractNiFiCommand:
public class MyCommand extends AbstractNiFiCommand {
public MyCommand() {
super("my-command");
}
@Override
protected void doExecute(NiFiClient client, Properties properties)
throws NiFiClientException, IOException, MissingOptionException, CommandException {
// TODO implement
}
@Override
public String getDescription() {
return "This is my new command";
}
}
Add the new command to NiFiCommandGroup:
commands.add(new MyCommand());
To add a NiFi Registry command, perform the same steps, but extend from AbstractNiFiRegistryCommand, and add the command to NiFiRegistryCommandGroup.