blob: 54ef759b19bdf35cdb79d6aa384823d28427a2b5 [file] [log] [blame]
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>MoveHDFS</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">MoveHDFS</h1><h2>Description: </h2><p>Rename existing files or a directory of files (non-recursive) on Hadoop Distributed File System (HDFS).</p><h3>Tags: </h3><p>hadoop, HCFS, HDFS, put, move, filesystem, moveHDFS</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Display Name</th><th>API Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name">Hadoop Configuration Resources</td><td>Hadoop Configuration Resources</td><td></td><td id="allowable-values"></td><td id="description">A file or comma separated list of files which contains the Hadoop file system configuration. Without this, Hadoop will search the classpath for a 'core-site.xml' and 'hdfs-site.xml' file or will revert to a default configuration. To use swebhdfs, see 'Additional Details' section of PutHDFS's documentation.<br/><br/><strong>This property expects a comma-separated list of file resources.</strong><br/><br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Kerberos Credentials Service</td><td>kerberos-credentials-service</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>KerberosCredentialsService<br/><strong>Implementation: </strong><a href="../../../nifi-kerberos-credentials-service-nar/1.19.1/org.apache.nifi.kerberos.KeytabCredentialsService/index.html">KeytabCredentialsService</a></td><td id="description">Specifies the Kerberos Credentials Controller Service that should be used for authenticating with Kerberos</td></tr><tr><td id="name">Kerberos User Service</td><td>kerberos-user-service</td><td></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>KerberosUserService<br/><strong>Implementations: </strong><a href="../../../nifi-kerberos-user-service-nar/1.19.1/org.apache.nifi.kerberos.KerberosPasswordUserService/index.html">KerberosPasswordUserService</a><br/><a href="../../../nifi-kerberos-user-service-nar/1.19.1/org.apache.nifi.kerberos.KerberosKeytabUserService/index.html">KerberosKeytabUserService</a><br/><a href="../../../nifi-kerberos-user-service-nar/1.19.1/org.apache.nifi.kerberos.KerberosTicketCacheUserService/index.html">KerberosTicketCacheUserService</a></td><td id="description">Specifies the Kerberos User Controller Service that should be used for authenticating with Kerberos</td></tr><tr><td id="name">Kerberos Principal</td><td>Kerberos Principal</td><td></td><td id="allowable-values"></td><td id="description">Kerberos principal to authenticate as. Requires nifi.kerberos.krb5.file to be set in your nifi.properties<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Kerberos Keytab</td><td>Kerberos Keytab</td><td></td><td id="allowable-values"></td><td id="description">Kerberos keytab associated with the principal. Requires nifi.kerberos.krb5.file to be set in your nifi.properties<br/><br/><strong>This property requires exactly one file to be provided..</strong><br/><br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Kerberos Password</td><td>Kerberos Password</td><td></td><td id="allowable-values"></td><td id="description">Kerberos password associated with the principal.<br/><strong>Sensitive Property: true</strong></td></tr><tr><td id="name">Kerberos Relogin Period</td><td>Kerberos Relogin Period</td><td id="default-value">4 hours</td><td id="allowable-values"></td><td id="description">Period of time which should pass before attempting a kerberos relogin.
This property has been deprecated, and has no effect on processing. Relogins now occur automatically.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Additional Classpath Resources</td><td>Additional Classpath Resources</td><td></td><td id="allowable-values"></td><td id="description">A comma-separated list of paths to files and/or directories that will be added to the classpath and used for loading native libraries. When specifying a directory, all files with in the directory will be added to the classpath, but further sub-directories will not be included.<br/><br/><strong>This property expects a comma-separated list of resources. Each of the resources may be of any of the following types: directory, file.</strong><br/></td></tr><tr><td id="name"><strong>Conflict Resolution Strategy</strong></td><td>Conflict Resolution Strategy</td><td id="default-value">fail</td><td id="allowable-values"><ul><li>replace <img src="../../../../../html/images/iconInfo.png" alt="Replaces the existing file if any." title="Replaces the existing file if any."></img></li><li>ignore <img src="../../../../../html/images/iconInfo.png" alt="Failed rename operation stops processing and routes to success." title="Failed rename operation stops processing and routes to success."></img></li><li>fail <img src="../../../../../html/images/iconInfo.png" alt="Failing to rename a file routes to failure." title="Failing to rename a file routes to failure."></img></li></ul></td><td id="description">Indicates what should happen when a file with the same name already exists in the output directory</td></tr><tr><td id="name"><strong>Input Directory or File</strong></td><td>Input Directory or File</td><td id="default-value">${path}</td><td id="allowable-values"></td><td id="description">The HDFS directory from which files should be read, or a single file to read.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Output Directory</strong></td><td>Output Directory</td><td></td><td id="allowable-values"></td><td id="description">The HDFS directory where the files will be moved to<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>HDFS Operation</strong></td><td>HDFS Operation</td><td id="default-value">move</td><td id="allowable-values"><ul><li>move</li><li>copy</li></ul></td><td id="description">The operation that will be performed on the source file</td></tr><tr><td id="name">File Filter Regex</td><td>File Filter Regex</td><td></td><td id="allowable-values"></td><td id="description">A Java Regular Expression for filtering Filenames; if a filter is supplied then only files whose names match that Regular Expression will be fetched, otherwise all files will be fetched</td></tr><tr><td id="name"><strong>Ignore Dotted Files</strong></td><td>Ignore Dotted Files</td><td id="default-value">true</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">If true, files whose names begin with a dot (".") will be ignored</td></tr><tr><td id="name">Remote Owner</td><td>Remote Owner</td><td></td><td id="allowable-values"></td><td id="description">Changes the owner of the HDFS file to this value after it is written. This only works if NiFi is running as a user that has HDFS super user privilege to change owner</td></tr><tr><td id="name">Remote Group</td><td>Remote Group</td><td></td><td id="allowable-values"></td><td id="description">Changes the group of the HDFS file to this value after it is written. This only works if NiFi is running as a user that has HDFS super user privilege to change group</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>Files that have been successfully renamed on HDFS are transferred to this relationship</td></tr><tr><td>failure</td><td>Files that could not be renamed on HDFS are transferred to this relationship</td></tr></table><h3>Reads Attributes: </h3><table id="reads-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename</td><td>The name of the file written to HDFS comes from the value of this attribute.</td></tr></table><h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename</td><td>The name of the file written to HDFS is stored in this attribute.</td></tr><tr><td>absolute.hdfs.path</td><td>The absolute path to the file on HDFS is stored in this attribute.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3><table id="restrictions"><tr><th>Required Permission</th><th>Explanation</th></tr><tr><td>read distributed filesystem</td><td>Provides operator the ability to retrieve any file that NiFi has access to in HDFS or the local filesystem.</td></tr><tr><td>write distributed filesystem</td><td>Provides operator the ability to delete any file that NiFi has access to in HDFS or the local filesystem.</td></tr></table><h3>Input requirement: </h3>This component allows an incoming relationship.<h3>System Resource Considerations:</h3>None specified.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.hadoop.PutHDFS/index.html">PutHDFS</a>, <a href="../org.apache.nifi.processors.hadoop.GetHDFS/index.html">GetHDFS</a></p></body></html>