| //// |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| http://www.apache.org/licenses/LICENSE-2.0 |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| //// |
| :documentationPath: /pipeline/transforms/ |
| :language: en_US |
| :description: The Token Replacement transform replaces tokens in an input string or file. |
| |
| = image:transforms/icons/token.svg[Token Replacement transform Icon, role="image-doc-icon"] Token Replacement |
| |
| [%noheader,cols="3a,1a", role="table-no-borders" ] |
| |=== |
| | |
| == Description |
| |
| The Token Replacement transform replaces tokens in an input string or file. |
| |
| The transform can then output this data either to a file or a field on the stream. |
| |
| A token contains a start string, a name, and an end string. |
| |
| For example ${my_token} could be a token. |
| |
| The start string, and end string are configurable and can be any series of characters. |
| |
| When replacing tokens in a file, this transform reads the entire file. |
| It does not operate on a single line at a time. |
| When replacing tokens in a file it is a best practice to output to a file also to prevent Hop from having to read the entire file into memory. |
| |
| | |
| == Supported Engines |
| [%noheader,cols="2,1a",frame=none, role="table-supported-engines"] |
| !=== |
| !Hop Engine! image:check_mark.svg[Supported, 24] |
| !Spark! image:question_mark.svg[Maybe Supported, 24] |
| !Flink! image:question_mark.svg[Maybe Supported, 24] |
| !Dataflow! image:question_mark.svg[Maybe Supported, 24] |
| !=== |
| |=== |
| |
| == Options |
| |
| === Input Tab |
| |
| * Input Type - Where to read the field to do the token replacement from. |
| Either text, field, or file. |
| * Input Type Text |
| ** Input Text - The text to token replace |
| * Input Type Field |
| ** Input Field - The input field to token replace |
| * Input Type File |
| ** Input filename - The name of the file to token replace |
| ** Filename is in field? |
| - Is the name of the file to token replace in a field? |
| ** Input filename field - The field the name of the file is in. |
| ** Add Input filename to result? |
| - Add the input filename(s) to the result files list. |
| |
| === Output Tab |
| |
| * Output Type - Where to put the token replaced string. |
| Either field or file. |
| * Output Type Field |
| ** Output field name - The name of the field to put the token replaced string in. |
| * Output Type File |
| ** Output filename - The name of the file to write to. |
| ** Filename is in field? |
| - Is the name of the output file in a field? |
| ** Output filename field - The name of the field the output filename is in. |
| ** Append output file? |
| - If the output file already exists should it be appended to. |
| If not checked, Pentaho will overwrite the file if it exists. |
| ** Create parent folder? |
| - Should Pentaho create the parent folder? |
| ** Output format - The format of the new line delimiter for the output file. |
| ** Output encoding - The character encoding to use when writing the file. |
| ** Split every - Split the output file into a new file every n rows. |
| ** Include stepnr in filename? |
| - Should the step number be included in the output filename? |
| ** Include partition nr in filename? |
| - Should the partition number be included in the output filename? |
| ** Include date in filename? |
| - Should the curren date be included in the output filename? |
| ** Include time in filename? |
| - Should the current time be included in the output filename? |
| ** Specify date format? |
| - Do you want to specify the date format to include in the output filename? |
| ** Date time format - The date/time format to include in the output filename. |
| ** Add output filenames to result? |
| - Add the output filename(s) to the result files list. |
| |
| === Tokens Tab |
| |
| * Token start string - The string that indicates the start of a token. |
| * Token end string - The string that indicates the end of a token. |
| Everything between the token start string and the token end string is the token name. |
| * Stream name - The name of the field on the stream containing the value to replace the token with. |
| * Token name - The name of the token to replace. |
| * Get Fields button - Gets the list of input fields, and tries to map them to an Avro field by an exact name match. |