components/camel-flatpack/src/main/docs/flatpack-component.adoc

= Flatpack Component
:doctitle: Flatpack
:shortname: flatpack
:artifactid: camel-flatpack
:description: Parse fixed width and delimited files using the FlatPack library.
:since: 1.4
:supportlevel: Stable
:tabs-sync-option:
:component-header: Both producer and consumer are supported

*Since Camel {since}*

*{component-header}*

The Flatpack component supports fixed width and delimited file parsing
via the http://flatpack.sourceforge.net[FlatPack library]. +
 *Notice:* This component only supports consuming from flatpack files to
Object model. You can not (yet) write from Object model to flatpack
format.


== URI format

----
flatpack:[delim|fixed]:flatPackConfig.pzmap.xml[?options]
----

Or for a delimited file handler with no configuration file just use

----
flatpack:someName[?options]
----


// component options: START
include::partial$component-configure-options.adoc[]
include::partial$component-endpoint-options.adoc[]
include::partial$component-endpoint-headers.adoc[]
// component options: END


== Usage

* `flatpack:fixed:foo.pzmap.xml` creates a fixed-width endpoint using
the `foo.pzmap.xml` file configuration.
* `flatpack:delim:bar.pzmap.xml` creates a delimited endpoint using the
`bar.pzmap.xml` file configuration.
* `flatpack:foo` creates a delimited endpoint called `foo` with no file
configuration.

=== Message Body

The component delivers the data in the IN message as a
`org.apache.camel.component.flatpack.DataSetList` object that has
converters for `java.util.Map` or `java.util.List`. +
 Usually you want the `Map` if you process one row at a time
(`splitRows=true`). Use `List` for the entire content
(`splitRows=false`), where each element in the list is a `Map`. +
 Each `Map` contains the key for the column name and its corresponding
value.

For example, to get the firstname from the sample below:

._Java-only: accessing row data from the exchange body_
[source,java]
----
  Map row = exchange.getIn().getBody(Map.class);
  String firstName = row.get("FIRSTNAME");
----

However, you can also always get it as a `List` (even for
`splitRows=true`). The same example:

._Java-only: accessing row data as a List from the exchange body_
[source,java]
----
  List data = exchange.getIn().getBody(List.class);
  Map row = (Map)data.get(0);
  String firstName = row.get("FIRSTNAME");
----

=== Header and Trailer records

The header and trailer notions in Flatpack are supported. However, you
*must* use fixed record IDs:

* `header` for the header record (must be lowercase)
* `trailer` for the trailer record (must be lowercase)

The example below illustrates this fact that we have a header and a
trailer. You can omit one or both of them if not needed.

[source,xml]
----
    <RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
        <COLUMN name="INDICATOR" length="3"/>
        <COLUMN name="DATE" length="8"/>
    </RECORD>

    <COLUMN name="FIRSTNAME" length="35" />
    <COLUMN name="LASTNAME" length="35" />
    <COLUMN name="ADDRESS" length="100" />
    <COLUMN name="CITY" length="100" />
    <COLUMN name="STATE" length="2" />
    <COLUMN name="ZIP" length="5" />

    <RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
        <COLUMN name="INDICATOR" length="3"/>
        <COLUMN name="STATUS" length="7"/>
    </RECORD>
----

=== Using as an Endpoint

A common use case is sending a file to this endpoint for further
processing in a separate route. For example:

[tabs]
====
Java::
+
[source,java]
----
from("file://someDirectory")
    .to("flatpack:foo");

from("flatpack:foo")
    ...
----

XML::
+
[source,xml]
----
<route>
    <from uri="file://someDirectory"/>
    <to uri="flatpack:foo"/>
</route>

<route>
    <from uri="flatpack:foo"/>
    ...
</route>
----

YAML::
+
[source,yaml]
----
- route:
    from:
      uri: file://someDirectory
      steps:
        - to:
            uri: flatpack:foo

- route:
    from:
      uri: flatpack:foo
      steps:
        - log:
            message: "${body}"
----
====

You can also convert the payload of each message created to a `Map` for
easy Bean Integration

=== Flatpack DataFormat

The xref:flatpack-component.adoc[Flatpack] component ships with the Flatpack data
format that can be used to format between fixed width or delimited text
messages to a `List` of rows as `Map`.

* marshal = from `List<Map<String, Object>>` to `OutputStream` (can be
converted to `String`)
* unmarshal = from `java.io.InputStream` (such as a `File` or `String`)
to a `java.util.List` as an
`org.apache.camel.component.flatpack.DataSetList` instance. +
 The result of the operation will contain all the data. If you need to
process each row one by one you can split the exchange, using
Splitter.

*Notice:* The Flatpack library does currently not support header and
trailers for the marshal operation.

==== Options

The data format has the following options:

[width="100%",cols="10%,10%,80%",options="header",]
|=======================================================================
|Option |Default |Description

|`definition` |`null` |The flatpack pzmap configuration file. Can be omitted in simpler
situations, but its preferred to use the pzmap.

|`fixed` |`false` |Delimited or fixed.

|`ignoreFirstRecord` |`true` |Whether the first line is ignored for delimited files (for the column
headers).

|`textQualifier` |`"` |If the text is qualified with a char such as `"`.

|`delimiter` |`,` |The delimiter char (could be `;` `,` or similar)

|`parserFactory` |`null` |Uses the default Flatpack parser factory.

|`allowShortLines` |`false` |Allows for lines to be shorter than
expected and ignores the extra characters.

|`ignoreExtraColumns` |`false` |Allows for lines to be longer than
expected and ignores the extra characters.
|=======================================================================

=== Using the data format

To use the data format, instantiate an instance and invoke the
marshal or unmarshal operation in the route builder:

._Java-only: programmatic FlatpackDataFormat configuration and unmarshal_
[source,java]
----
  FlatpackDataFormat fp = new FlatpackDataFormat();
  fp.setDefinition(new ClassPathResource("INVENTORY-Delimited.pzmap.xml"));
  ...
  from("file:order/in").unmarshal(df).to("seda:queue:neworder");
----

The sample above will read files from the `order/in` folder and
unmarshal the input using the Flatpack configuration file
`INVENTORY-Delimited.pzmap.xml` that configures the structure of the
files. The result is a `DataSetList` object we store on the SEDA queue.

._Java-only: programmatic FlatpackDataFormat configuration and marshal_
[source,java]
----
FlatpackDataFormat df = new FlatpackDataFormat();
df.setDefinition(new ClassPathResource("PEOPLE-FixedLength.pzmap.xml"));
df.setFixed(true);
df.setIgnoreFirstRecord(false);

from("seda:people").marshal(df).convertBodyTo(String.class).to("jms:queue:people");
----

In the code above we marshal the data from an Object representation as a
`List` of rows as `Maps`. The rows as `Map` contains the column name as
the key, and the corresponding value. This structure can be created
in Java code from e.g., a processor. We marshal the data according to the
Flatpack format and convert the result as a `String` object and store it
on a JMS queue.

=== Dependencies

To use Flatpack in your camel routes, you need to add a dependency on
*camel-flatpack* which implements this data format.

If you use maven, you could add the following to your `pom.xml`,
substituting the version number for the latest release.

[source,xml]
----
<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-flatpack</artifactId>
  <version>x.x.x</version>
</dependency>
----