| == Splitting big XML payloads |
| |
| === Introduction |
| |
| This example shows how to deal with big XML files in Camel. |
| |
| The XPath tokenizer will load the entire XML content into memory, so |
| it's not well suited for very big XML payloads. + |
| Instead you can use the StAX or XML tokenizers to efficiently iterate |
| the XML payload in a streamed fashion. + |
| For more information please read the |
| http://camel.apache.org/splitter.html[official documentation]. |
| |
| There are 2 tests: |
| |
| [arabic] |
| . `+StaxTokenizerTest+` : requires using JAXB and process messages using |
| a SAX ContentHandler |
| . `+XmlTokenizerTest+` : easier to use but can't handle complex XML |
| structures (i.e. nested naming clash) |
| |
| The test XML contains a simple collection of records. |
| |
| [source,xml] |
| ---- |
| <?xml version="1.0" encoding="UTF-8"?> |
| <records xmlns="http://fvaleri.it/records"> |
| <record> |
| <key>0</key> |
| <value>The quick brown fox jumps over the lazy dog</value> |
| </record> |
| </records> |
| ---- |
| |
| You can customize numOfRecords and maxWaitTime to do performance tests |
| with different payloads. + |
| Max JVM heap is restricted to 20 MB to show that it works with a very |
| limited amount of memory (see `+pom.xml+`). |
| |
| There are also a number of optional runtime settings: - no cache enabled |
| - no parallel processing - no mock endpoints with in-memory exchange |
| store - enabled Throughput Logging for DEBUG level - disabled JMX |
| instrumentation |
| |
| === Build and run |
| |
| The test XML file is built once beforehand using `+@BeforeClass+`. |
| |
| [source,sh] |
| ---- |
| mvn clean test -DskipTests=false |
| ---- |
| |
| === Test results |
| |
| Tested on MacBook Pro 2,8 GHz Intel Core i7; 16 GB 2133 MHz LPDDR3; Java |
| 1.8.0_181. |
| |
| [cols=",,,,",options="header",] |
| |=== |
| |tokenizer |numOfRecords |maxWaitTime (ms) |XML size (kB) |time (ms) |
| |StAX |40000 |5000 |3543 |3052 |
| |XML |40000 |5000 |3543 |2756 |
| |StAX |1000000 |20000 |89735 |11740 |
| |XML |1000000 |20000 |89735 |11137 |
| |StAX |15000000 |200000 |1366102 |132176 |
| |XML |15000000 |200000 |1366102 |132549 |
| |=== |
| |
| === Help and contributions |
| |
| If you hit any problem using Camel or have some feedback, then please |
| https://camel.apache.org/support.html[let us know]. |
| |
| We also love contributors, so |
| https://camel.apache.org/contributing.html[get involved] :-) |
| |
| The Camel riders! |
