commit f4996f0fe86643e9282d37a06263c3cc5a82de81
author Steve Lawrence <slawrence@apache.org> Tue Jan 09 07:39:43 2024 -0500
committer Steve Lawrence <slawrence@apache.org> Tue Jan 09 09:43:39 2024 -0500
tree 574bb0b936f75187d94fd910a6ec768fbbb11a36
parent 20d6ea3479c85e431faa260424ecb5e2675d7d8c

Improve performance of padding removal when parsing

The current algorithm to remove right padding of left justified strings
first reverses the String, removes leading pad characters using
dropWhile, and then reverses the result. The two reverses are linear in
the length of the String, and requires allocating multiple String
instances and copying characters from one to the other. And this is done
regardless of how many, if any, pad chars exist in the String. This
logic is very clear, but is fairly inefficient, enough to show up while
profiling.

To improve performance, this rewrites the algorithm to scan through the
String in reverse to find the index of the last pad character and then
uses the substring() function to create a new String with those pad
characters removed. This is now linear in the number of pad characters
in a String instead of the full length of the string. Additionally, the
use of substring() avoids character copies, since it just allocates a
new String using the same underlying String value but with different
indices.

I have not looked into detail how scala implements dropWhile() for
Strings (skimming the code, it looks like it will allocate a new String
and copy characters), but for consistency and maximum performance, this
also updates the algorithm that removes left padding of right justified
strings to use similar logic as the new right padding algorithm. By
using substring() we should avoid possible copies.

In one test with lots of left justified strings, many of which are
padded, this saw about a 15% improvement in parse times (excluding
infoset creating using the null infoset outputter), and padding removal
no longer shows up while profiling.

DAFFODIL-2868