tree 0bbe34eaebebd52efcd7fba288703dfe38d585f2
parent ec5934ad8e6aec99f495b094826f69e6df3f6d2b
author Yordan Pavlov <yordan.pavlov@outlook.com> 1614887108 -0500
committer Andrew Lamb <andrew@nerdnetworks.org> 1614887108 -0500

ARROW-11799: [Rust] fix len of string and binary arrays created from unbound iterator

While looking for a way to make loading array data from parquet files faster, I stumbled on an edge case where string and binary arrays are created with an incorrect length from an iterator with no upper bound.

Here is an example for such an iterator:

```
 // iterator that doesn't declare (upper) size bound
let string_iter = (0..).scan(0usize, |pos, i| {
    if *pos < 10 {
        *pos += 1;
        Some(Some(format!("value {}", i)))
    }
    else {
         // actually returns up to 10 values
         None
     }
})
// limited using take()
.take(100);
```

For even more details please see the new tests I have added in this PR.
Fortunately this is easy to fix by using the length of the child offset array.

@jorgecarleitao

Closes #9588 from yordan-pavlov/fix_array_len_from_unbound_iter

Authored-by: Yordan Pavlov <yordan.pavlov@outlook.com>
Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>