tree cd3ccdc3f764474bc1297ee5227e2d30103eecb1
parent 9c16b972c0b1d2ee83bb56d7ca92b1e74b0753d1
author Tim Saucer <timsaucer@gmail.com> 1789415022 -0400
committer Tim Saucer <timsaucer@gmail.com> 1789415022 -0400

Let tpchgen-cli shard the data instead of resharding it

`tpchgen-cli` has `--parts N`, which writes a table as N Parquet files into a
directory named for it. That is exactly the shape the storage library's table
provider wants -- it scans `*.parquet` under a directory, one partition per
file -- so `run_tpch.py` can generate what it needs in one call instead of
reading a 220 MB single-file `lineitem` and slicing it into shards itself.

Drops `reshard`, the column list it needed, and the `--data`/`--rows`
arguments, along with the prerequisite that a reader generate a 1 GB dataset
by hand before the script will run at all. `--scale` replaces `--rows` as the
size knob, defaulting to a tenth of scale factor 1.

The CI step installs `tpchgen-cli` itself rather than inheriting it from the
TPC-H dataset step, since it no longer reads that dataset.