blob: f6dea28270dc40c82eb6341398b594e376dfbb97 [file] [view]
---
title: Quickstart
---
<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
## Installation
Install from source:
```shell
git clone https://github.com/apache/sedona-spatialbench.git
cd sedona-spatialbench
cargo install --path spatialbench-cli
```
After installation, you should be able to run:
```shell
spatialbench-cli --help
```
## Generate SF1 Data
To generate the full dataset at scale factor 1 in Parquet format:
```shell
spatialbench-cli --scale-factor 1
```
This creates six tables:
* trip
* customer
* driver
* vehicle
* zone
* building
Output is written to the current directory by default.
## Customizing Output Files
We'll go over a few common options to customize the output files. To see all available options, run `spatialbench-cli --help`.
### Generate a Subset of Tables
```shell
spatialbench-cli --scale-factor 1 --tables trip,building
```
### Partition Table Output into Multiple Files
Specify the number of partitions manually:
```shell
spatialbench-cli --scale-factor 10 --tables trip --parts 4
```
Or let the CLI determine the number of files using target size:
```shell
spatialbench-cli --scale-factor 10 --mb-per-file 512
```
### Set Output Directory
```shell
spatialbench-cli --scale-factor 1 --output-dir data/sf1
```
### Generate Data Directly to S3
You can generate data directly to Amazon S3 or S3-compatible storage by providing an S3 URI as the output directory:
```shell
# Set AWS credentials
export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_REGION="us-west-2" # Must match your bucket's region
# Generate to S3
spatialbench-cli --scale-factor 10 --mb-per-file 256 --output-dir s3://my-bucket/spatialbench/sf10
# For S3-compatible services (MinIO, etc.)
export AWS_ENDPOINT="http://localhost:9000"
spatialbench-cli --scale-factor 1 --output-dir s3://my-bucket/data
```
The S3 writer uses streaming multipart upload, buffering data in 32 MB chunks before uploading parts. All standard AWS environment variables are supported, including `AWS_SESSION_TOKEN` for temporary credentials.
## Configuring Spatial Distributions
SpatialBench uses a spatial data generator to generate synthetic points and polygons using realistic spatial distributions.
To read more about the different spatial distributions offered by SpatialBench see [here](https://sedona.apache.org/spatialbench/spatialbench-distributions/).
For more details about tuning the spatial distributions and the full YAML schema and examples, see [CONFIGURATION.md](https://github.com/apache/sedona-spatialbench/blob/main/spatialbench-cli/CONFIGURATION.md).
You can override these defaults at runtime by passing a YAML file via the `--config` flag:
```shell
spatialbench-cli --scale-factor 1 --config spatialbench-config.yml
```
If `--config` is not provided, SpatialBench checks for `./spatialbench-config.yml`. If absent, it falls back to built-in defaults.