tree: 612085f7cd226fa8bdebb1371cccf8356e48352d [search]
  1. bench.sh
  2. README.md
  3. seed.sh
dev-support/perf/README.md

Performance benchmark: GET /records/{id} reference resolution

Reproduces the workload measured in apache/sourcelume-registry#24: model records whose trainingData lists reference a growing number of dataset records, and what resolving those references costs on the read path.

The scripts talk only to the public record API (POST /records, GET /records/{id}), so they measure whichever storage plugin the registry runs — the default PostgreSQL distribution or the Atlas one — and the numbers stay comparable across backends.

Prerequisites

  • a running registry — the PostgreSQL distribution or the Atlas stack
  • curl and jq
  • a POSIX shell — on Windows run from Git Bash or WSL2, like the rest of dev-support
  • a disposable registry: seeded records accumulate, see Notes

Usage

./dev-support/perf/seed.sh     # seeds 161 datasets + models sized 0/12/40/161 refs
./dev-support/perf/bench.sh    # n=30 timed reads per model record

bench.sh picks the newest manifest in out/; pass an explicit path to measure an older seed. Example output (Atlas distribution, PostgreSQL backend):

record                                                        n       min    median       max  refs
perf-20261113T103000Z-model-refs0                            30     301ms     333ms     402ms     0
perf-20261113T103000Z-model-refs12                           30     355ms     391ms     460ms    12
perf-20261113T103000Z-model-refs40                           30     412ms     448ms     511ms    40

Knobs

variabledefaultmeaning
REGISTRY_URLhttp://localhost:8082registry base URL — same API port on both distributions
PERF_PREFIXperf-<utc timestamp>record id prefix
PERF_DATASETS161dataset records seeded
PERF_REFS0 12 40 161trainingData sizes, one model record each
PERF_ITERS30timed reads per record
PERF_WARMUP2untimed reads before timing
PERF_OUT./out (git-ignored)manifest directory

Notes

  • Every record id carries PERF_PREFIX, so each run writes new records rather than colliding with earlier seeds (POST /records returns 409 for a VALIDATED id). Point the scripts at a disposable registry if you run them repeatedly.
  • seed.sh posts one record at a time on purpose: ingest is synchronous and the dataset writes are not what this benchmark measures.
  • To compare backends, seed and time each distribution with the same PERF_* values; medians land in the same units.