Span oracle

With --all-features, every SQL string a test parses through TestedDialects also has its spans checked, once per dialect of the set. A node‘s correct span covers every token the node consumed, including its own keywords and parentheses, and excludes a statement’s trailing ;.

CheckApplies toClassMeaning
extenttop-level statementsemptyno span
inexactdiffers from the first to last token of a single-statement input
invalidlies outside the input or is inverted
overlapstarts before the previous statement ends
structureevery node with a spanemptyno span, although the node renders text
not-emptya span, although the node renders nothing
invalidlies outside the input or is inverted
outside-parentnot inside the nearest ancestor span
reparseStatement, Query, Select, TableFactor, Expr, OrderByExpr, ValueWithSpan, Ident, ObjectNameinexactthe source under the span does not parse back to the node
edgesevery other node that renders textstart, end, boththe source under the span starts or ends on a different token than the rendering

The oracle runs with --all-features only, because the baseline stores node renderings and bigdecimal changes how numbers render. It is skipped when tests/span_baseline.tsv does not exist, as in the published crate.

Baseline

tests/span_baseline.tsv lists every known failure. Its columns are the input hash, the parser options hash, the dialects, the check, the class, the node as Type::Variant, the occurrence of that node and text within the input, and the node's text.

A test fails when a finding is missing from the baseline, and also when a baseline finding no longer occurs. The failure message lists both.

  • A new finding in code you changed is a regression, so fix the span. A new test for syntax whose spans are already incomplete adds findings, so record them.
  • A finding that no longer occurs means a span got fixed, so record the baseline again.

Recording

rm -rf target/span-oracle
SPAN_ORACLE=record cargo test --all-features &&
  find target/span-oracle -name '*.tsv' -exec cat {} + | LC_ALL=C sort -u > tests/span_baseline.tsv

Recording ignores the committed file, so a merge conflict in it is resolved by recording again. CI runs the same recording and fails if the result differs from the committed file.