docs/benchmarks/methodology.md

---
title: Benchmark Methodology
sidebar_position: 2
id: methodology
license: |
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
---

A useful serialization benchmark records enough context to reproduce the workload and avoid
comparing different semantics as if they were equivalent.

## Required context

- repository commit and dependency versions;
- hardware, operating system, runtime version, and relevant environment variables;
- schema and object population;
- xlang or native mode, compatibility mode, reference tracking, and registration;
- warm-up, forks/processes, iterations, duration, and concurrency;
- serialized representation and operation measured;
- exact command and raw result location.

## Interpret the reports

Compare one operation and equivalent data semantics. Do not compare an xlang payload with a native
payload or a full object reconstruction with Row Format field access without naming that semantic
difference. JIT- and codegen-based implementations require representative warm-up.

Run the active benchmark harness for decisions that depend on current code. Checked-in reports are
evidence for the commit and environment they record.