Black-box tests that drive the built doriscli binary against a real, already-deployed Doris cluster and report which commands pass, fail, or are skipped. Everything goes through doriscli itself — there is no separate MySQL client and no mocking — so a green run means the CLI genuinely works end to end against that cluster.
# from the repo root (doris-cli/) cp tests/e2e/cluster.env.example tests/e2e/cluster.env $EDITOR tests/e2e/cluster.env # fill in host/port/user/password ./start-testing.sh
or pass the connection on the command line:
./start-testing.sh --host fe.example.com --port 9030 --http-port 8030 \ --user root --password 'secret'
The runner builds doriscli, probes connectivity, creates a throwaway doriscli_selftest database, runs every suite, drops the database, and prints a summary. Exit code is 0 only if nothing failed (skips don't fail the run). Full per-command output for the run is saved under tests/e2e/results/run-<timestamp>.log.
Requirements: bash, jq, and either a Rust toolchain (to build) or a prebuilt binary via --bin.
| Status | Meaning |
|---|---|
| PASS | The command behaved exactly as the contract requires. |
| FAIL | The command misbehaved: wrong exit code, malformed JSON, or a missing/!= expected field. These are the ones to look at. |
| SKIP | A precondition of the cluster (not a bug in the CLI) was absent, so the test couldn‘t run — e.g. the FE HTTP API isn’t reachable, or audit_log isn't enabled. Reported separately; never fails the run. |
The SKIP state exists because several doriscli features depend on optional cluster capabilities. Treating “audit_log disabled” as a CLI failure would cry wolf; treating it as a silent pass would hide untested surface. So it's its own bucket, and the summary lists exactly what was skipped and why.
Run a subset with --only "<suites>"; list them with --list.
cargo test (offline, unless --no-unit)The crate‘s in-tree unit tests — primarily the profile-text parsers (section_parser, fragment_parser, operator_parser, value_parser). No cluster needed. Includes parses_a_real_captured_profile, an offline regression that parses a real profile captured by the e2e run (see below) and asserts the load-bearing invariants. It’s a visible no-op until you commit the fixture tests/e2e/fixtures/sample_profile.txt that a cluster run generates.
cli — argument contract (offline)--version / -V, --help (usage + subcommand listing), and the error paths: sql with no query, unknown subcommand, tablet with no table, profile with no action, and an unknown flag. Verifies non-zero exit on misuse.
auth — connection management + stateless mode (needs cluster)Uses an isolated $HOME, so your real ~/.doris is never touched.
auth list on an empty config → empty-list shape.auth add → saves an env (first one becomes default); auth list reflects it.auth status → connects over MySQL; asserts .mysql_status == "connected" (the command always exits 0, so the field is the real connectivity check).use / use <name> → show and switch the default env.auth add --mysql mysql://… → URI parsing (skipped if the password isn't URI-safe).auth remove → deletes an env.DORIS_HOST+DORIS_USER): auth add is refused, auth status still connects from env vars, and no files are written to disk (verified against a pristine HOME).sql — execution (needs cluster)The JSON envelope (query_id, exec_time_ms, rows_returned, columns[], rows[]), type mapping (string vs number), -f <file>, --set (single and repeated), --no-cache, --profile (yields a query_id), --format table/csv, empty result sets, a COUNT(*) over the loaded data, and the error path (SQL error: on a bad reference, non-zero exit).
tablet — bucket / tablet analysis (needs cluster + seeded data)Overview: model, bucket_type, bucket_key, bucket_count, sort_key, total_rows; the health.tablet_skew summary; columns[].ndv (SKIP if column stats weren't collected). UNIQUE model detection on a second table. --detail (per-tablet + per-backend) and --detail --partition (narrowed to one partition). Negative: a missing table exits non-zero.
profile — query profiles (needs cluster + the FE HTTP profile API)This suite is the point of the harness: it sends real SQL with --profile, fetches the real profile, and asserts on the parsed values, not just the JSON shape. It runs three profiled queries — a group-by over events (twice) and a hash join events⨝dim_users — then checks, against the live profile:
profile list / list --active → arrays carrying the real SHOW QUERY PROFILE fields (MySQL only, always testable).profile get <id> → the parsed Profile ID equals the id we sent, the parsed SQL is our query text, total_time_ms is a positive parsed number, the operator tree includes a SCAN and an AGG, query_stats.total_scan_rows is ≈ the rows we loaded, and the fragment breakdown is well-formed.get --full → the full fragments→pipelines→operators tree with populated all_counters; get --raw → the raw text round-trips (and is captured as the offline regression fixture, see below).profile diff → two real runs of the same query: parsed totals on both sides and a numeric time_ratio.JOIN operator and ≥2 SCAN operators are parsed.The full profile text comes only over HTTP (REST v2 / legacy; the SQL path yields summary metadata with no operators). So the FE HTTP profile API is a hard requirement: if http_status != connected, these parse tests FAIL, not SKIP — a silent skip behind a green run would hide the entire parser. Two narrower conditions still SKIP, because they're cluster preconditions and not parser bugs: a profile evicted before we could fetch it (raise max_query_profile_num), and profile history, which needs __internal_schema.audit_log. Operator→table attribution auto-SKIPs on pre-4.0 Doris (its operator headers omit table_name). A negative case (unknown query id exits non-zero) rounds it out.
setup_data creates two tables in doriscli_selftest (DDL is mode-agnostic; replication_num=1 works on self-hosted and is required in cloud mode):
events — DUPLICATE model, range-partitioned by event_date into two partitions (Jan/Feb 2024), DISTRIBUTED BY HASH(user_id) BUCKETS 4 → 8 tablets. Loaded with --rows (default 2000) rows via a generated INSERT fed through sql -f (dates computed server-side by DATE_ADD, so no dependency on any particular Doris version). ANALYZE TABLE … WITH SYNC runs best-effort so tablet's columns[].ndv is populated.dim_users — UNIQUE model, 2 buckets, a handful of rows.The database is dropped on exit (even on Ctrl-C) unless you pass --keep.
See ./start-testing.sh --help. Highlights: --only, --no-unit, --keep, --release, --bin <path>, --no-build, --rows <n>, --config <file>.
auth status‘s HTTP probe and is required by the whole profile suite — it’s the only source of profile text, so without it the profile parse tests FAIL (they used to SKIP). Point --http-port at the right port before running profile.DORIS_TEST_INIT_SQL='USE @<compute_group>' so queries run against a live compute group; otherwise setup fails early with a hint.CREATE/DROP/INSERT on the self-test database (and SELECT on information_schema). For full profile coverage the cluster should retain profiles (enable_profile is set per-query by --profile) and, for profile history, have the audit log enabled.