docs/computer-use-ui-coverage.md

# Computer Use Semantic UI Coverage

Status: maintained source and runtime inventory for Maka Desktop's
Computer Use target, state, focus, action and effect semantics.

This document defines what "covered" means for Computer Use operability. A page
is not complete merely because it renders or because every button has a
non-empty name. Critical surfaces must expose an unambiguous target, state,
keyboard/focus behavior, and an observable effect.

This gate is not a claim of complete WCAG conformance or complete assistive
technology behavior. In particular, live-region and announcement completeness
are not measured here; they require separate accessibility testing.

## Automated surface inventory

| Surface | Runtime states | Semantic actions and effects |
|---|---|---|
| Settings | Every current top-level navigation entry is enumerated from the running UI; provider list/detail/catalog/add, subagent editor, memory populated, permission diagnostics, import empty/ready, usage variants, Daily Review selector and narrow states are in Storybook | Navigation exposes `aria-current`; focused nested stories exercise dialogs, disclosures, selectors and editors |
| Extensions | Skills empty/installed/bundled/update/disabled/narrow/inspector; MCP setup/marketplace/configured/inspector/editor/failure/narrow | Page selection, inspector/editor opening and actionable-node identity |
| Scheduled work | Empty/configured/long/narrow/task inspector; Daily Review loading/error/refreshing/report | Task selection, dialog focus, selector state and report actions |
| Conversation shell | New task, settled conversation, streaming, permission wait, native conversation, modes, context and inline completion | Composer submit effect, unique per-task and per-turn actions, current regions and workbar selection |
| Workbar | Launcher plus side chat, changes, active terminal, browser chrome, files, tasks and trace states | Selected tabs, terminal input reaches the PTY bridge, browser navigation reaches the browser bridge |
| Dialogs and overlays | Rename, scheduled-task form, Mermaid fullscreen, side-chat close, onboarding QR, WeChat QR, Runtime Host SSH and remote directory chooser | Dialog naming/focus, rename callback, close confirmation, SSH input, directory navigation and registration |
| Generated content | Markdown, tool output, attachments, Mermaid and HTML artifact iframe | Scoped copy actions, fullscreen dialog and sandboxed iframe semantics |
| Embedded browser page | Live `WebContentsView` on a loopback fixture | Separate observe -> semantic ref -> fill/click -> business-effect smoke through the production browser bridge |

The Storybook catalog is exhaustive for its source-defined entries and the
smoke runner carries a required Computer Use story manifest for critical
runtime boundaries. The Electron accessibility test dynamically enumerates
settings navigation, then covers modules, global overlays, conversations and
all workbar entry points.

## AX completion gates

Every measured final state fails on:

- an AX tree with zero exposed nodes, including trees containing only ignored
  Chromium source records;
- an actionable node without a name;
- two actionable nodes with the same role, name and semantic scope;
- more than one primary `main` landmark;
- a dialog without a name;
- a checkbox, radio, switch, menu checkbox/radio, option, tab, combobox,
  slider or spinbutton without its required state/value;
- focus inside an inert or `aria-hidden` surface;
- a visible modal dialog that does not own focus.

Critical stories and Electron journeys additionally assert action-specific
effects. Overlay journeys wait for the specifically named prior dialog to
close before opening and auditing the next one. Transport success or a generic
dialog match alone is not accepted.

Repeated message and answer actions use a bounded excerpt of visible text plus
a stable human-readable timestamp. Opaque storage IDs remain machine data and
are not spoken as the user-facing differentiator.

## Platform boundaries

- Renderer and same-process iframe semantics are validated through Chromium's
  full AX tree.
- Embedded browser content is a separate `WebContentsView`; it is validated
  through its production semantic snapshot/action bridge, not falsely claimed
  as part of the renderer tree.
- xterm surfaces run with `screenReaderMode` and have active PTY and SSH input
  fixtures.
- Third-party page markup, Chromium's PDF plugin, native file dialogs and
  operating-system permission dialogs remain owned by their respective
  platform/provider. Maka validates the controls it owns around those surfaces
  and fails closed when the external target is unavailable or ambiguous.
- Native macOS AX and Windows UIA packaged-app spot checks remain release gates;
  Linux CI cannot substitute for those platform trees.

## Performance contract

The product adds no accessibility dependency, production AX/DOM walker, global
`MutationObserver`, timer, polling loop, OCR model or hidden duplicate agent UI.
Semantic names and state are emitted by the existing React render. All full-tree
walking, duplicate detection and inventory enforcement run only in tests and
developer tooling. Existing lazy module, workbar and dialog loading remains
unchanged.