OpenHome / Product & developer experience

Full findings

The complete client-safe index: 100 entries across strengths, coverage, observations and open findings. Priority, status and evidence class keep the limits of each entry visible.

How to read this index

Entries include their canonical ID, title, summary, credibility lenses and—where present—evidence limits and acceptance criteria. Local evidence files and private registry exports are not linked; public source pages are available in each finding’s disclosure. Confidence comes from each entry’s status and evidence class.

What works

K01 · — · verified in scope

Working: API-key-only Agent/Ability author, push, trigger, edit, commit, retrigger, sync and disable/re-enable loop

The bounded API-key workflow completed Agent/Ability creation, push, text invocation, edit, commit, retrigger and source sync. This gives developers a working foundation for iteration. Preserve the successful loop while clarifying session-specific enablement and revision state.

Evidence limits and acceptance criteria

Evidence limits

Verified in the bounded API-key/text journey. Disable/re-enable success in fresh sessions does not establish existing-session behavior; X01 records the separate active-session limit. Native audio and hardware were not accepted.

Acceptance criteria

Trial journey, live evidence. Preserve as regression acceptance for future fixes.

K02 · — · verified in scope

Working: hosted trial returns the expected scripted greeting; normal conversation resumes

Working: hosted trial returns the expected scripted greeting; normal conversation resumes

Evidence limits and acceptance criteria

Evidence limits

The greeting is hardcoded fixture output, not independent platform verification or model correctness. Six expected-response checks are a small client-side sample, not production latency percentiles or audio acceptance.

Acceptance criteria

Timing evidence. Five Ability replies and one ordinary reply matched expectations; not audio or production reliability proof.

K04 · — · verified in scope

Positive: a contextual onboarding tour introduces conversation controls

The supplied screenshots show a guided tour explaining how to start and stop a conversation, with progress indicators and Back, Next and Skip controls. Gabriel explicitly values having this guidance.

View screenshot evidence (2)
Evidence limits and acceptance criteria

Evidence limits

Presence and the displayed instruction are screenshot-backed; usefulness is the user's assessment. Successful completion, adoption impact and full accessibility were not tested.

Acceptance criteria

Preserve the contextual explanation, progress and skip/back/next flow while fixing readability. Verify completion, dismissal, reopening and keyboard access; measure whether new users reach their first conversation without assistance.

K05 · — · verified in scope

Positive: sleep-mode guidance gives the user a way to resume

The conversation visibly tells the user to call the Agent's name to wake it from sleep mode. The user specifically appreciated the guidance on how to reawaken it.

Evidence limits and acceptance criteria

Evidence limits

The guidance is visible and was valued by the user. This does not prove that the attempted phrase was the correct wake phrase or that wake-up recognition worked reliably.

Acceptance criteria

Preserve actionable wake-up guidance as a clearly identified status message. State the actual accepted wake phrase or action for the current Agent and verify successful recovery without requiring the user to guess.

K06 · — · verified in scope

Core SDK helpers work in bounded hosted checks

40 of 43 methods exercised: 18 working in bounded checks, 22 partially tested and 3 blocked by remaining prerequisites. The corrected foreground suite completed 39 command/recovery cases; instrumented timing samples are available. Synthetic audio is not microphone/speaker acceptance.

Evidence limits and acceptance criteria

Evidence limits

Partial results and three prerequisite blockers are explicit; no universal SDK certification or native audio acceptance.

Acceptance criteria

Preserve these bounded contracts in repeatable regression checks; expand coverage using the per-method prerequisites.

K07 · — · verified in scope

Relative-time alarm and conversational recovery succeeded in user testing

Gabriel reports a one-minute-from-now request rang about one minute later at 11:08 p.m. A separate screenshot shows a later flooring exchange receiving an ordinary reply after an Alarm interaction; it does not show or establish that the reported one-minute alarm rang.

Evidence limits and acceptance criteria

Evidence limits

Single user-reported timing trial; not a measured timing guarantee or certification of the separate Timer Ability. Recovery screenshots rule out a consistent lock-in claim.

Acceptance criteria

Preserve relative-time scheduling and easy topic recovery while correcting unwanted activation.

Coverage confirmed

K03 · — · verified in scope

Covered: 81 indexed docs and public repository map; supported CLI is Python, separate from archived TypeScript CLI

Covered: 81 indexed docs and public repository map; supported CLI is Python, separate from archived TypeScript CLI

Evidence limits and acceptance criteria

Evidence limits

This is audit inventory coverage, not a product capability pass or evidence that every page and example is correct.

Acceptance criteria

Coverage map, repo map, platform brief. Recheck upstream revisions before implementation.

Coverage gaps

G01 · — · unverified

Unverified: native mobile, microphone/audio quality, interruption, hardware and Local Link execution

Unverified: native mobile, microphone/audio quality, interruption, hardware and Local Link execution

Evidence limits and acceptance criteria

Evidence limits

Synthetic audio recording/playback and empty stream lifecycle now pass bounded transport checks. Physical microphone/speaker quality, active interruption behavior and hardware remain unverified.

Acceptance criteria

Trial limits. Native iPhone access was blocked; responsive web is not native-app acceptance.

G02 · — · unverified

Unverified: server stage latency, broad reliability, production percentiles, longer/generative Ability performance

Unverified: server stage latency, broad reliability, production percentiles, longer/generative Ability performance

Evidence limits and acceptance criteria

Evidence limits

New instrumented SDK timing samples distinguish in-method time from client roundtrip. They do not establish server-stage traces, percentiles or broad reliability.

Acceptance criteria

Latency review. Obtain correlated traces and a broader sample; six short turns are diagnostic only.

Product findings

D01 · P1 · open

Source migration path can delete unpushed user code; destructive path not executed

Source migration path can delete unpushed user code; destructive path not executed

View cited public sources (4)
Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Migration preserves modified/untracked user files and offers recoverable backup.

D02 · P1 · open

Companion dashboard framed production-ready despite missing auth/persistence; not a finding about hosted web-app security

Companion dashboard framed production-ready despite missing auth/persistence; not a finding about hosted web-app security

View cited public sources (1)
Evidence limits and acceptance criteria

Evidence limits

Docker validation completed the published wheel install in a fresh virtual environment, including pyaudio, because the image already had PortAudio/compiler prerequisites. This does not establish a universal OS failure or a complete install matrix; no upstream fix is claimed.

Acceptance criteria

Demo limitations explicit; production requirements separately tested.

D03 · P1 · open

API-key/JWT guidance lacks a clear conditional contract; key-only path passed

The tested server accepts an API key, but the docs and CLI message do not explain when a JWT fallback may actually be needed

View cited public sources (5)
Evidence limits and acceptance criteria

Evidence limits

Key-only create/update/commit passed in the tested account and deployment. This review did not establish that JWT is never required on an older or differently configured server; no such server was tested.

Acceptance criteria

Document API-key-only as the tested baseline and identify any server/version cases where JWT is needed; key-only and expired-JWT recovery paths are separately actionable.

D04 · P1 · open

Background Ability package contract is unclear; validator requires main.py

Background Ability docs present a background.py-only layout, while the validator requires main.py

View cited public sources (4)
Evidence limits and acceptance criteria

Evidence limits

Current evidence confirms a local documentation/validator contract mismatch, not a background runtime failure. Hosted promotion and the official asyncio example were not retested in this pass; no upstream fix is claimed.

Acceptance criteria

Background Ability docs, validator and promotion checks agree on required files; a documented background-only fixture passes applicable checks without an undocumented main.py shim.

D05 · P2 · open

Basic starter omits required cleanup on error; runtime symptom untested

Basic starter omits required cleanup on error; runtime symptom untested

View cited public sources (3)
Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Success and exception paths resume normal flow.

D06 · P1 · open

Legacy template links return 404; current replacements are incomplete

13 link occurrences across 9 legacy paths return 404; most have lowercase replacements, but Local remains unresolved

View cited public sources (7)
Evidence limits and acceptance criteria

Evidence limits

Read-only HTTP/tree checks establish path availability, not that replacement instructions are equivalent. Most lowercase replacements resolved; the Local replacement remained unresolved. The separate examples repository was unavailable; no claim is made that every template or integration is broken.

Acceptance criteria

Every documented source target resolves to the intended current example or has an explicit tested replacement; Local guidance identifies the supported bridge or device target.

D07 · P1 · open

Recommended wheel lacks starter assets; artifact and lookup reproduced

Recommended wheel lacks starter assets; artifact and lookup reproduced

View cited public sources (3)
Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Clean package install discovers and scaffolds bundled starters.

D08 · P2 · open

Prerequisites and install paths diverge; package version alone does not identify source

Prerequisites and install paths diverge; package version alone does not identify source

View cited public sources (3)
Evidence limits and acceptance criteria

Evidence limits

Docker validation completed the published wheel install in a fresh virtual environment, including pyaudio, because the image already had PortAudio/compiler prerequisites. This does not establish a universal OS failure or a complete install matrix; no upstream fix is claimed.

Acceptance criteria

Tested install matrix and identifiable build revision.

D09 · P2 · open

ability.md, trigger and category guidance conflict; complete runtime routing contract unverified

ability.md, trigger and category guidance conflict; complete runtime routing contract unverified

View cited public sources (3)
Evidence limits and acceptance criteria

Evidence limits

Current public pages and the pinned template still disagree about ability.md versus category/trigger metadata. A read-only hosted text matrix against the existing harmless fixture tested an unrelated control, an exact trigger, a sentence containing the trigger, an explicit negation, a longer-word near match and a near miss; the trigger-containing cases activated the fixture. This establishes bounded text behavior, not the server algorithm, audio behavior, disabled-state behavior or a universal routing rule; no upstream fix is claimed.

Acceptance criteria

One documented contract with positive/negative routing tests.

D10 · P2 · open

Parameter names, sync/async and constructor instructions disagree

Parameter names, sync/async and constructor instructions disagree

View cited public sources (7)
Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Runnable examples match the authoritative SDK interface.

D11 · P2 · open

Hermes instructions disagree about approval; denial path corroborated in source

Hermes instructions disagree about approval; denial path corroborated in source

View cited public sources (3)
Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Document and test approval, denial and recovery consistently.

D12 · P2 · open

Required-field instructions disagree; specific minimum form validation not proved

The current Quickstart says name, description and category are the minimum to save; the current Creating an Agent guide says only name and description block saving. The public docs conflict, while the app's minimum-field behavior was last checked on September 15.

View cited public sources (2)
Evidence limits and acceptance criteria

Evidence limits

Both public pages were checked on September 22. The live form behavior was checked on September 15, not re-tested in this pass; this establishes a current documentation conflict, not current UI behavior or a failed save.

Acceptance criteria

Clean-account minimum Agent creation matches guide; related W05.

D13 · P2 · open

“Any OpenRouter model” promise conflicts with another page's fixed selection contract

“Any OpenRouter model” promise conflicts with another page's fixed selection contract

View cited public sources (2)
Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Clearly distinguish Agent model selection from Ability API calls.

D14 · P2 · open

Example secret storage can travel with submissions; source/packaging risk, no observed secret leak

Example secret storage can travel with submissions; source/packaging risk, no observed secret leak

View cited public sources (4)
Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Shareable examples externalize secrets; packaging checks prevent accidental inclusion.

D15 · P2 · open

Legacy client, CLI bridge and DevKit entry paths lack clear chooser/migration guidance

Legacy client, CLI bridge and DevKit entry paths lack clear chooser/migration guidance

View cited public sources (4)
Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

One supported path per use case with migration and retirement notes.

W03 · P2 · open

Receipt displays new draft v2 while history committed v1; data coherent, labels ambiguous

Receipt displays new draft v2 while history committed v1; data coherent, labels ambiguous

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Receipt names both committed release and next working draft.

W04 · P2 · open

Disabled switch with Enabled text; Sep 15 fresh calls respected disable. Sep 18 existing-session counterexample is X01

Disabled switch with Enabled text; Sep 15 fresh calls respected disable. Sep 18 existing-session counterexample is X01

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Text, switch, API and runtime agree after toggles; test existing and fresh connections.

W07 · P2 · open

Settings navigation and voice-public-use switch lack accessible names/state at inspected narrow layout

In the inspected narrow Settings view, five icon-only navigation buttons had no accessible names; the voice-creation public-use toggle exposed no aria-pressed or aria-checked state. Desktop tab labels were present. The API Keys tab exists, and Profile also provides a Copy API key shortcut.

Evidence limits and acceptance criteria

Evidence limits

Bounded observation of five Settings controls and one voice-creation toggle, not a full accessibility audit. The earlier claim that the API Keys tab was absent is withdrawn; the documented route exists.

Acceptance criteria

Give each Settings navigation control a programmatic name at narrow and desktop breakpoints; expose the public-use control as a correctly named switch with its current checked state. Verify keyboard operation and screen-reader output for these controls, without implying a full accessibility audit.

W08 · P2 · open

CLI own-Ability inventory differs from web defaults; effective runtime scope unclear

CLI own-Ability inventory differs from web defaults; effective runtime scope unclear

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Explicit owned/default/installed/enabled/Agent scope reconciles with invocation.

W10 · P2 · open

Sign-in identity vs connected Google service, provider defaults and effective configuration need clearer explanations

Sign-in identity vs connected Google service, provider defaults and effective configuration need clearer explanations

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

User can identify account, connected scopes, key ownership and effective Agent config without revealing secrets.

C03 · P2 · open

Missing credential with network disabled reaches retries/network failure instead of local credential diagnosis

Missing credential with network disabled reaches retries/network failure instead of local credential diagnosis

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Missing-key error before network; separately test expired/revoked-key response.

C04 · P2 · open

Create is documented as auto-push, but no-trigger runs complete local-only with exit 0

The shipped README describes create as auto-pushing to the account, while the reviewed source skips upload when triggers are absent, prints a prose hint and exits 0.

View cited public sources (3)
Evidence limits and acceptance criteria

Evidence limits

The CLI tells an interactive user that the no-trigger run was not pushed, so this is a contract and automation ambiguity rather than proof of a hidden upload failure or data loss. No authenticated upload was attempted in this offline reproduction; the triggered path stopped at the expected missing-credential check.

Acceptance criteria

Align create documentation, help and implementation; return an explicit local-only versus uploaded result and documented exit semantics that automation can evaluate without parsing prose.

C05 · P3 · open

CLI root help mislabels text chat as an interactive voice session

The captured root help describes chat as an interactive voice session, but the tested command exchanges typed text; call is the separate microphone-and-speaker path. The current CLI guide explicitly distinguishes text chat from voice calls, so the mismatch is localized to the CLI help wording.

View cited public sources (1)
Evidence limits and acceptance criteria

Evidence limits

The raw CLI help and text-session behavior were captured at the pinned source revision; the current guide now states the distinction clearly. The issue concerns help terminology, not missing audio capability.

Acceptance criteria

Describe chat as typed text, call as microphone/speaker voice, and local as the bridge, matching their actual behavior.

C06 · P2 · open

Call source prints no-response text then returns success; live timeout branch untested

Call source prints no-response text then returns success; live timeout branch untested

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Controlled timeout produces distinct machine-readable result and documented non-success exit.

L01 · P2 · open

Three calls spent 7.165–7.228 s before sending the request; function total 8.616–8.685 s. Measured plus greeting-wait source

Three calls spent 7.165–7.228 s before sending the request; function total 8.616–8.685 s. Measured plus greeting-wait source

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Expose setup/greeting/send/response phases; diagnose server delay and evaluate a supported reusable test session.

L02 · P2 · open

Initial six replies had ~1 s between partial and final text; deeper E07 measured 4.831 s. Backend reasons unknown

Initial six replies had ~1 s between partial and final text; deeper E07 measured 4.831 s. Backend reasons unknown

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Correlate server spans; report first and final text separately and preserve audio semantics. See deeper timing.

L03 · P2 · open

CLI live redraw can retain trailing characters and omit corrected final content

Offline probes of unchanged cmd_chat reproduced stale suffix text, including the reported dy. pattern, and omission of differing final content after a live update. The careful hosted run captured clean live/final pairs; original overlap and duplicate greeting were not independently reproduced.

Evidence limits and acceptance criteria

Evidence limits

Synthetic callback replay with basic CR/LF terminal semantics demonstrates a rendering mechanism, not the original session event order. Hosted capture waited for greeting completion; early-input overlap remains untested. No upstream fix.

Acceptance criteria

Capture exact command/terminal; verify clean final lines and prompts in TTY and redirected output.

X01 · P1 · open

Existing connection invokes Ability after acknowledged disable; fresh connection respects disable

Existing connection invokes Ability after acknowledged disable; fresh connection respects disable; September 21 retest read enabled=false before retrigger and after calculation 42 in the existing session. Fresh-session trigger did not return the fixture greeting. Shared web-demo follow-up observed two new invocations after disable, returning 42 and 17; a fresh demo session did not show fixture execution. Follow-up linked the shared-session initialization credential to the distinct user account U, resolving the earlier client-side identity ambiguity.

Evidence limits and acceptance criteria

Evidence limits

Bounded harmless text fixture, new invocation after stop, not in-flight cancellation. Persisted disabled state verified around execution. Shared-session initialization submitted the distinct user account key with the developer Agent share token; server internal attribution and intended disable scope remain unknown. Two post-disable shared invocations observed, not unlimited execution. Direct U-key CLI baseline was not established; separate-account installed-copy behavior remains open.

Acceptance criteria

Once disable is acknowledged, block every new invocation in existing and new conversations, including queued requests checked before execution. Update active tool lists/trigger routing, enforce server-side state at invocation, and return a structured disabled reason for clear conversational feedback. Define and test already-running cancellation separately.

X02 · P1 · open

Explicit no-send instructions fail across Gmail turns and a real email is delivered

Existing marketplace Gmail sent an email after a conversation-wide no-send instruction and a second turn requesting wording only and repeating do not send. Recipient Gmail independently verified sender, subject and Inbox delivery. Earlier single-turn negative cases entered Gmail without observed delivery; synthetic persistent-write and calculation cases remain separate evidence.

Evidence limits and acceptance criteria

Evidence limits

One real-email multi-turn reproduction with a separate positive control; no universal failure-rate, model-cause or cross-integration claim. Recipient metadata verifies delivery; sender Sent-folder/backend logs and Ability source were not inspected. Earlier basic and quoted negation cases entered Gmail but had no observed delivery in bounded checks. Synthetic storage readback was SDK-mediated; its writer intentionally omitted internal confirmation.

Acceptance criteria

Honor no-send/preview constraints through activation and follow-up turns. Recipient, subject and body are not send authorization. Positive control sends once; basic negative, quoted-command and exact two-turn preview cases send zero, verified against provider evidence. Require clear later authorization before sending; confirmation copy alone must not substitute for execution checks.

X03 · P2 · open

Document when code updates take effect in existing conversations

In the September 18 working-draft test, an existing conversation retained the older code marker after an upload verified by source download; a fresh conversation used the new marker. Keeping a revision for the life of a conversation may be intentional. The audit asks for an explicit documented revision strategy and visible loaded revision, not mandatory hot reload.

View cited public sources (4)
Evidence limits and acceptance criteria

Evidence limits

One September 18 working-draft update comparison; programmed revision markers are not platform release numbers. Backend mechanism and intended strategy remain unknown. Revision retention may be intentional and is not by itself evidence of a runtime defect or failed save. Current public docs checked September 22 do not specify web-chat revision selection. DevKit restart guidance concerns separate hardware behavior; published-release and physical-device update semantics were not independently tested here.

Acceptance criteria

Explicitly document when working-draft and published-code updates take effect, whether existing conversations retain their loaded revision, and when reconnect/reload is required. Expose the executing revision and verify behavior against that contract. Intentional session pinning is acceptable; immediate hot reload is not required by this finding.

X04 · P2 · open

One-shot returns acknowledgement and closes before later business-result message

One-shot returns acknowledgement and closes before later business-result message

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Separate message-final from invocation-complete; test multi-output Abilities.

X05 · P2 · open

404/429/503/JSON/network failures share missing-topic sentinel; empty 200 extract returns a period

404/429/503/JSON/network failures share missing-topic sentinel; empty 200 extract returns a period

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Distinct failure states, useful fallback, explicit deadline and appropriate bounded retry.

X06 · P2 · open

Loop template treats an exit-word substring as an explicit stop command

The pinned loop starter checks whether an exit word occurs anywhere in the user's text. With a synthetic SDK, “Tell me about nonstop flights” produced “Goodbye!” and zero model calls; the “Tell me about planets” control made one model call. resume_normal_flow ran once in each case. This is a specific starter-template bug, not a general conversation-agent or barge-in defect.

View cited public sources (1)
Evidence limits and acceptance criteria

Evidence limits

One negative input and one ordinary-request control against the unchanged loop-template at source revision 1416d065f9f77e0a0f04d34c3f46f4c809751492; no broader substring/quoted-word matrix, hosted Ability, live conversation, audio or hardware run. This is scoped to the starter-template exit check and does not establish platform-wide trigger or interruption behavior. No upstream fix is claimed.

Acceptance criteria

Use explicit exit intent or token/phrase boundaries. Add regressions for ordinary words such as “nonstop” and “unstoppable,” and for quoted mentions of exit words. Confirm ordinary requests reach the model while an intentional standalone exit still ends the loop.

X07 · P1 · open

Model exception escapes without resume_normal_flow(); related D05

Model exception escapes without resume_normal_flow(); related D05

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Finally-based cleanup plus hosted recovery test; backend permanent hang not proved.

Y01 · P1 · open

Privacy/locality claims need configuration and product scope

Broad local/private positioning needs the same mode-specific qualifications used in the Venice announcement. DevKit participation terms permit data collection/training; AI Twin has separate policies. This does not establish that every deployment sends all data or that fully local modes are unavailable.

View cited public sources (8)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Each entry point identifies default processing, vendors, stored data, training choices and deletion limits; verify each mode with data-flow tests.

Y02 · P1 · open

Open-source scope is not mapped to the full product

Public Ability/CLI source is real. Full firmware, hosted runtime and application source/license coverage is not established by the reviewed repositories or a testimonial.

View cited public sources (3)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Publish a component/repository/license map including proprietary and hosted dependencies.

Y03 · P2 · open

Speed claims lack comparable measurements

Instant responses and short setup timelines lack named conditions. The diagnostic text timings do not measure voice latency or disprove every optimized configuration.

View cited public sources (4)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Publish separately measured setup, first-text, first-audio and completion times with sample size, hardware, provider and network.

Y05 · P2 · open

Hardware breadth needs a supported configuration matrix

Broad Linux/device promises are wider than the named Raspberry Pi setup paths. Custom-Pi audio interruption limits and integration prerequisites should travel with the claim.

View cited public sources (7)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Versioned hardware/OS/audio/protocol matrix and representative device acceptance, with unsupported combinations marked.

Y06 · P2 · open

Feature and catalog counts have no auditable definition

Website totals, category totals and spotlight demos need a versioned catalog defining features, Abilities, integrations and readiness. Source-folder counts are not a valid rebuttal of a differently scoped feature count.

View cited public sources (5)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Export counted IDs and released status; link each advertised integration to a passing end-to-end test.

Y07 · P1 · open

JavaScript FAQ conflicts with the public Python path

Developer FAQ advertises JavaScript; the unchanged public contribution validator rejects a JavaScript-only fixture for missing Python files. No separate private JavaScript SDK was inspected.

View cited public sources (3)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Supply an end-to-end supported JavaScript path or correct and version the FAQ.

Y08 · P2 · open

DevKit update copy and restart guidance need a single scope

The September 18 rendered DevKit page records “instant synchronization without restart”; the current DevKit FAQ says changes to an active device, including Ability installs and configuration, require an Agent restart. This is a documentation conflict scoped to the DevKit update path, not a reproduced runtime failure or a claim about CLI/web text behavior.

View cited public sources (2)
Evidence limits and acceptance criteria

Evidence limits

The developer-page wording is preserved from a rendered September 18 review; the DevKit FAQ was checked September 22. This establishes a documentation-scope conflict only. No physical hardware update was tested, and the finding does not generalize to the CLI or web text client.

Acceptance criteria

Distinguish syncing/pushing an update from when an active DevKit Agent loads it. State the documented restart requirement for active devices; if no-restart applies to another mode, name that mode and support it separately. Do not use web-session tests as hardware evidence.

Y09 · P2 · open

Application review cadence contradicts itself

Developer hero/form says daily; FAQ says weekly.

View cited public sources (1)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

All current program touchpoints state the same owner-approved review cadence.

Y10 · P2 · open

DevKit revision and hardware claims need a named SKU

Hero says v0.1 while FAQ says V2; hardware specifications are not tied to a verified shipped SKU. This is ambiguous naming, not proof that advertised components are absent.

View cited public sources (1)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Name shipped SKU, hardware versus software revision, BOM and tested connectivity/audio features.

Y12 · P2 · open

Smart-home page hero retains robotics wording

The current home-automation page is headed “Smart Home & IoT,” but its hero says it is “Empowering the next generation of robotic solutions.” The extracted page text also repeats a home-automation heading; that duplicate has not been confirmed as a visible on-page defect.

View cited public sources (2)
Evidence limits and acceptance criteria

Evidence limits

The current hero wording was verified in page text on September 22; the robotics page is linked for context. Text extraction showed a repeated heading, but no fresh visual check established whether that duplication is visible in the rendered design.

Acceptance criteria

Use a smart-home-specific hero. Verify whether the repeated heading is visible at desktop/mobile before treating it as a separate defect.

Y13 · P2 · open

Developer offer and program versions need one current contract

Conditional free hardware, grants, credits, bounty availability and older tiered programs lack a single dated current offer. Historical differences do not prove unpaid awards or broken promises.

View cited public sources (4)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Publish current eligibility, award amounts, deadlines, stock, costs and terms; label historical announcements.

Y14 · P2 · open

Memory success and efficiency claims lack reproducible results

Memorable article reports an eight-task result and broader completion/cost improvements, but reviewed copy does not provide a reproducible experiment sufficient to validate general performance.

View cited public sources (1)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Publish task corpus, repetitions, baseline, versions, failures, tool calls, latency and token/cost records.

Y15 · P1 · open

CLI marketing routes new users to the archived product

Live CLI announcement recommends npm client plus JSON/MCP/session-token features. The supported current CLI is the separate Python client; do not transfer either implementation’s feature claims to the other.

View cited public sources (3)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Add historical-version banner and supported install link; maintain a versioned command/auth capability table.

Y16 · P1 · open

Ability tutorial API does not match public package

Blog teaches CapabilityWorker/register_capability imports from openhome; neither is exported by the supported public package. Current class-based templates use a different authoring contract. This does not prove an unpublished runtime cannot supply an alternative API.

View cited public sources (1)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Run the exact first-Ability tutorial in a documented clean environment through validation and hosted invocation.

Y17 · P1 · open

Bounty page states two payment amounts

The same bounty article promises $200 in introduction/payment sections and $250 in rules. Current payable amount was not independently established.

View cited public sources (2)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

One dated approved amount and eligibility contract; align article, form and program page.

Y18 · P2 · open

Precise industry outcome figures need source attribution

Smart-home article presents energy, hotel and hospital improvement percentages without adjacent attributable study details in reviewed text. These are industry assertions, not verified OpenHome customer outcomes.

View cited public sources (1)
Evidence limits and acceptance criteria

Evidence limits

Copy/source review unless otherwise specified; not a legal determination or proof of every deployment.

Acceptance criteria

Link original studies with population, dates and method, or replace exact figures with qualified qualitative statements.

W11 · P1 · open

Google OAuth warning undermines connection credibility

User screenshot shows Google’s unverified-app warning. The production connection journey needs appropriate OAuth verification and consistent product identity.

Evidence limits and acceptance criteria

Evidence limits

User observations and supplied screenshots are attributed; source inspection and initial browser rendering are separate from live integration/audio acceptance.

Acceptance criteria

Confirm production OAuth configuration and appropriate verification for requested scopes; clean-account consent has expected branding and no unverified-app interstitial.

W12 · P1 · open

Google partial consent ends in scope-mismatch failure and disconnected state

User reports token exchange rejected a changed scope set after selected permissions were granted; OpenHome still shows disconnected. A provider grant does not establish that OpenHome received/stored a usable token. Full error intentionally omitted.

Evidence limits and acceptance criteria

Evidence limits

User observations and supplied screenshots are attributed; source inspection and initial browser rendering are separate from live integration/audio acceptance.

Acceptance criteria

Test full and partial consent, previously granted scopes and cancellation; explain required/optional access, recover from mismatch and show connection state consistent with a verified usable token.

W13 · P2 · open

Discord connection exposes test-app branding

User reports Discord consent names the application openhome test app. Audit user-facing application names, provider consent, redirects, support identity and error copy for production-appropriate wording.

Evidence limits and acceptance criteria

Evidence limits

User observations and supplied screenshots are attributed; source inspection and initial browser rendering are separate from live integration/audio acceptance.

Acceptance criteria

Discord consent uses the intended production brand; audit other integration and account surfaces for leftover test/development copy.

W14 · P2 · open

Slack consent shows an approval warning despite working integration

User screenshot shows Slack’s app-not-approved notice; user reports the integration works. This is a distribution/credibility concern, not evidence of broken functionality.

Evidence limits and acceptance criteria

Evidence limits

User observations and supplied screenshots are attributed; source inspection and initial browser rendering are separate from live integration/audio acceptance.

Acceptance criteria

Confirm intended distribution/review path, address approval as applicable and explain expected warnings; preserve verified installation and functional integration behavior.

D16 · P2 · open

Supplied audio-stream example has lifecycle and recovery defects in source

Unchanged HTML renders in Chrome with a placeholder key. Source shows unconditional reconnect after close, a playback/active-state race, an out-of-scope buffer variable and uncleared ping intervals. No microphone access or live call was attempted.

Evidence limits and acceptance criteria

Evidence limits

User observations and supplied screenshots are attributed; source inspection and initial browser rendering are separate from live integration/audio acceptance.

Acceptance criteria

With isolated credentials and explicit microphone testing, verify start/deny/retry/end/restart, no reconnect or active tracks after End, correct playback initialization, quota recovery and no timer leaks.

D17 · P2 · open

Local validator accepts constructs rejected by hosted validation

The public validator accepted the SDK fixture; hosted upload rejected inspect and getattr with HTTP 400. This is validator parity friction, not evidence those constructs should be allowed or that SDK methods fail. Fixture updated to use explicit documented calls.

View cited public sources (2)
Evidence limits and acceptance criteria

Evidence limits

Observed with the evaluation fixture; no request to weaken hosted sandbox policy. The accepted fixture later had an unrelated reply timeout.

Acceptance criteria

Local preflight rejects the same forbidden constructs as hosted upload with a documented runtime/validator version and actionable location; preserve sandbox restrictions.

D18 · P3 · open

API sidebar omits HTTP-method badges on Upload Ability and Get Ability

The supplied screenshot shows Upload Ability and Get Ability without HTTP-method badges while adjacent endpoints have badges. The Upload Ability card displays POST, and the retrieved reference identifies Upload Ability as POST and Get Ability as GET. This is inconsistent navigation metadata, not evidence of endpoint failure.

View screenshot evidence (1)
View cited public sources (1)
Evidence limits and acceptance criteria

Evidence limits

Visual evidence is the user-supplied screenshot; reference text was retrieved on September 18, but no fresh browser rendering or endpoint invocation was performed for this finding. Screenshot capture date is not independently established.

Acceptance criteria

Upload Ability shows POST and Get Ability shows GET in endpoint navigation, matching the reference and authoritative API contract. Check every endpoint entry for consistent labels and method badges in desktop and narrow-screen navigation; retain readable method text for assistive technology.

W15 · P2 · open

Onboarding tour controls lose readable contrast across hover/control states

Gabriel reports a hover-text problem in the product tour. Two supplied screenshots show Skip nearly disappearing against white in one state and Next appearing very pale in the other. The guided tour itself is a positive feature (K04).

View screenshot evidence (2)
Evidence limits and acceptance criteria

Evidence limits

The screenshots establish the low-contrast appearances, but not the exact pointer or disabled state. No computed contrast ratio or fresh interaction test was captured.

Acceptance criteria

Check the real normal, hover, focus, active and disabled states of Skip, Back and Next. Keep available actions readable with appropriate text contrast, visible focus and an unambiguous disabled state; preserve the tour and skip path.

W16 · P2 · open

Interruption settings need plain-language explanations

The settings screenshot shows Interactive Interruption, Auto Interruption and Interruption Sensitivity without adjacent explanations. The user could not tell what each option changes or how the controls interact.

View screenshot evidence (1)
Evidence limits and acceptance criteria

Evidence limits

The screenshot shows controls and their displayed state, not their underlying semantics. Do not invent definitions or attribute the later interruption to a specific setting without a controlled test.

Acceptance criteria

Add accessible help that defines each control using the product-confirmed behavior: whose speech can be interrupted, what triggers interruption, how sensitivity affects it, defaults and interaction between settings. Include a short example and a reversible way to compare settings.

W17 · P1 · open

Conversation display reordered user and assistant messages

The user reports saying the wake phrase after the sleep guidance and then seeing the thread reorder. The screenshot places an assistant response to that phrase above the user bubble containing it, making the conversation appear out of sequence.

Evidence limits and acceptance criteria

Evidence limits

Screenshots show reply placement and the user reports reordering. No continuous transition capture, correlated event IDs or persisted-history comparison establishes where ordering changed.

Acceptance criteria

Keep stable message IDs and chronological turn ordering during streaming, finalization, wake-up, reconnect and reload. Reproduce with event timestamps and compare visible order to the recorded turn sequence; never move an earlier user turn after its reply.

W18 · P1 · open

An earlier displayed user message appeared to be rewritten

Gabriel reports that the thread subsequently rewrote a displayed user message, changing it from the wake phrase to a later request. The available static screenshots are ambiguous and do not establish that both displays refer to the same message; this remains a user-reported transcript-stability concern pending stable-identity evidence.

Evidence limits and acceptance criteria

Evidence limits

Static screenshots and user report do not establish that the same stored message was overwritten or distinguish provisional transcription from repeated speech. No stable message IDs, continuous timeline or persisted-history comparison was captured.

Acceptance criteria

Tie transcript updates to a stable utterance/message ID. Restrict provisional transcription updates to the current utterance and label them; preserve earlier finalized text when later speech arrives. Verify display and stored history after reload.

W19 · P1 · open

Image creation unexpectedly submitted a pending message

The user reports that creating a photo while a message was waiting to be sent caused the message to send automatically, without making that consequence clear beforehand.

Evidence limits and acceptance criteria

Evidence limits

User report only. The draft, image creation and send transitions were not recorded or independently reproduced; exact payload and intended interaction contract remain to be established.

Acceptance criteria

Before image generation/attachment completes, make any send behavior explicit. For a draft composer, retain text and the generated image for preview with an explicit Send action; if immediate send is intended, state that in the initiating action. Test generation success, cancellation and failure with a pending draft.

W20 · P3 · open

Sidebar hover colors are inconsistent for AI Twin

The user reports an orange hover state for AI Twin while other sidebar entries hover white. The difference appears unexplained in the navigation.

Evidence limits and acceptance criteria

Evidence limits

User report only; no sidebar hover screenshot or style inspection was captured. A documented intentional navigation distinction has not been ruled out.

Acceptance criteria

Apply consistent hover, selected and keyboard-focus styling across equivalent sidebar links; if AI Twin intentionally has a different role, make that distinction clear and verify readable contrast.

W21 · P2 · open

Agent cards show a pointer cursor outside the working click target

On the Agents page, the user reports that agent items show a pointer cursor suggesting the card is clickable, but only the Chat button responds.

Evidence limits and acceptance criteria

Evidence limits

User report only; no click-event trace or fresh test of card-body versus button interaction was captured.

Acceptance criteria

Match the pointer and hover affordance to the actual clickable area, or make the intended card action work consistently. Keep nested controls distinct and verify keyboard, mouse and touch behavior.

W22 · P1 · open

System/status messages appear to contaminate the conversational output stream

The user reports system messages appearing in the output stream as conversation content. Sleep/wake guidance is visible in ordinary chat bubbles in the supplied transcript, but whether system text also enters model context or synthesized speech is not established.

Evidence limits and acceptance criteria

Evidence limits

Visible sleep/wake status text shares the conversation stream. This does not establish hidden-prompt leakage, model-context contamination or synthesized speech of system text.

Acceptance criteria

Give lifecycle/status events an explicit event type and visual treatment distinct from user and assistant turns. Test rendered transcript, model context, exported history and speech independently so intended guidance is preserved without being misattributed or injected into ordinary reply content.

W23 · P1 · open

Automatic interruption cut in while the user was still giving a long response

September 21 live browser testing: Gabriel repeatedly reports premature replies during natural pauses, including with Interactive Interruption off. A longer utterance succeeded with both switches off and high sensitivity. Settings were not isolated consistently; some short transcript fragments were reportedly never spoken.

View screenshot evidence (2)
Evidence limits and acceptance criteria

Evidence limits

See VOICE_SESSION_REVIEW_20260921.md for the dated live follow-up and screenshot provenance. Repeated user-observed cutoffs and visible interleaved replies support the experience issue. No synchronized audio timeline identifies exact pause lengths or cause. Auto Interruption off is a candidate workaround, not an isolated causal result.

Acceptance criteria

Reproduce with long speech, short pauses, hesitation and varied background noise; record speech activity, end-of-turn detection, timeout and interruption events. Respect ongoing user speech and provide recovery for a premature cutoff. Explain and test the applicable interruption settings.

W24 · P2 · open

Test email reported sent but arrived in recipient Spam

Existing Gmail Ability reported “Done! Email sent.” Independent Gmail API metadata verified the expected sender, recipient and subject, but the message carried the SPAM label. Successful sending did not establish inbox placement. A later negative-case email reached Inbox, so observed placement varied across the two tests.

Evidence limits and acceptance criteria

Evidence limits

Two synthetic test messages between user-owned Gmail accounts: the positive control had SPAM label; the later negative-case message had INBOX label. Cause and prevalence remain unknown; no demonstrated attribution to OpenHome versus reputation, content, authentication or recipient filtering.

Acceptance criteria

Investigate authentication, sender reputation, content and recipient filtering using message headers and representative owned-mailbox tests. Distinguish send acceptance from inbox delivery in product language and support diagnostics; do not promise inbox placement without evidence.

W25 · P2 · open

Negative email request gets a readiness prompt instead of acknowledging restraint

For “Do not send mail … Do not send this email,” the existing Gmail Ability responded “Gmail ready. What would you like to do?” No matching negative-case email was observed in two bounded mailbox checks, but the response did not acknowledge the instruction to refrain.

Evidence limits and acceptance criteria

Evidence limits

Basic case response and bounded recipient absence are independently documented. A subsequent multi-turn preview case did send despite repeated negative instructions (X02); better acknowledgement alone is not sufficient protection.

Acceptance criteria

Respond “Understood—I won’t send that email.” Preserve the no-send constraint through follow-up turns, avoid entering a send workflow solely because its trigger was mentioned, and require a clear later authorization before sending.

W26 · P3 · open

Unexplained Test message appears in conversation

User reports receiving an unexpected “Test” message while starting the walkthrough. Screenshot shows a standalone left-aligned Test bubble, giving the experience the appearance of unfinished placeholder copy. User subsequently clarified that it appeared when microphone access began; that timing is user-reported, not event-traced.

View screenshot evidence (2)
Evidence limits and acceptance criteria

Evidence limits

Visible text is screenshot-backed; arrival is user-reported. No event trace distinguishes new output, restored history, Agent configuration or debug/default text. Not evidence of a pre-production deployment or another ordering defect.

Acceptance criteria

Identify the message source and remove unintended test/default text. Give legitimate status events meaningful text and explicit attribution.

W28 · P1 · open

Narrative and exit phrases activate Alarm without a present setup request

Repeated user speech screenshots show stories and exit requests mentioning alarm activating Alarm, followed by a time question or no-change response. Later topic changes successfully resume ordinary conversation.

View screenshot evidence (4)
Evidence limits and acceptance criteria

Evidence limits

Unwanted activation is screenshot-backed. Persistent lock-in is not supported. ABORT is visible in one supplied socket log; no universal routing algorithm or unintended alarm creation is inferred.

Acceptance criteria

Distinguish narrative, negation, cancellation and current intent before activation. Return to conversation cleanly and preserve successful explicit relative-time alarm requests.

W29 · P1 · open

User reports unspoken words appearing as speech input

Gabriel reports So, hey and okay appearing without speaking; supplied screenshot displays a User So bubble and interruption indicators.

View screenshot evidence (1)
Evidence limits and acceptance criteria

Evidence limits

Text is screenshot-backed; absence of speech is user-reported. No recorded input isolates recognition, feedback, ambient sound or stale-message causes.

Acceptance criteria

Test controlled silence, speaker-muted and speaker-enabled conditions with audio/transcript correlation. Do not create finalized user turns from non-speech.

W30 · P2 · open

Long speech-response waits lack understandable progress and recovery state

Gabriel reports waits over 30 seconds during alarm-exit attempts without clear processing feedback. Screenshots show a User marker but do not explain whether input is provisional, accepted, processing or failed.

View screenshot evidence (1)
Evidence limits and acceptance criteria

Evidence limits

Duration is user-reported. No request-correlated timing establishes backend latency. WebSocket use and unknown-log console warnings are not themselves proof of failure.

Acceptance criteria

Distinguish listening, transcription, processing, completed and failed states. Provide bounded waiting feedback and recovery without forcing repeated speech.

W31 · P1 · open

Reported turn blending: replies accumulate while the user's spoken turn continues

Gabriel reports premature replies during non-terminating pauses, followed by inadequate yielding when he resumes. Several agent replies and speech starts appear while transcription extends the same user message, confusing turn ownership and transcript meaning.

Evidence limits and acceptance criteria

Evidence limits

User-reported sequence; no synchronized audio or backend trace. ASR/STT produces input transcription; TTS produces output speech. Overlapping generation and trigger mechanisms are hypotheses. The available static story screenshots are ambiguous and do not clarify turn blending or establish message identity, provisional/final status, whether speech repeated, or Agent-generation timing; they are excluded from the client-facing gallery. Overlaps W23; not a separate proven root cause.

Acceptance criteria

Correlate audio, provisional/final transcripts, turn/generation IDs and playback for incomplete thoughts, natural pauses and resumed speech. Keep listening until the user yields; promptly stop obsolete output when speech resumes; prevent duplicate replies and preserve response-to-input-version attribution. Establish measured timing criteria from the baseline.

W32 · P2 · open

Separate conversation threads are not discoverable within a personality

Gabriel reports one text history per personality with no separate-thread workflow in the tested UI. Unrelated topics accumulate, parallel work and retrieval become awkward, and a fresh-conversation recovery path is unclear. Users cannot easily understand the boundary between conversation context and personality memory.

Evidence limits and acceptance criteria

Evidence limits

User observation of the tested interface; no exhaustive navigation review or backend limitation established. One screenshot cannot establish global feature absence. Actual model-context reuse is unverified.

Acceptance criteria

Expose New conversation and a named conversation list, or make an existing equivalent discoverable. Create two independent histories under one personality, switch and reload without mixing or losing them. Explain conversation reset versus persistent personality memory.

W33 · P3 · open

Wide desktop gutters waste conversation space and weaken perceived polish

The supplied screenshot shows broad empty gutters beside the conversation panel, between left navigation and right settings. Gabriel finds the layout inefficient and less premium. Improve space allocation while retaining readable message line lengths.

View screenshot evidence (1)
Evidence limits and acceptance criteria

Evidence limits

Visible spacing is screenshot-backed; perceived premium quality is user feedback. No task blockage, measured productivity loss or cross-viewport defect was established. Capture time is unverified; September 22 is the report date.

Acceptance criteria

Reduce excessive outer gutters with balanced responsive panel sizing. Check wide/narrow viewports, settings open/closed, zoom, overflow and keyboard access; retain readable prose widths and accessible settings.

W34 · P2 · open

New Agent response exposes raw Markdown markers

A supplied screenshot of a new Agent shows literal heading markers, asterisks and list separators in the visible reply. This makes the answer harder to scan and can expose formatting syntax as content. The screenshot establishes this response only, not prevalence or cause.

View screenshot evidence (1)
Evidence limits and acceptance criteria

Evidence limits

One user-supplied screenshot establishes literal Markdown syntax in one Agent response. No broader prompt matrix, rendering-path inspection or root-cause test was performed. A system-level instruction is a recommendation, not a verified fix.

Acceptance criteria

Define a stable output-format contract independent of user-supplied prompts; render supported Markdown in the chat UI or otherwise present its structure legibly. Treat system-level formatting instructions as a guardrail, not a substitute for rendering. Test headings, bold text, numbered steps and bullets in visible text and spoken output, including user prompts that request conflicting formatting.

W35 · P1 · open

Markdown-heavy replies are reported as difficult to understand in speech

Gabriel supplied three audio examples and reports that the speech output does not turn Markdown-heavy reply text into comprehensible spoken language. The set includes one approximately 55-second clip and two clips of approximately 8 and 7 seconds. These user-supplied examples warrant a core conversation-quality investigation; this pass did not independently replay or transcribe them.

Listen to supplied audio evidence (3 clips)
Evidence limits and acceptance criteria

Evidence limits

The three clips and symptom description are supplied by Gabriel. The files' container metadata indicates approximate durations of 8.1, 55.1 and 7.3 seconds; duration does not establish what is audible. This pass did not independently replay, transcribe or correlate the audio with source text, TTS input or provider events, so prevalence and cause are unknown.

Acceptance criteria

Run a controlled text-and-speech matrix for headings, emphasis, numbered steps, bullets, links and long answers. Normalize Markdown before speech generation so syntax tokens are not read aloud and hierarchy becomes natural phrasing/prosody; preserve meaning and keep spoken output aligned with visible text. Correlate source response, normalized TTS input, audio chunks and playback, including long responses and interruptions.

W36 · P2 · open

One supplied long speech sample reportedly starts with a distorted voice

Gabriel reports a severely distorted voice at the beginning of the approximately 55-second Markdown-to-speech example. Keep this onset-quality symptom distinct from the broader Markdown intelligibility concern until the initial audio segment and later speech are compared.

Listen to supplied audio evidence (1 clip)
Evidence limits and acceptance criteria

Evidence limits

One user-supplied clip and Gabriel's description support a reported onset symptom only. No independent listening, repeated trial, waveform inspection or provider/playback trace establishes the distortion's extent, recurrence or cause.

Acceptance criteria

Compare the first spoken segment with the remainder of the same response and repeat across controlled sessions. Correlate selected voice, synthesized chunk boundaries, encoding and playback timestamps; define an intelligibility/voice-quality check before attributing the symptom to any layer.

W27 · P2 · open

Typed pause while asleep omits the live user turn; reload restores it

Browser testing reproduced the user's missing-turn observation: sending pause while asleep displayed an acknowledgement without the user bubble, while the awake control displayed both. Reload restored the missing pause in its correct position. The confirmed impact is an incomplete live transcript, not lost stored input.

Evidence limits and acceptance criteria

Evidence limits

September 21 browser validation reproduced the missing live user bubble when pause was submitted while asleep; the awake control displayed both turns. Reload and Activity reopening restored the missing pause ahead of its acknowledgement. This supports a live-display omission, not stored-message loss. One asleep/awake pair was tested; backend cause and speech behavior remain unverified.

Acceptance criteria

Represent accepted typed input in the transcript, or clearly represent it as a user-initiated control action if intentionally handled outside ordinary chat. Verify awake and asleep cases, event attribution and history after reload; do not silently omit the action while showing its acknowledgement.

Observed and reported behavior

W01 · — · unverified

Unverified observation: private selection may not reflect persisted state

Unverified observation: private selection may not reflect persisted state; no exposure shown

Evidence limits and acceptance criteria

Evidence limits

The controlled 2026-09-15 run reported a misleading privacy/readback contract while server flags remained private and unsubmitted; it did not demonstrate public exposure. A 2026-09-21 read-only check found the original trial Agent absent, and the accessible replacement Agent was not an exact-fixture retest. This observation is excluded from the current reportable findings until reproduced.

Acceptance criteria

Reopen the original trial fixture and reproduce the exact save/reload mismatch before restoring reportable status.

W02 · — · unverified

Unverified observation: ordinary turns may display stale Ability activity cards

Unverified observation: ordinary turns may display stale Ability activity cards; routing cause unproved

Evidence limits and acceptance criteria

Evidence limits

The controlled 2026-09-15 run reported stale Ability cards after ordinary turns, but no server execution trace established a routing-engine failure. A 2026-09-21 read-only check found the original fixture absent; a different existing conversation showed no cards, and no new message was sent. This observation is excluded from the current reportable findings until reproduced.

Acceptance criteria

Reopen the original trial fixture and reproduce the stale-card sequence before restoring reportable status.

W05 · — · explained

Earlier Agent onboarding mismatch is no longer present in the current public guide

On September 15, the inspected UI offered Design Agent with AI or Pick from a Template, while the then-reviewed guide described Quick Creation, Pro Creation and AI Twin. The current Quickstart and Creating an Agent pages now describe the same two entry paths as the captured UI. This explains the documentation-side mismatch; the live UI was not rechecked on September 22.

View cited public sources (2)
Evidence limits and acceptance criteria

Evidence limits

The September 15 UI evidence and September 22 public guides are from different dates. The current interface was not re-opened, so this does not prove the whole onboarding journey is now accurate or successful. D12 remains a separate, current conflict about required fields.

Acceptance criteria

Keep as an explained historical observation unless a fresh same-day comparison shows the app and current guide diverge again; then record both versions and the exact path.

W06 · — · explained

Transient tour-control stall during initial conversation loading; reopening recovered

In one first-use session, Skip and Next did not respond while the conversation showed Loading. Reopening Chat recovered; Skip worked once connected. This is a bounded recovery observation, not evidence of a persistent tour defect.

Evidence limits and acceptance criteria

Evidence limits

One transient in-app-browser session; reopening recovered and Skip worked after connection. No repeat, persistent failure, or controlled delayed-connection test establishes a systemic defect. Do not count as an open product issue.

Acceptance criteria

If the stall recurs, preserve a usable Skip action while connecting and provide clear cancel/retry behavior for a genuine setup failure.

W09 · — · unverified

Selected Agent voice was not discoverable in the separate voice catalog

Agent creation showed Matilda as the selected voice, while a completed search on the standalone Voices page returned no result in its My, OpenHome or Public groups. This is a cross-surface discoverability question, not evidence that Matilda is missing, unavailable or incorrectly assigned.

Evidence limits and acceptance criteria

Evidence limits

One selected voice and one standalone search. No product contract promises the two catalogs are identical, and no evidence shows the selected voice cannot be used. Keep as an unverified UX observation rather than a confirmed issue.

Acceptance criteria

Clarify whether the Agent selector and standalone Voices page draw from different catalogs. If they do, label the source and link selected-voice details; if they share a catalog, make selectable voices searchable.

C01 · — · explained

Comparative gap: CLI lacks first-use diagnostics

Compared with mature coding CLIs, the reviewed OpenHome CLI lacks root build identification, general readiness and authentication-status commands.

Evidence limits and acceptance criteria

Evidence limits

This is a comparative product-gap observation against mature coding CLIs, not a broken advertised OpenHome feature. The reviewed build's parser behavior was reproduced, but current replacement releases were not tested.

Acceptance criteria

Offer concise first-use guidance plus separate build, read-only readiness and credential-status surfaces without exposing secrets.

C02 · — · explained

templates --json observation folded into Y15; not a separate broken CLI contract

The pinned Python CLI rejected templates --json, but the supported Python CLI does not promise that flag. The older CLI article advertises JSON/MCP capabilities for a different product generation; that version-routing mismatch is Y15. Keep this probe as supporting evidence, not a separate product defect. A general machine-readable-output interface remains a proposed enhancement.

View cited public sources (2)
Evidence limits and acceptance criteria

Evidence limits

One command was tested at the pinned Python CLI revision. Other command output contracts were not comprehensively surveyed. Current documentation describes this Python CLI but does not promise templates --json.

Acceptance criteria

If machine-readable output is prioritized, specify a versioned schema and test it separately; do not describe the current absence as a regression unless a supported contract is published.

X08 · — · explained

Platform renames class to account Ability name; other bytes match after that exact substitution

Platform renames class to account Ability name; other bytes match after that exact substitution

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Document generated-source transformation and preserve meaningful sync differences.

Y04 · — · unverified

Reviewed public pages do not show scope or results for broad quality claims

The reviewed pages make broad claims about emotional insight, automatic support for 30+ languages and flawless Ability scenarios, but do not present linked evaluation scope or results. This is a public-substantiation observation only: it does not establish that OpenHome lacks internal tests or that the claims are false.

View cited public sources (7)
Evidence limits and acceptance criteria

Evidence limits

The cited public pages were reviewed; internal evaluation plans or results were not requested or available. This does not establish that internal testing is absent, nor does this review independently evaluate the claims.

Acceptance criteria

If the claims are retained, optionally link a concise evidence note stating supported configurations, evaluation scope, results and limitations; otherwise qualify the public wording. Do not imply internal testing is absent without checking with the product team.

Y11 · — · unverified

Local Link copy could scope its local-execution claim more explicitly

The article describes a Mac/local-computer command-execution template and says there are no cloud dependencies for execution. Current Local Connect documentation describes a CLI Local Link bridge on the user's computer. This supports local command execution, not a conclusion about end-to-end voice/control transport, dedicated DevKit hardware, or other OpenHome clients. Optional copy clarification could make that boundary explicit; no contradiction or runtime failure was established.

View cited public sources (4)
Evidence limits and acceptance criteria

Evidence limits

The source describes Local Link as a Mac/local-computer template; current Local Connect docs describe a CLI bridge. No bridge was started and no live traffic was captured. This is an optional copy-clarity observation, not a finding that the claim is false, a reproduced data-flow/security failure, or evidence about dedicated DevKit hardware, web chat, or every deployment.

Acceptance criteria

Optional clarity improvement: state that the cited Local Link template executes commands on the user's Mac, and clarify separately if the article intends to make a claim about the voice/control transport or end-to-end data path. Do not infer behavior for DevKit hardware or web conversation from the CLI bridge.

Evaluation tooling notes

T01 · — · open

The hardcoded “revision two verified” fixture phrase sounds like independent verification

The hardcoded “revision two verified” fixture phrase sounds like independent verification; it is not a platform certification feature

Evidence limits and acceptance criteria

Evidence limits

The original wording and detailed report delimit the evidence; no upstream fix is claimed.

Acceptance criteria

Use neutral fixture speech and a separate explicit expected-response assertion. Proposed; hosted wording unchanged.

T02 · — · explained

SDK activation timeout was caused by undeclared fixture state

Declaring MatchingCapability state fields resolved the fixture's startup failure. Corrected assertions and protocol assumptions allowed the foreground suite to complete. The original timeout is not an SDK method failure.

Evidence limits and acceptance criteria

Evidence limits

Evaluation-tooling corrections, not an upstream product fix. Earlier receipts are preserved; later corrected checks determine current statuses.

Acceptance criteria

Preserve these bounded contracts in repeatable regression checks; expand coverage using the per-method prerequisites.