The complete client-safe index: 100 entries across strengths, coverage, observations and open findings. Priority, status and evidence class keep the limits of each entry visible.
How to read this index
Entries include their canonical ID, title, summary, credibility lenses and—where present—evidence limits and acceptance criteria. Local evidence files and private registry exports are not linked; public source pages are available in each finding’s disclosure. Confidence comes from each entry’s status and evidence class.
The bounded API-key workflow completed Agent/Ability creation, push, text invocation, edit, commit, retrigger and source sync. This gives developers a working foundation for iteration. Preserve the successful loop while clarifying session-specific enablement and revision state.
Evidence limits and acceptance criteria
Evidence limits
Verified in the bounded API-key/text journey. Disable/re-enable success in fresh sessions does not establish existing-session behavior; X01 records the separate active-session limit. Native audio and hardware were not accepted.
Acceptance criteria
Trial journey, live evidence. Preserve as regression acceptance for future fixes.
K02 · — · verified in scope
Working: hosted trial returns the expected scripted greeting; normal conversation resumes
Working: hosted trial returns the expected scripted greeting; normal conversation resumes
Evidence limits and acceptance criteria
Evidence limits
The greeting is hardcoded fixture output, not independent platform verification or model correctness. Six expected-response checks are a small client-side sample, not production latency percentiles or audio acceptance.
Acceptance criteria
Timing evidence. Five Ability replies and one ordinary reply matched expectations; not audio or production reliability proof.
K04 · — · verified in scope
Positive: a contextual onboarding tour introduces conversation controls
The supplied screenshots show a guided tour explaining how to start and stop a conversation, with progress indicators and Back, Next and Skip controls. Gabriel explicitly values having this guidance.
View screenshot evidence (2)
Tour state A. The tour introduces conversation controls and provides Back, Next and Skip. User-supplied screenshot; capture date is not independently verified.Tour state B. The tour includes progress indicators and navigation controls. User-supplied screenshot; capture date is not independently verified.
Evidence limits and acceptance criteria
Evidence limits
Presence and the displayed instruction are screenshot-backed; usefulness is the user's assessment. Successful completion, adoption impact and full accessibility were not tested.
Acceptance criteria
Preserve the contextual explanation, progress and skip/back/next flow while fixing readability. Verify completion, dismissal, reopening and keyboard access; measure whether new users reach their first conversation without assistance.
K05 · — · verified in scope
Positive: sleep-mode guidance gives the user a way to resume
The conversation visibly tells the user to call the Agent's name to wake it from sleep mode. The user specifically appreciated the guidance on how to reawaken it.
Evidence limits and acceptance criteria
Evidence limits
The guidance is visible and was valued by the user. This does not prove that the attempted phrase was the correct wake phrase or that wake-up recognition worked reliably.
Acceptance criteria
Preserve actionable wake-up guidance as a clearly identified status message. State the actual accepted wake phrase or action for the current Agent and verify successful recovery without requiring the user to guess.
40 of 43 methods exercised: 18 working in bounded checks, 22 partially tested and 3 blocked by remaining prerequisites. The corrected foreground suite completed 39 command/recovery cases; instrumented timing samples are available. Synthetic audio is not microphone/speaker acceptance.
Evidence limits and acceptance criteria
Evidence limits
Partial results and three prerequisite blockers are explicit; no universal SDK certification or native audio acceptance.
Acceptance criteria
Preserve these bounded contracts in repeatable regression checks; expand coverage using the per-method prerequisites.
K07 · — · verified in scope
Relative-time alarm and conversational recovery succeeded in user testing
Gabriel reports a one-minute-from-now request rang about one minute later at 11:08 p.m. A separate screenshot shows a later flooring exchange receiving an ordinary reply after an Alarm interaction; it does not show or establish that the reported one-minute alarm rang.
Evidence limits and acceptance criteria
Evidence limits
Single user-reported timing trial; not a measured timing guarantee or certification of the separate Timer Ability. Recovery screenshots rule out a consistent lock-in claim.
Acceptance criteria
Preserve relative-time scheduling and easy topic recovery while correcting unwanted activation.
Coverage confirmed
K03 · — · verified in scope
Covered: 81 indexed docs and public repository map; supported CLI is Python, separate from archived TypeScript CLI
Unverified: native mobile, microphone/audio quality, interruption, hardware and Local Link execution
Evidence limits and acceptance criteria
Evidence limits
Synthetic audio recording/playback and empty stream lifecycle now pass bounded transport checks. Physical microphone/speaker quality, active interruption behavior and hardware remain unverified.
Acceptance criteria
Trial limits. Native iPhone access was blocked; responsive web is not native-app acceptance.
G02 · — · unverified
Unverified: server stage latency, broad reliability, production percentiles, longer/generative Ability performance
Unverified: server stage latency, broad reliability, production percentiles, longer/generative Ability performance
Evidence limits and acceptance criteria
Evidence limits
New instrumented SDK timing samples distinguish in-method time from client roundtrip. They do not establish server-stage traces, percentiles or broad reliability.
Acceptance criteria
Latency review. Obtain correlated traces and a broader sample; six short turns are diagnostic only.
Product findings
D01 · P1 · open
Source migration path can delete unpushed user code; destructive path not executed
Docker validation completed the published wheel install in a fresh virtual environment, including pyaudio, because the image already had PortAudio/compiler prerequisites. This does not establish a universal OS failure or a complete install matrix; no upstream fix is claimed.
Acceptance criteria
Demo limitations explicit; production requirements separately tested.
D03 · P1 · open
API-key/JWT guidance lacks a clear conditional contract; key-only path passed
Key-only create/update/commit passed in the tested account and deployment. This review did not establish that JWT is never required on an older or differently configured server; no such server was tested.
Acceptance criteria
Document API-key-only as the tested baseline and identify any server/version cases where JWT is needed; key-only and expired-JWT recovery paths are separately actionable.
D04 · P1 · open
Background Ability package contract is unclear; validator requires main.py
Current evidence confirms a local documentation/validator contract mismatch, not a background runtime failure. Hosted promotion and the official asyncio example were not retested in this pass; no upstream fix is claimed.
Acceptance criteria
Background Ability docs, validator and promotion checks agree on required files; a documented background-only fixture passes applicable checks without an undocumented main.py shim.
D05 · P2 · open
Basic starter omits required cleanup on error; runtime symptom untested
Read-only HTTP/tree checks establish path availability, not that replacement instructions are equivalent. Most lowercase replacements resolved; the Local replacement remained unresolved. The separate examples repository was unavailable; no claim is made that every template or integration is broken.
Acceptance criteria
Every documented source target resolves to the intended current example or has an explicit tested replacement; Local guidance identifies the supported bridge or device target.
D07 · P1 · open
Recommended wheel lacks starter assets; artifact and lookup reproduced
Docker validation completed the published wheel install in a fresh virtual environment, including pyaudio, because the image already had PortAudio/compiler prerequisites. This does not establish a universal OS failure or a complete install matrix; no upstream fix is claimed.
Acceptance criteria
Tested install matrix and identifiable build revision.
Current public pages and the pinned template still disagree about ability.md versus category/trigger metadata. A read-only hosted text matrix against the existing harmless fixture tested an unrelated control, an exact trigger, a sentence containing the trigger, an explicit negation, a longer-word near match and a near miss; the trigger-containing cases activated the fixture. This establishes bounded text behavior, not the server algorithm, audio behavior, disabled-state behavior or a universal routing rule; no upstream fix is claimed.
Acceptance criteria
One documented contract with positive/negative routing tests.
D10 · P2 · open
Parameter names, sync/async and constructor instructions disagree
The current Quickstart says name, description and category are the minimum to save; the current Creating an Agent guide says only name and description block saving. The public docs conflict, while the app's minimum-field behavior was last checked on September 15.
Both public pages were checked on September 22. The live form behavior was checked on September 15, not re-tested in this pass; this establishes a current documentation conflict, not current UI behavior or a failed save.
Acceptance criteria
Clean-account minimum Agent creation matches guide; related W05.
D13 · P2 · open
“Any OpenRouter model” promise conflicts with another page's fixed selection contract
In the inspected narrow Settings view, five icon-only navigation buttons had no accessible names; the voice-creation public-use toggle exposed no aria-pressed or aria-checked state. Desktop tab labels were present. The API Keys tab exists, and Profile also provides a Copy API key shortcut.
Evidence limits and acceptance criteria
Evidence limits
Bounded observation of five Settings controls and one voice-creation toggle, not a full accessibility audit. The earlier claim that the API Keys tab was absent is withdrawn; the documented route exists.
Acceptance criteria
Give each Settings navigation control a programmatic name at narrow and desktop breakpoints; expose the public-use control as a correctly named switch with its current checked state. Verify keyboard operation and screen-reader output for these controls, without implying a full accessibility audit.
W08 · P2 · open
CLI own-Ability inventory differs from web defaults; effective runtime scope unclear
The shipped README describes create as auto-pushing to the account, while the reviewed source skips upload when triggers are absent, prints a prose hint and exits 0.
The CLI tells an interactive user that the no-trigger run was not pushed, so this is a contract and automation ambiguity rather than proof of a hidden upload failure or data loss. No authenticated upload was attempted in this offline reproduction; the triggered path stopped at the expected missing-credential check.
Acceptance criteria
Align create documentation, help and implementation; return an explicit local-only versus uploaded result and documented exit semantics that automation can evaluate without parsing prose.
C05 · P3 · open
CLI root help mislabels text chat as an interactive voice session
The captured root help describes chat as an interactive voice session, but the tested command exchanges typed text; call is the separate microphone-and-speaker path. The current CLI guide explicitly distinguishes text chat from voice calls, so the mismatch is localized to the CLI help wording.
The raw CLI help and text-session behavior were captured at the pinned source revision; the current guide now states the distinction clearly. The issue concerns help terminology, not missing audio capability.
Acceptance criteria
Describe chat as typed text, call as microphone/speaker voice, and local as the bridge, matching their actual behavior.
C06 · P2 · open
Call source prints no-response text then returns success; live timeout branch untested
Offline probes of unchanged cmd_chat reproduced stale suffix text, including the reported dy. pattern, and omission of differing final content after a live update. The careful hosted run captured clean live/final pairs; original overlap and duplicate greeting were not independently reproduced.
Evidence limits and acceptance criteria
Evidence limits
Synthetic callback replay with basic CR/LF terminal semantics demonstrates a rendering mechanism, not the original session event order. Hosted capture waited for greeting completion; early-input overlap remains untested. No upstream fix.
Acceptance criteria
Capture exact command/terminal; verify clean final lines and prompts in TTY and redirected output.
Existing connection invokes Ability after acknowledged disable; fresh connection respects disable; September 21 retest read enabled=false before retrigger and after calculation 42 in the existing session. Fresh-session trigger did not return the fixture greeting. Shared web-demo follow-up observed two new invocations after disable, returning 42 and 17; a fresh demo session did not show fixture execution. Follow-up linked the shared-session initialization credential to the distinct user account U, resolving the earlier client-side identity ambiguity.
Evidence limits and acceptance criteria
Evidence limits
Bounded harmless text fixture, new invocation after stop, not in-flight cancellation. Persisted disabled state verified around execution. Shared-session initialization submitted the distinct user account key with the developer Agent share token; server internal attribution and intended disable scope remain unknown. Two post-disable shared invocations observed, not unlimited execution. Direct U-key CLI baseline was not established; separate-account installed-copy behavior remains open.
Acceptance criteria
Once disable is acknowledged, block every new invocation in existing and new conversations, including queued requests checked before execution. Update active tool lists/trigger routing, enforce server-side state at invocation, and return a structured disabled reason for clear conversational feedback. Define and test already-running cancellation separately.
X02 · P1 · open
Explicit no-send instructions fail across Gmail turns and a real email is delivered
Existing marketplace Gmail sent an email after a conversation-wide no-send instruction and a second turn requesting wording only and repeating do not send. Recipient Gmail independently verified sender, subject and Inbox delivery. Earlier single-turn negative cases entered Gmail without observed delivery; synthetic persistent-write and calculation cases remain separate evidence.
Evidence limits and acceptance criteria
Evidence limits
One real-email multi-turn reproduction with a separate positive control; no universal failure-rate, model-cause or cross-integration claim. Recipient metadata verifies delivery; sender Sent-folder/backend logs and Ability source were not inspected. Earlier basic and quoted negation cases entered Gmail but had no observed delivery in bounded checks. Synthetic storage readback was SDK-mediated; its writer intentionally omitted internal confirmation.
Acceptance criteria
Honor no-send/preview constraints through activation and follow-up turns. Recipient, subject and body are not send authorization. Positive control sends once; basic negative, quoted-command and exact two-turn preview cases send zero, verified against provider evidence. Require clear later authorization before sending; confirmation copy alone must not substitute for execution checks.
X03 · P2 · open
Document when code updates take effect in existing conversations
In the September 18 working-draft test, an existing conversation retained the older code marker after an upload verified by source download; a fresh conversation used the new marker. Keeping a revision for the life of a conversation may be intentional. The audit asks for an explicit documented revision strategy and visible loaded revision, not mandatory hot reload.
One September 18 working-draft update comparison; programmed revision markers are not platform release numbers. Backend mechanism and intended strategy remain unknown. Revision retention may be intentional and is not by itself evidence of a runtime defect or failed save. Current public docs checked September 22 do not specify web-chat revision selection. DevKit restart guidance concerns separate hardware behavior; published-release and physical-device update semantics were not independently tested here.
Acceptance criteria
Explicitly document when working-draft and published-code updates take effect, whether existing conversations retain their loaded revision, and when reconnect/reload is required. Expose the executing revision and verify behavior against that contract. Intentional session pinning is acceptable; immediate hot reload is not required by this finding.
X04 · P2 · open
One-shot returns acknowledgement and closes before later business-result message
The pinned loop starter checks whether an exit word occurs anywhere in the user's text. With a synthetic SDK, “Tell me about nonstop flights” produced “Goodbye!” and zero model calls; the “Tell me about planets” control made one model call. resume_normal_flow ran once in each case. This is a specific starter-template bug, not a general conversation-agent or barge-in defect.
One negative input and one ordinary-request control against the unchanged loop-template at source revision 1416d065f9f77e0a0f04d34c3f46f4c809751492; no broader substring/quoted-word matrix, hosted Ability, live conversation, audio or hardware run. This is scoped to the starter-template exit check and does not establish platform-wide trigger or interruption behavior. No upstream fix is claimed.
Acceptance criteria
Use explicit exit intent or token/phrase boundaries. Add regressions for ordinary words such as “nonstop” and “unstoppable,” and for quoted mentions of exit words. Confirm ordinary requests reach the model while an intentional standalone exit still ends the loop.
X07 · P1 · open
Model exception escapes without resume_normal_flow(); related D05
Broad local/private positioning needs the same mode-specific qualifications used in the Venice announcement. DevKit participation terms permit data collection/training; AI Twin has separate policies. This does not establish that every deployment sends all data or that fully local modes are unavailable.
Public Ability/CLI source is real. Full firmware, hosted runtime and application source/license coverage is not established by the reviewed repositories or a testimonial.
Instant responses and short setup timelines lack named conditions. The diagnostic text timings do not measure voice latency or disprove every optimized configuration.
Broad Linux/device promises are wider than the named Raspberry Pi setup paths. Custom-Pi audio interruption limits and integration prerequisites should travel with the claim.
Website totals, category totals and spotlight demos need a versioned catalog defining features, Abilities, integrations and readiness. Source-folder counts are not a valid rebuttal of a differently scoped feature count.
Developer FAQ advertises JavaScript; the unchanged public contribution validator rejects a JavaScript-only fixture for missing Python files. No separate private JavaScript SDK was inspected.
The September 18 rendered DevKit page records “instant synchronization without restart”; the current DevKit FAQ says changes to an active device, including Ability installs and configuration, require an Agent restart. This is a documentation conflict scoped to the DevKit update path, not a reproduced runtime failure or a claim about CLI/web text behavior.
The developer-page wording is preserved from a rendered September 18 review; the DevKit FAQ was checked September 22. This establishes a documentation-scope conflict only. No physical hardware update was tested, and the finding does not generalize to the CLI or web text client.
Acceptance criteria
Distinguish syncing/pushing an update from when an active DevKit Agent loads it. State the documented restart requirement for active devices; if no-restart applies to another mode, name that mode and support it separately. Do not use web-session tests as hardware evidence.
Hero says v0.1 while FAQ says V2; hardware specifications are not tied to a verified shipped SKU. This is ambiguous naming, not proof that advertised components are absent.
The current home-automation page is headed “Smart Home & IoT,” but its hero says it is “Empowering the next generation of robotic solutions.” The extracted page text also repeats a home-automation heading; that duplicate has not been confirmed as a visible on-page defect.
The current hero wording was verified in page text on September 22; the robotics page is linked for context. Text extraction showed a repeated heading, but no fresh visual check established whether that duplication is visible in the rendered design.
Acceptance criteria
Use a smart-home-specific hero. Verify whether the repeated heading is visible at desktop/mobile before treating it as a separate defect.
Y13 · P2 · open
Developer offer and program versions need one current contract
Conditional free hardware, grants, credits, bounty availability and older tiered programs lack a single dated current offer. Historical differences do not prove unpaid awards or broken promises.
Memorable article reports an eight-task result and broader completion/cost improvements, but reviewed copy does not provide a reproducible experiment sufficient to validate general performance.
Live CLI announcement recommends npm client plus JSON/MCP/session-token features. The supported current CLI is the separate Python client; do not transfer either implementation’s feature claims to the other.
Blog teaches CapabilityWorker/register_capability imports from openhome; neither is exported by the supported public package. Current class-based templates use a different authoring contract. This does not prove an unpublished runtime cannot supply an alternative API.
Smart-home article presents energy, hotel and hospital improvement percentages without adjacent attributable study details in reviewed text. These are industry assertions, not verified OpenHome customer outcomes.
User screenshot shows Google’s unverified-app warning. The production connection journey needs appropriate OAuth verification and consistent product identity.
Evidence limits and acceptance criteria
Evidence limits
User observations and supplied screenshots are attributed; source inspection and initial browser rendering are separate from live integration/audio acceptance.
Acceptance criteria
Confirm production OAuth configuration and appropriate verification for requested scopes; clean-account consent has expected branding and no unverified-app interstitial.
W12 · P1 · open
Google partial consent ends in scope-mismatch failure and disconnected state
User reports token exchange rejected a changed scope set after selected permissions were granted; OpenHome still shows disconnected. A provider grant does not establish that OpenHome received/stored a usable token. Full error intentionally omitted.
Evidence limits and acceptance criteria
Evidence limits
User observations and supplied screenshots are attributed; source inspection and initial browser rendering are separate from live integration/audio acceptance.
Acceptance criteria
Test full and partial consent, previously granted scopes and cancellation; explain required/optional access, recover from mismatch and show connection state consistent with a verified usable token.
User reports Discord consent names the application openhome test app. Audit user-facing application names, provider consent, redirects, support identity and error copy for production-appropriate wording.
Evidence limits and acceptance criteria
Evidence limits
User observations and supplied screenshots are attributed; source inspection and initial browser rendering are separate from live integration/audio acceptance.
Acceptance criteria
Discord consent uses the intended production brand; audit other integration and account surfaces for leftover test/development copy.
W14 · P2 · open
Slack consent shows an approval warning despite working integration
User screenshot shows Slack’s app-not-approved notice; user reports the integration works. This is a distribution/credibility concern, not evidence of broken functionality.
Evidence limits and acceptance criteria
Evidence limits
User observations and supplied screenshots are attributed; source inspection and initial browser rendering are separate from live integration/audio acceptance.
Acceptance criteria
Confirm intended distribution/review path, address approval as applicable and explain expected warnings; preserve verified installation and functional integration behavior.
D16 · P2 · open
Supplied audio-stream example has lifecycle and recovery defects in source
Unchanged HTML renders in Chrome with a placeholder key. Source shows unconditional reconnect after close, a playback/active-state race, an out-of-scope buffer variable and uncleared ping intervals. No microphone access or live call was attempted.
Evidence limits and acceptance criteria
Evidence limits
User observations and supplied screenshots are attributed; source inspection and initial browser rendering are separate from live integration/audio acceptance.
Acceptance criteria
With isolated credentials and explicit microphone testing, verify start/deny/retry/end/restart, no reconnect or active tracks after End, correct playback initialization, quota recovery and no timer leaks.
D17 · P2 · open
Local validator accepts constructs rejected by hosted validation
The public validator accepted the SDK fixture; hosted upload rejected inspect and getattr with HTTP 400. This is validator parity friction, not evidence those constructs should be allowed or that SDK methods fail. Fixture updated to use explicit documented calls.
Observed with the evaluation fixture; no request to weaken hosted sandbox policy. The accepted fixture later had an unrelated reply timeout.
Acceptance criteria
Local preflight rejects the same forbidden constructs as hosted upload with a documented runtime/validator version and actionable location; preserve sandbox restrictions.
D18 · P3 · open
API sidebar omits HTTP-method badges on Upload Ability and Get Ability
The supplied screenshot shows Upload Ability and Get Ability without HTTP-method badges while adjacent endpoints have badges. The Upload Ability card displays POST, and the retrieved reference identifies Upload Ability as POST and Get Ability as GET. This is inconsistent navigation metadata, not evidence of endpoint failure.
View screenshot evidence (1)
API sidebar labels. The screenshot shows Upload Ability and Get Ability without method badges. It documents navigation labeling only; it does not indicate endpoint behavior. User-supplied screenshot; capture date is not independently verified.
Visual evidence is the user-supplied screenshot; reference text was retrieved on September 18, but no fresh browser rendering or endpoint invocation was performed for this finding. Screenshot capture date is not independently established.
Acceptance criteria
Upload Ability shows POST and Get Ability shows GET in endpoint navigation, matching the reference and authoritative API contract. Check every endpoint entry for consistent labels and method badges in desktop and narrow-screen navigation; retain readable method text for assistive technology.
W15 · P2 · open
Onboarding tour controls lose readable contrast across hover/control states
Gabriel reports a hover-text problem in the product tour. Two supplied screenshots show Skip nearly disappearing against white in one state and Next appearing very pale in the other. The guided tour itself is a positive feature (K04).
View screenshot evidence (2)
Tour state A. Skip appears almost white against the white tour panel; the exact pointer or disabled state is unknown. User-supplied screenshot; capture date is not independently verified.Tour state B. Next appears very pale against the white tour panel; the exact pointer or disabled state is unknown. User-supplied screenshot; capture date is not independently verified.
Evidence limits and acceptance criteria
Evidence limits
The screenshots establish the low-contrast appearances, but not the exact pointer or disabled state. No computed contrast ratio or fresh interaction test was captured.
Acceptance criteria
Check the real normal, hover, focus, active and disabled states of Skip, Back and Next. Keep available actions readable with appropriate text contrast, visible focus and an unambiguous disabled state; preserve the tour and skip path.
W16 · P2 · open
Interruption settings need plain-language explanations
The settings screenshot shows Interactive Interruption, Auto Interruption and Interruption Sensitivity without adjacent explanations. The user could not tell what each option changes or how the controls interact.
View screenshot evidence (1)
Interruption settings during a long turn. Both interruption switches appear off and sensitivity appears high. The capture shows displayed state, not what each setting means or whether a setting caused a later outcome. User-supplied screenshot; capture date is not independently verified.
Evidence limits and acceptance criteria
Evidence limits
The screenshot shows controls and their displayed state, not their underlying semantics. Do not invent definitions or attribute the later interruption to a specific setting without a controlled test.
Acceptance criteria
Add accessible help that defines each control using the product-confirmed behavior: whose speech can be interrupted, what triggers interruption, how sensitivity affects it, defaults and interaction between settings. Include a short example and a reversible way to compare settings.
W17 · P1 · open
Conversation display reordered user and assistant messages
The user reports saying the wake phrase after the sleep guidance and then seeing the thread reorder. The screenshot places an assistant response to that phrase above the user bubble containing it, making the conversation appear out of sequence.
Evidence limits and acceptance criteria
Evidence limits
Screenshots show reply placement and the user reports reordering. No continuous transition capture, correlated event IDs or persisted-history comparison establishes where ordering changed.
Acceptance criteria
Keep stable message IDs and chronological turn ordering during streaming, finalization, wake-up, reconnect and reload. Reproduce with event timestamps and compare visible order to the recorded turn sequence; never move an earlier user turn after its reply.
W18 · P1 · open
An earlier displayed user message appeared to be rewritten
Gabriel reports that the thread subsequently rewrote a displayed user message, changing it from the wake phrase to a later request. The available static screenshots are ambiguous and do not establish that both displays refer to the same message; this remains a user-reported transcript-stability concern pending stable-identity evidence.
Evidence limits and acceptance criteria
Evidence limits
Static screenshots and user report do not establish that the same stored message was overwritten or distinguish provisional transcription from repeated speech. No stable message IDs, continuous timeline or persisted-history comparison was captured.
Acceptance criteria
Tie transcript updates to a stable utterance/message ID. Restrict provisional transcription updates to the current utterance and label them; preserve earlier finalized text when later speech arrives. Verify display and stored history after reload.
W19 · P1 · open
Image creation unexpectedly submitted a pending message
The user reports that creating a photo while a message was waiting to be sent caused the message to send automatically, without making that consequence clear beforehand.
Evidence limits and acceptance criteria
Evidence limits
User report only. The draft, image creation and send transitions were not recorded or independently reproduced; exact payload and intended interaction contract remain to be established.
Acceptance criteria
Before image generation/attachment completes, make any send behavior explicit. For a draft composer, retain text and the generated image for preview with an explicit Send action; if immediate send is intended, state that in the initiating action. Test generation success, cancellation and failure with a pending draft.
The user reports an orange hover state for AI Twin while other sidebar entries hover white. The difference appears unexplained in the navigation.
Evidence limits and acceptance criteria
Evidence limits
User report only; no sidebar hover screenshot or style inspection was captured. A documented intentional navigation distinction has not been ruled out.
Acceptance criteria
Apply consistent hover, selected and keyboard-focus styling across equivalent sidebar links; if AI Twin intentionally has a different role, make that distinction clear and verify readable contrast.
W21 · P2 · open
Agent cards show a pointer cursor outside the working click target
On the Agents page, the user reports that agent items show a pointer cursor suggesting the card is clickable, but only the Chat button responds.
Evidence limits and acceptance criteria
Evidence limits
User report only; no click-event trace or fresh test of card-body versus button interaction was captured.
Acceptance criteria
Match the pointer and hover affordance to the actual clickable area, or make the intended card action work consistently. Keep nested controls distinct and verify keyboard, mouse and touch behavior.
W22 · P1 · open
System/status messages appear to contaminate the conversational output stream
The user reports system messages appearing in the output stream as conversation content. Sleep/wake guidance is visible in ordinary chat bubbles in the supplied transcript, but whether system text also enters model context or synthesized speech is not established.
Evidence limits and acceptance criteria
Evidence limits
Visible sleep/wake status text shares the conversation stream. This does not establish hidden-prompt leakage, model-context contamination or synthesized speech of system text.
Acceptance criteria
Give lifecycle/status events an explicit event type and visual treatment distinct from user and assistant turns. Test rendered transcript, model context, exported history and speech independently so intended guidance is preserved without being misattributed or injected into ordinary reply content.
W23 · P1 · open
Automatic interruption cut in while the user was still giving a long response
September 21 live browser testing: Gabriel repeatedly reports premature replies during natural pauses, including with Interactive Interruption off. A longer utterance succeeded with both switches off and high sensitivity. Settings were not isolated consistently; some short transcript fragments were reportedly never spoken.
View screenshot evidence (2)
Natural-pause interaction. The screenshot shows an interruption indicator while Gabriel says he is about to tell a story. The image supports the displayed sequence, not exact pause timing or an audio-derived cutoff. User-supplied screenshot; capture date is not independently verified.Long turn with interruption controls off. The capture shows a substantial user turn with both interruption switches off. Gabriel reports completing it; the screenshot alone cannot validate the audio or isolate either setting. User-supplied screenshot; capture date is not independently verified.
Evidence limits and acceptance criteria
Evidence limits
See VOICE_SESSION_REVIEW_20260921.md for the dated live follow-up and screenshot provenance. Repeated user-observed cutoffs and visible interleaved replies support the experience issue. No synchronized audio timeline identifies exact pause lengths or cause. Auto Interruption off is a candidate workaround, not an isolated causal result.
Acceptance criteria
Reproduce with long speech, short pauses, hesitation and varied background noise; record speech activity, end-of-turn detection, timeout and interruption events. Respect ongoing user speech and provide recovery for a premature cutoff. Explain and test the applicable interruption settings.
W24 · P2 · open
Test email reported sent but arrived in recipient Spam
Existing Gmail Ability reported “Done! Email sent.” Independent Gmail API metadata verified the expected sender, recipient and subject, but the message carried the SPAM label. Successful sending did not establish inbox placement. A later negative-case email reached Inbox, so observed placement varied across the two tests.
Evidence limits and acceptance criteria
Evidence limits
Two synthetic test messages between user-owned Gmail accounts: the positive control had SPAM label; the later negative-case message had INBOX label. Cause and prevalence remain unknown; no demonstrated attribution to OpenHome versus reputation, content, authentication or recipient filtering.
Acceptance criteria
Investigate authentication, sender reputation, content and recipient filtering using message headers and representative owned-mailbox tests. Distinguish send acceptance from inbox delivery in product language and support diagnostics; do not promise inbox placement without evidence.
W25 · P2 · open
Negative email request gets a readiness prompt instead of acknowledging restraint
For “Do not send mail … Do not send this email,” the existing Gmail Ability responded “Gmail ready. What would you like to do?” No matching negative-case email was observed in two bounded mailbox checks, but the response did not acknowledge the instruction to refrain.
Evidence limits and acceptance criteria
Evidence limits
Basic case response and bounded recipient absence are independently documented. A subsequent multi-turn preview case did send despite repeated negative instructions (X02); better acknowledgement alone is not sufficient protection.
Acceptance criteria
Respond “Understood—I won’t send that email.” Preserve the no-send constraint through follow-up turns, avoid entering a send workflow solely because its trigger was mentioned, and require a clear later authorization before sending.
User reports receiving an unexpected “Test” message while starting the walkthrough. Screenshot shows a standalone left-aligned Test bubble, giving the experience the appearance of unfinished placeholder copy. User subsequently clarified that it appeared when microphone access began; that timing is user-reported, not event-traced.
View screenshot evidence (2)
Standalone Test message. A standalone left-aligned Test bubble is visible. Its source and the event that caused it are not shown. User-supplied screenshot; capture date is not independently verified.Test messages in the conversation. The screenshot shows repeated Test bubbles in the conversation. It does not identify whether they came from the walkthrough, restored history, configuration, or a debug/default path. User-supplied screenshot; capture date is not independently verified.
Evidence limits and acceptance criteria
Evidence limits
Visible text is screenshot-backed; arrival is user-reported. No event trace distinguishes new output, restored history, Agent configuration or debug/default text. Not evidence of a pre-production deployment or another ordering defect.
Acceptance criteria
Identify the message source and remove unintended test/default text. Give legitimate status events meaningful text and explicit attribution.
W28 · P1 · open
Narrative and exit phrases activate Alarm without a present setup request
Repeated user speech screenshots show stories and exit requests mentioning alarm activating Alarm, followed by a time question or no-change response. Later topic changes successfully resume ordinary conversation.
View screenshot evidence (4)
Alarm card during a story. A story about difficulty setting an alarm is followed by an Alarm card. The screenshot shows the conversation sequence, not audio intent classification or an alarm being scheduled. User-supplied screenshot; capture date is not independently verified.Story activation and later topic change. An Alarm card appears during a story; a later flooring exchange receives an ordinary reply. This supports one reported unwanted activation and subsequent topic recovery, not persistent lock-in. User-supplied screenshot; capture date is not independently verified.Alarm card after an exit request. After “Exit alarm setup. I’m telling a story, not requesting an alarm,” an Alarm card is visible. The screenshot does not establish an alarm was created. User-supplied screenshot; capture date is not independently verified.Contrast: explicit alarm request. This capture starts with “I wanted to set an alarm in the morning” and the Agent asks what time. It is consistent with a genuine setup request and is not evidence of a false trigger. User-supplied screenshot; capture date is not independently verified.
Evidence limits and acceptance criteria
Evidence limits
Unwanted activation is screenshot-backed. Persistent lock-in is not supported. ABORT is visible in one supplied socket log; no universal routing algorithm or unintended alarm creation is inferred.
Acceptance criteria
Distinguish narrative, negation, cancellation and current intent before activation. Return to conversation cleanly and preserve successful explicit relative-time alarm requests.
W29 · P1 · open
User reports unspoken words appearing as speech input
Gabriel reports So, hey and okay appearing without speaking; supplied screenshot displays a User So bubble and interruption indicators.
View screenshot evidence (1)
User “So” bubble and interruption indicators. The screenshot shows a User “So” bubble and interruption indicators. Gabriel reports not speaking it; the image cannot establish whether speech, feedback, ambient audio, or stale input produced it. User-supplied screenshot; capture date is not independently verified.
Evidence limits and acceptance criteria
Evidence limits
Text is screenshot-backed; absence of speech is user-reported. No recorded input isolates recognition, feedback, ambient sound or stale-message causes.
Acceptance criteria
Test controlled silence, speaker-muted and speaker-enabled conditions with audio/transcript correlation. Do not create finalized user turns from non-speech.
W30 · P2 · open
Long speech-response waits lack understandable progress and recovery state
Gabriel reports waits over 30 seconds during alarm-exit attempts without clear processing feedback. Screenshots show a User marker but do not explain whether input is provisional, accepted, processing or failed.
View screenshot evidence (1)
Conversation and DevTools context. The capture shows a User marker beside a DevTools Network panel. The resource list does not measure request duration or prove a backend failure; the wait duration remains user-reported. User-supplied screenshot; capture date is not independently verified.
Evidence limits and acceptance criteria
Evidence limits
Duration is user-reported. No request-correlated timing establishes backend latency. WebSocket use and unknown-log console warnings are not themselves proof of failure.
Acceptance criteria
Distinguish listening, transcription, processing, completed and failed states. Provide bounded waiting feedback and recovery without forcing repeated speech.
W31 · P1 · open
Reported turn blending: replies accumulate while the user's spoken turn continues
Gabriel reports premature replies during non-terminating pauses, followed by inadequate yielding when he resumes. Several agent replies and speech starts appear while transcription extends the same user message, confusing turn ownership and transcript meaning.
Evidence limits and acceptance criteria
Evidence limits
User-reported sequence; no synchronized audio or backend trace. ASR/STT produces input transcription; TTS produces output speech. Overlapping generation and trigger mechanisms are hypotheses. The available static story screenshots are ambiguous and do not clarify turn blending or establish message identity, provisional/final status, whether speech repeated, or Agent-generation timing; they are excluded from the client-facing gallery. Overlaps W23; not a separate proven root cause.
Acceptance criteria
Correlate audio, provisional/final transcripts, turn/generation IDs and playback for incomplete thoughts, natural pauses and resumed speech. Keep listening until the user yields; promptly stop obsolete output when speech resumes; prevent duplicate replies and preserve response-to-input-version attribution. Establish measured timing criteria from the baseline.
W32 · P2 · open
Separate conversation threads are not discoverable within a personality
Gabriel reports one text history per personality with no separate-thread workflow in the tested UI. Unrelated topics accumulate, parallel work and retrieval become awkward, and a fresh-conversation recovery path is unclear. Users cannot easily understand the boundary between conversation context and personality memory.
Evidence limits and acceptance criteria
Evidence limits
User observation of the tested interface; no exhaustive navigation review or backend limitation established. One screenshot cannot establish global feature absence. Actual model-context reuse is unverified.
Acceptance criteria
Expose New conversation and a named conversation list, or make an existing equivalent discoverable. Create two independent histories under one personality, switch and reload without mixing or losing them. Explain conversation reset versus persistent personality memory.
W33 · P3 · open
Wide desktop gutters waste conversation space and weaken perceived polish
The supplied screenshot shows broad empty gutters beside the conversation panel, between left navigation and right settings. Gabriel finds the layout inefficient and less premium. Improve space allocation while retaining readable message line lengths.
View screenshot evidence (1)
Conversation panel and side gutters. The desktop screenshot shows broad empty space beside the conversation panel. It documents this viewport only; it does not establish responsive behavior at other sizes. User-supplied screenshot; capture date is not independently verified.
Evidence limits and acceptance criteria
Evidence limits
Visible spacing is screenshot-backed; perceived premium quality is user feedback. No task blockage, measured productivity loss or cross-viewport defect was established. Capture time is unverified; September 22 is the report date.
Acceptance criteria
Reduce excessive outer gutters with balanced responsive panel sizing. Check wide/narrow viewports, settings open/closed, zoom, overflow and keyboard access; retain readable prose widths and accessible settings.
A supplied screenshot of a new Agent shows literal heading markers, asterisks and list separators in the visible reply. This makes the answer harder to scan and can expose formatting syntax as content. The screenshot establishes this response only, not prevalence or cause.
View screenshot evidence (1)
New Agent response with literal Markdown. The visible reply contains literal heading markers, asterisks and list separators. This establishes one displayed response, not prevalence or cause. User-supplied screenshot; capture date is not independently verified.
Evidence limits and acceptance criteria
Evidence limits
One user-supplied screenshot establishes literal Markdown syntax in one Agent response. No broader prompt matrix, rendering-path inspection or root-cause test was performed. A system-level instruction is a recommendation, not a verified fix.
Acceptance criteria
Define a stable output-format contract independent of user-supplied prompts; render supported Markdown in the chat UI or otherwise present its structure legibly. Treat system-level formatting instructions as a guardrail, not a substitute for rendering. Test headings, bold text, numbered steps and bullets in visible text and spoken output, including user prompts that request conflicting formatting.
W35 · P1 · open
Markdown-heavy replies are reported as difficult to understand in speech
Gabriel supplied three audio examples and reports that the speech output does not turn Markdown-heavy reply text into comprehensible spoken language. The set includes one approximately 55-second clip and two clips of approximately 8 and 7 seconds. These user-supplied examples warrant a core conversation-quality investigation; this pass did not independently replay or transcribe them.
Listen to supplied audio evidence (3 clips)
Short speech example 1
Approximately 8.1 seconds by file-container metadata; user-supplied example. MP3 is the browser-playback copy; the supplied M4A is retained as the original. Download MP3 · Download original M4A. User-supplied content; not independently replayed or transcribed in this pass.
Long Markdown-to-speech example
Approximately 55.1 seconds by file-container metadata; user reports difficult-to-understand speech and a distorted opening. MP3 is the browser-playback copy; the supplied M4A is retained as the original. Download MP3 · Download original M4A. User-supplied content; not independently replayed or transcribed in this pass.
Short speech example 2
Approximately 7.3 seconds by file-container metadata; user-supplied example. MP3 is the browser-playback copy; the supplied M4A is retained as the original. Download MP3 · Download original M4A. User-supplied content; not independently replayed or transcribed in this pass.
Evidence limits and acceptance criteria
Evidence limits
The three clips and symptom description are supplied by Gabriel. The files' container metadata indicates approximate durations of 8.1, 55.1 and 7.3 seconds; duration does not establish what is audible. This pass did not independently replay, transcribe or correlate the audio with source text, TTS input or provider events, so prevalence and cause are unknown.
Acceptance criteria
Run a controlled text-and-speech matrix for headings, emphasis, numbered steps, bullets, links and long answers. Normalize Markdown before speech generation so syntax tokens are not read aloud and hierarchy becomes natural phrasing/prosody; preserve meaning and keep spoken output aligned with visible text. Correlate source response, normalized TTS input, audio chunks and playback, including long responses and interruptions.
W36 · P2 · open
One supplied long speech sample reportedly starts with a distorted voice
Gabriel reports a severely distorted voice at the beginning of the approximately 55-second Markdown-to-speech example. Keep this onset-quality symptom distinct from the broader Markdown intelligibility concern until the initial audio segment and later speech are compared.
Listen to supplied audio evidence (1 clip)
Long example: reported distorted opening
The same user-supplied long clip is cited for its opening voice-quality symptom; it is not a separate trial. MP3 is the browser-playback copy; the supplied M4A is retained as the original. Download MP3 · Download original M4A. User-supplied content; not independently replayed or transcribed in this pass.
Evidence limits and acceptance criteria
Evidence limits
One user-supplied clip and Gabriel's description support a reported onset symptom only. No independent listening, repeated trial, waveform inspection or provider/playback trace establishes the distortion's extent, recurrence or cause.
Acceptance criteria
Compare the first spoken segment with the remainder of the same response and repeat across controlled sessions. Correlate selected voice, synthesized chunk boundaries, encoding and playback timestamps; define an intelligibility/voice-quality check before attributing the symptom to any layer.
W27 · P2 · open
Typed pause while asleep omits the live user turn; reload restores it
Browser testing reproduced the user's missing-turn observation: sending pause while asleep displayed an acknowledgement without the user bubble, while the awake control displayed both. Reload restored the missing pause in its correct position. The confirmed impact is an incomplete live transcript, not lost stored input.
Evidence limits and acceptance criteria
Evidence limits
September 21 browser validation reproduced the missing live user bubble when pause was submitted while asleep; the awake control displayed both turns. Reload and Activity reopening restored the missing pause ahead of its acknowledgement. This supports a live-display omission, not stored-message loss. One asleep/awake pair was tested; backend cause and speech behavior remain unverified.
Acceptance criteria
Represent accepted typed input in the transcript, or clearly represent it as a user-initiated control action if intentionally handled outside ordinary chat. Verify awake and asleep cases, event attribution and history after reload; do not silently omit the action while showing its acknowledgement.
Observed and reported behavior
W01 · — · unverified
Unverified observation: private selection may not reflect persisted state
Unverified observation: private selection may not reflect persisted state; no exposure shown
Evidence limits and acceptance criteria
Evidence limits
The controlled 2026-09-15 run reported a misleading privacy/readback contract while server flags remained private and unsubmitted; it did not demonstrate public exposure. A 2026-09-21 read-only check found the original trial Agent absent, and the accessible replacement Agent was not an exact-fixture retest. This observation is excluded from the current reportable findings until reproduced.
Acceptance criteria
Reopen the original trial fixture and reproduce the exact save/reload mismatch before restoring reportable status.
W02 · — · unverified
Unverified observation: ordinary turns may display stale Ability activity cards
Unverified observation: ordinary turns may display stale Ability activity cards; routing cause unproved
Evidence limits and acceptance criteria
Evidence limits
The controlled 2026-09-15 run reported stale Ability cards after ordinary turns, but no server execution trace established a routing-engine failure. A 2026-09-21 read-only check found the original fixture absent; a different existing conversation showed no cards, and no new message was sent. This observation is excluded from the current reportable findings until reproduced.
Acceptance criteria
Reopen the original trial fixture and reproduce the stale-card sequence before restoring reportable status.
W05 · — · explained
Earlier Agent onboarding mismatch is no longer present in the current public guide
On September 15, the inspected UI offered Design Agent with AI or Pick from a Template, while the then-reviewed guide described Quick Creation, Pro Creation and AI Twin. The current Quickstart and Creating an Agent pages now describe the same two entry paths as the captured UI. This explains the documentation-side mismatch; the live UI was not rechecked on September 22.
The September 15 UI evidence and September 22 public guides are from different dates. The current interface was not re-opened, so this does not prove the whole onboarding journey is now accurate or successful. D12 remains a separate, current conflict about required fields.
Acceptance criteria
Keep as an explained historical observation unless a fresh same-day comparison shows the app and current guide diverge again; then record both versions and the exact path.
W06 · — · explained
Transient tour-control stall during initial conversation loading; reopening recovered
In one first-use session, Skip and Next did not respond while the conversation showed Loading. Reopening Chat recovered; Skip worked once connected. This is a bounded recovery observation, not evidence of a persistent tour defect.
Evidence limits and acceptance criteria
Evidence limits
One transient in-app-browser session; reopening recovered and Skip worked after connection. No repeat, persistent failure, or controlled delayed-connection test establishes a systemic defect. Do not count as an open product issue.
Acceptance criteria
If the stall recurs, preserve a usable Skip action while connecting and provide clear cancel/retry behavior for a genuine setup failure.
W09 · — · unverified
Selected Agent voice was not discoverable in the separate voice catalog
Agent creation showed Matilda as the selected voice, while a completed search on the standalone Voices page returned no result in its My, OpenHome or Public groups. This is a cross-surface discoverability question, not evidence that Matilda is missing, unavailable or incorrectly assigned.
Evidence limits and acceptance criteria
Evidence limits
One selected voice and one standalone search. No product contract promises the two catalogs are identical, and no evidence shows the selected voice cannot be used. Keep as an unverified UX observation rather than a confirmed issue.
Acceptance criteria
Clarify whether the Agent selector and standalone Voices page draw from different catalogs. If they do, label the source and link selected-voice details; if they share a catalog, make selectable voices searchable.
Compared with mature coding CLIs, the reviewed OpenHome CLI lacks root build identification, general readiness and authentication-status commands.
Evidence limits and acceptance criteria
Evidence limits
This is a comparative product-gap observation against mature coding CLIs, not a broken advertised OpenHome feature. The reviewed build's parser behavior was reproduced, but current replacement releases were not tested.
Acceptance criteria
Offer concise first-use guidance plus separate build, read-only readiness and credential-status surfaces without exposing secrets.
C02 · — · explained
templates --json observation folded into Y15; not a separate broken CLI contract
The pinned Python CLI rejected templates --json, but the supported Python CLI does not promise that flag. The older CLI article advertises JSON/MCP capabilities for a different product generation; that version-routing mismatch is Y15. Keep this probe as supporting evidence, not a separate product defect. A general machine-readable-output interface remains a proposed enhancement.
One command was tested at the pinned Python CLI revision. Other command output contracts were not comprehensively surveyed. Current documentation describes this Python CLI but does not promise templates --json.
Acceptance criteria
If machine-readable output is prioritized, specify a versioned schema and test it separately; do not describe the current absence as a regression unless a supported contract is published.
X08 · — · explained
Platform renames class to account Ability name; other bytes match after that exact substitution
The reviewed pages make broad claims about emotional insight, automatic support for 30+ languages and flawless Ability scenarios, but do not present linked evaluation scope or results. This is a public-substantiation observation only: it does not establish that OpenHome lacks internal tests or that the claims are false.
The cited public pages were reviewed; internal evaluation plans or results were not requested or available. This does not establish that internal testing is absent, nor does this review independently evaluate the claims.
Acceptance criteria
If the claims are retained, optionally link a concise evidence note stating supported configurations, evaluation scope, results and limitations; otherwise qualify the public wording. Do not imply internal testing is absent without checking with the product team.
Y11 · — · unverified
Local Link copy could scope its local-execution claim more explicitly
The article describes a Mac/local-computer command-execution template and says there are no cloud dependencies for execution. Current Local Connect documentation describes a CLI Local Link bridge on the user's computer. This supports local command execution, not a conclusion about end-to-end voice/control transport, dedicated DevKit hardware, or other OpenHome clients. Optional copy clarification could make that boundary explicit; no contradiction or runtime failure was established.
The source describes Local Link as a Mac/local-computer template; current Local Connect docs describe a CLI bridge. No bridge was started and no live traffic was captured. This is an optional copy-clarity observation, not a finding that the claim is false, a reproduced data-flow/security failure, or evidence about dedicated DevKit hardware, web chat, or every deployment.
Acceptance criteria
Optional clarity improvement: state that the cited Local Link template executes commands on the user's Mac, and clarify separately if the article intends to make a claim about the voice/control transport or end-to-end data path. Do not infer behavior for DevKit hardware or web conversation from the CLI bridge.
Evaluation tooling notes
T01 · — · open
The hardcoded “revision two verified” fixture phrase sounds like independent verification
Declaring MatchingCapability state fields resolved the fixture's startup failure. Corrected assertions and protocol assumptions allowed the foreground suite to complete. The original timeout is not an SDK method failure.
Evidence limits and acceptance criteria
Evidence limits
Evaluation-tooling corrections, not an upstream product fix. Earlier receipts are preserved; later corrected checks determine current statuses.
Acceptance criteria
Preserve these bounded contracts in repeatable regression checks; expand coverage using the per-method prerequisites.