- add phrase routing for radio open/resume and genre launch
- inspect radio genre and station metadata before locking the outbound entity values
- prefer the same payload shape the menu path uses instead of a generic cloud speech reply
- Exit criteria:
- voice `open the radio` launches radio successfully
- voice `play country music` launches a country station
- no fallback cloud placeholder reply is spoken on success
### 2. ESML Apostrophe Encoding Bug
- Status: `ready`
- Tags: `polish`
- Why now: this is a small, high-confidence speech quality bug affecting many paths.
- Current evidence:
- [ResponsePlanToSocketMessagesMapper.cs](C:/Projects/JiboExperiments/OpenJibo/src/Jibo.Cloud/dotnet/src/Jibo.Cloud.Application/Services/ResponsePlanToSocketMessagesMapper.cs) currently escapes `'` to `'`
- the robot is pronouncing the encoded form instead of treating it as natural text
- Implementation notes:
- stop encoding apostrophes in spoken ESML text unless a capture proves a narrower escaping rule is needed
- keep escaping for `&`, `<`, and `>`
- Exit criteria:
- contractions and possessives sound natural again in live speech
### 3. Backup / OTA Yes-No Reliability
- Status: `ready`
- Tags: `protocol`, `stt`
- Why now: the update and backup prompts are real daily-use system flows and still feel fragile.
- Current evidence:
-`surprises-ota` is a real robot-side skill family in [index.js](C:/Projects/JiboOs/V3.1/build/opt/jibo/Jibo/Skills/@be/be/node_modules/@be/surprises-ota/index.js)
- we already improved constrained yes-no routing, but live tests still show some turns collapse into empty transcript or generic speech
- Implementation notes:
- keep local rules only on constrained replies
- improve empty-turn retry behavior for settings and OTA prompts
- capture whether stock OS uses a different yes-no prompt shape in backup versus update flows
- Why now: the latest capture bundle shows a second yes-no family where the robot asks whether it can share something, and spoken `yes` is still being handled like unconstrained speech instead of a reply to the active prompt.
- Current evidence:
- the attached `jibo test 13` session includes both examples in one bundle:
- a proactive or share-style prompt where spoken `yes` was treated as generic speech
- a later update prompt where spoken `no` was accepted correctly
- Why now: clock, date, timer, and alarm menu hooks are already visible in captures and the robot repo has a real `@be/clock` skill.
- Current evidence:
- [protocol-inventory.md](C:/Projects/JiboExperiments/OpenJibo/docs/protocol-inventory.md) already tracks menu intents for `askForTime`, `askForDate`, `timerValue`, and `alarmValue`
-`JiboOs` shows `@be/clock` branches on `entities.domain = clock | timer | alarm`, uses `intent = menu` for menu launches, and has distinct local value-collection rules such as `clock/timer_set_value`
- [artifact-output/jibo-test-15](C:/Projects/JiboExperiments/artifact-output/jibo-test-15) shows stock OS 1.9 rejecting our older `timerValue` top-level launch with `found no matching transition`, which points to a stock-style `start` flow plus local follow-up value rules instead
- Implementation notes:
- compare our custom time/date path against actual menu payloads
- keep direct clock/date/day local, but treat timer and alarm as a two-part flow: stock start intent plus bare follow-up parsing on `clock/*_set_value`
- voice `open the clock` now routes to the direct local `askForTime` clock-view path instead of the broader clock menu
- voice `what time is it`, `what's today's date`, and `what day is it` now use stock-shaped local `@be/clock` handoffs instead of custom cloud-only speech
- voice `set a timer for five minutes`, `set an alarm for 7:30 am`, `set an alarm for 830`, and `set an alarm for 8 30` now emit direct `timerValue` / `alarmValue` payloads with the entities the local skill expects
- partial timer/alarm requests such as `set a timer` and `set an alarm` now stay on a controlled clarification reply path instead of drifting into Nimbus/chat echo
-`JiboOs` shows `@be/gallery` opens from `intent = menu`, while `snapshot` and `photobooth` actually map into `@be/create` with `createOnePhoto` and `createSomePhotos`
- media and update metadata now persist to a local state file in the hosted `.NET` path, so gallery and staged update state are no longer strictly process-memory-only
-`Media.Create` now retains uploaded metadata plus a best-effort raw body placeholder and serves the same media URL back through `/media/{path}`
- unstaged update queries no longer fabricate a placeholder no-op manifest, which should reduce the phantom `always has updates` behavior during normal operation
- real staged updates can still be created explicitly through the protocol layer when we are ready to prove end-to-end delivery
- Why later: the original architecture and recent proactive captures suggest proactivity is a first-class cloud subsystem, not just ordinary chat that starts itself.
- Current evidence:
- the attached original Jibo architecture diagram shows a cloud-side `Proactivity Selector`, `Proactivity Catalog`, and robot-side proactive trigger plumbing
- [jibo test 13.txt](C:/Projects/JiboExperiments/artifact-output/jibo-test-13/jibo%20test%2013.txt) and its websocket artifacts show a proactive-style `I have something to share with you` offer and later proactive `TRIGGER` traffic
-`@be/surprises`, `@be/surprises-date`, and `@be/surprises-ota` already exist as local robot-side building blocks
- Questions to answer:
- what minimum cloud-side selector we need for stock-OS-compatible surprise offers
- how proactive `TRIGGER` traffic should map into a hosted OpenJibo proactivity service
- whether `surprises-date/offer_date_fact` should be the first end-to-end proactive offer we intentionally support
- Implementation notes:
- model proactivity as its own orchestrator separate from ordinary conversational turn routing
- include offer, constrained yes/no, fulfillment, and dismissal behavior in the design
- preserve the artifact linkage to the original architecture diagram and `jibo-test-13`
- Why later: the original architecture explicitly calls out `History`, and that likely maps to the kind of durable personal memory we want for names, preferences, and remembered facts.
- Current evidence:
- the attached original Jibo architecture diagram includes a dedicated `History` component in cloud storage
- stock Jibo behavior historically included awareness of names, birthdays, holidays, and special dates
- Questions to answer:
- what data belongs in memory versus account/profile versus skill-specific storage
- how much of the original behavior was robot-local versus cloud-backed
- what the first safe OpenJibo memory slice should be
- Implementation notes:
- plan for person identity, preferred name, birthday, relationship facts, and notable dates
- keep the first design privacy-aware and easy to host
- treat this as shared infrastructure that other skills can consume rather than a standalone feature
- Why later: the original architecture diagram suggests `Lasso` sits between the hub and outside data sources, which likely explains how Jibo knew about news, calendar items, holidays, and other structured world events.
- Current evidence:
- the attached original Jibo architecture diagram shows `Lasso` connected to 3rd-party data such as AP News, Dark Sky, GCalendar, Wolfram, and other external sources
- stock Jibo behavior historically covered holidays, birthdays, special events, and topical knowledge
- Questions to answer:
- whether `Lasso` should be recreated as a single aggregation service or as several focused providers behind a shared interface
- which parts are needed for news, weather, calendar, commute, astrology/date facts, and holidays
- what subset is practical for a hosted OpenJibo v1
- Implementation notes:
- treat holidays and special dates as first-class backlog scope here
- use this item to drive future provider work for news, weather, calendar, commute, and event awareness
- Why later: these are already stubbed in `.NET`, but the robot-side ownership still needs clearer mapping.
- Current evidence:
- current `.NET` placeholders live in [InMemoryJiboExperienceContentRepository.cs](C:/Projects/JiboExperiments/OpenJibo/src/Jibo.Cloud/dotnet/src/Jibo.Cloud.Infrastructure/Content/InMemoryJiboExperienceContentRepository.cs)
- Nimbus has personal-report hooks, but the exact cloud contract still needs confirmation
- Questions to answer:
- should calendar and commute be independent feature paths or sections inside personal report
- what minimum provider data shape lets Jibo present these naturally
- Why later: there is a real local `@be/who-am-i` skill, which likely covers user identification, name capture, and enrollment cues that matter for a modern identity layer.
- Current evidence:
-`@be/who-am-i` exists in the stock skill inventory
- the skill source references `jibo.kb.loop`, loop owner / loop member lookup, enrollment state, hypothesis views, and a `Who Am I_ Collect Name` flow
- Questions to answer:
- whether `who am I` is primarily recognition, enrollment, or profile correction
- how name, face, and voice enrollment were originally split between robot-local state and cloud services
- what the minimum hosted-cloud contract is to make identity feel native again
- Implementation notes:
- tie this work back to the broader `History / Memory Layer`
- capture whether the first useful slice is recognition-only, rename-only, or full enrollment support
### 16. Onboarding, Loop Management, And Fresh Start
- Status: `discovery`
- Tags: `protocol`, `docs`
- Why later: stock Jibo onboarding and household management were app-driven, and a hosted OpenJibo path will need a replacement for adding/removing people and setting ownership cleanly.
- Current evidence:
-`@be/first-contact`, `@be/introductions`, `@be/tutorial`, and `@be/restore` all exist in the stock skill inventory
-`@be/who-am-i` and `@be/chitchat` both reference `jibo.kb.loop`, loop owner, and loop members
-`@be/restore` and `@be/settings` show explicit wipe / restore / reboot behavior, which suggests there is a meaningful "fresh start" lifecycle to support
- Questions to answer:
- how a new owner or household should be provisioned without the original mobile app
- how to add, remove, and re-enroll loop members safely
- whether the right replacement is a lightweight web app, an operator-only admin flow, or both
- Implementation notes:
- include ownership transfer, fresh start, and post-restore re-onboarding in scope
- figure out what minimum loop-management UI or API a hosted OpenJibo v1 needs
- Why later: this is a simple, high-value device-control command that should feel native once the local payload shape is confirmed.
- User goals:
-`turn it up`
-`turn it down`
-`increase the volume`
-`decrease the volume`
- Current evidence:
- stock Jibo exposes volume control through the robot UX, so there should be an existing local path or service contract we can mirror
- this belongs with the other lightweight voice device controls rather than generic cloud chat
- Implementation notes:
- inspect the stock `@be` inventory and captures for volume-related intents, rules, or settings hooks
- prefer a local robot control payload over synthetic cloud speech
- decide whether first pass should support relative changes only, or also absolute requests like `set volume to 5`
- Exit criteria:
- voice increase and decrease commands adjust the robot volume reliably
- the behavior feels local and immediate, not like a chat reply
### 19. How Old Are You / Robot Age Persona
- Status: `discovery`
- Tags: `protocol`, `content`
- Why later: this is a strong personality/detail feature, but it may depend on first-power-up metadata or a stock persona path we have not mapped yet.
- User goals:
-`how old are you`
- age replies that sound like stock Jibo, including first-boot date and zodiac/personality flavor when available
- Current evidence:
- observed stock-style response from a YouTube transcript:
-`I was first powered up on January 31st, 2018, which makes me five days old. I'm an Aquarius.`
- this suggests the answer may be based on a stored first-powered-up date, not just a fixed build timestamp
- Implementation notes:
- inspect the stock `@be` inventory and captures for age, birthday, zodiac, or first-contact metadata hooks
- decide whether the first OpenJibo slice should:
- use stored robot first-boot / first-cloud-seen metadata
- compute age dynamically from that date
- optionally add zodiac flavor from the same date
- if no stock path is found, provide a cloud-powered fallback that still sounds native
- Exit criteria:
-`how old are you` returns a stable, personality-consistent answer
- the answer is grounded in stored robot lifecycle data instead of a hard-coded line
### 20. Command Vs Question Reply Style
- Status: `ready`
- Tags: `content`, `polish`
- Why later: Jibo historically responded differently when you commanded a skill versus when you asked about liking or wanting to do that skill, and that conversational nuance is part of what made him feel smart.
- User goals:
-`dance` or `do a dance` should sound like a willing action reply, then perform the skill
-`do you like to dance` should sound like an answer to the question first, not the same canned command reply
- Current evidence:
- observed behavior from stock Jibo:
- command-style `dance` -> something like `I like to dance` then dance
- question-style `do you like to dance?` -> something like `You bet I do`
- current OpenJibo skill replies are mostly canned by skill, without distinguishing question intent versus imperative intent
- Implementation notes:
- evolve simple reply collections into structured variants such as:
-`commandReplies`
-`questionReplies`
- optional `confirmationReplies`
- add a lightweight classifier for imperative versus question tone before reaching for a full LLM
- start with `dance`, then reuse the pattern for other expressive skills where stock Jibo clearly answered differently depending on phrasing
- keep the first version rule-based and cheap so it still works well before a future LLM-backed layer exists
- Exit criteria:
- at least one skill family such as `dance` gives distinct replies for command versus question forms
- the approach is reusable for other skill reply families without a large rewrite
