Compare commits
7 Commits
a9118c142f
...
ab47ad7a2d
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ab47ad7a2d | ||
|
|
2ea7afe2e7 | ||
|
|
efdb5bcf01 | ||
|
|
3a150faf4b | ||
|
|
d28d735d16 | ||
|
|
7b05452937 | ||
|
|
32d63584d6 |
@@ -8,6 +8,12 @@ It is intentionally broader than the current Node server. The Node server is a p
|
||||
|
||||
Day-to-day feature sequencing now lives in [feature-backlog.md](/C:/Projects/JiboExperiments/OpenJibo/docs/feature-backlog.md).
|
||||
|
||||
Cloud release hygiene:
|
||||
|
||||
- keep a visible OpenJibo Cloud version string
|
||||
- expose it through diagnostics such as `/health` and the spoken `cloud version` command
|
||||
- bump the shared version constant whenever we deploy a meaningful hosted-cloud change
|
||||
|
||||
## Current Scope
|
||||
|
||||
- stable .NET cloud scaffold
|
||||
@@ -158,6 +164,26 @@ Latest stock-OS WOD findings:
|
||||
- Spoken WOD guesses should preferentially snap to the closest offered hint when Whisper lands very close to one of the menu words, since near-misses like `haglet` for `aglet` are common in live testing.
|
||||
- The stock robot still misroutes constrained local turns if the cloud echoes `globals/*` rules back on the reply. For spoken WOD guesses and settings/update `no`, we should only return the local rule (`word-of-the-day/puzzle`, `settings/download_now_later`, etc.) so Global Service does not relaunch Nimbus.
|
||||
|
||||
Latest radio discovery findings:
|
||||
|
||||
- `@be/radio` is a true local skill, not a cloud placeholder.
|
||||
- Its `open(result, refresh, previousSkillName)` path treats `result.nlu.intent === "menu"` as a `play` launch.
|
||||
- `result.nlu.entities.station` is the genre selector, and `Country` is a real supported station key from the robot's `genres.json`.
|
||||
- The smallest stock-shaped cloud handoff for voice launch is therefore a local `SKILL_REDIRECT` to `@be/radio` with `nlu.intent = "menu"`, optional `entities.station`, and a silent completion to settle the hotphrase cloud response.
|
||||
|
||||
Latest news discovery findings:
|
||||
|
||||
- Nimbus explicitly treats `match.cloudSkill === "news"` like the GQA path and waits on `cloudSkillResponse`.
|
||||
- The first OpenJibo news pass should therefore use a real cloud-skill shape, not a generic placeholder chat reply.
|
||||
- For now, the content can stay synthetic while the protocol is grounded: `match.cloudSkill = "news"` plus a supported `SLIM` announcement response is enough to validate the robot path before provider-backed headlines arrive later.
|
||||
|
||||
Latest clock discovery findings:
|
||||
|
||||
- `@be/clock` is a real local skill with `clock`, `timer`, and `alarm` domains.
|
||||
- Menu launches use `intent = "menu"` with `entities.domain` set to the target sub-area.
|
||||
- Direct timer and alarm actions use `timerValue` and `alarmValue` utterances, not a generic chat path.
|
||||
- A practical first OpenJibo slice is therefore: keep custom spoken time/date answers for now, but route `open clock`, `open timer`, `open alarm`, `set a timer ...`, and `set an alarm ...` through stock-shaped local `@be/clock` handoffs.
|
||||
|
||||
## Speech, Animation, And ESML
|
||||
|
||||
The current joke flow is only a small foothold into Jibo expressiveness.
|
||||
|
||||
@@ -40,6 +40,7 @@ Parallel tags:
|
||||
- Current evidence:
|
||||
- [index.js](C:/Projects/JiboOs/V3.1/build/opt/jibo/Jibo/Skills/@be/be/node_modules/@be/radio/index.js) resumes from `lastStation`
|
||||
- the same file treats `menu` as a `play` launch and reads `result.nlu.entities.station`
|
||||
- the same file confirms `menu + no station` is the clean resume path and `menu + station=Country` becomes a direct genre launch
|
||||
- Implementation notes:
|
||||
- add phrase routing for radio open/resume and genre launch
|
||||
- inspect radio genre and station metadata before locking the outbound entity values
|
||||
@@ -88,6 +89,7 @@ Parallel tags:
|
||||
- the attached `jibo test 13` session includes both examples in one bundle:
|
||||
- a proactive or share-style prompt where spoken `yes` was treated as generic speech
|
||||
- a later update prompt where spoken `no` was accepted correctly
|
||||
- the share prompt uses `surprises-date/offer_date_fact` with `$YESNO`, and the failing reply leaked `globals/*` rules back into a Nimbus relaunch
|
||||
- Implementation notes:
|
||||
- compare the active listen rules, ASR hints, and local skill ownership for the share-style prompt versus OTA prompts
|
||||
- make constrained yes-no detection cover this prompt family without regressing the already-working update `no` path
|
||||
@@ -109,21 +111,29 @@ Parallel tags:
|
||||
- Implementation notes:
|
||||
- decide whether the first pass is a simple headline summary or a closer personal-report style payload
|
||||
- confirm whether stock OS expects `news` as a dedicated cloud skill or under the broader personal-report family
|
||||
- Latest progress:
|
||||
- first pass should use Nimbus's supported cloud path by setting `match.cloudSkill = news` and returning a supported `SLIM` announcement
|
||||
- provider-backed headlines can follow later under the `Lasso / Knowledge And Event Aggregation` track
|
||||
- Exit criteria:
|
||||
- `tell me the news` reaches a non-placeholder live path
|
||||
- robot behavior feels Nimbus-native rather than generic chat playback
|
||||
|
||||
### 6. Clock Family Audit
|
||||
|
||||
- Status: `ready`
|
||||
- Status: `in_progress`
|
||||
- Tags: `protocol`
|
||||
- Why now: clock, date, timer, and alarm menu hooks are already visible in captures and the robot repo has a real `@be/clock` skill.
|
||||
- Current evidence:
|
||||
- [protocol-inventory.md](C:/Projects/JiboExperiments/OpenJibo/docs/protocol-inventory.md) already tracks menu intents for `askForTime`, `askForDate`, `timerValue`, and `alarmValue`
|
||||
- `@be/clock` exists in the robot skill inventory
|
||||
- `JiboOs` shows `@be/clock` branches on `entities.domain = clock | timer | alarm`, uses `intent = menu` for menu launches, and accepts direct `timerValue` / `alarmValue` utterances with structured entities
|
||||
- Implementation notes:
|
||||
- compare our custom time/date path against actual menu payloads
|
||||
- decide whether timer and alarm should stay robot-local with cloud acknowledgement, or whether cloud needs to shape the launch and follow-up turns
|
||||
- Progress so far:
|
||||
- voice `open clock`, `open timer`, and `open alarm` now synthesize stock-shaped local `@be/clock` launches
|
||||
- voice `set a timer for five minutes` and `set an alarm for 7:30 am` now emit direct `timerValue` / `alarmValue` payloads with the domain and value entities the local skill expects
|
||||
- time/date remain on the existing custom cloud reply path for now
|
||||
- Exit criteria:
|
||||
- time/date behavior stays correct
|
||||
- timer and alarm launch or set correctly from both menu and voice where applicable
|
||||
@@ -161,7 +171,25 @@ Parallel tags:
|
||||
- what payload shape triggers the local animation / embodiment layer
|
||||
- whether the first pass should be cloud speech only or forecast plus presentation metadata
|
||||
|
||||
### 9. Surprises Routing
|
||||
### 9. Proactivity Selector And Surprise Offers
|
||||
|
||||
- Status: `discovery`
|
||||
- Tags: `protocol`, `content`, `docs`
|
||||
- Why later: the original architecture and recent proactive captures suggest proactivity is a first-class cloud subsystem, not just ordinary chat that starts itself.
|
||||
- Current evidence:
|
||||
- the attached original Jibo architecture diagram shows a cloud-side `Proactivity Selector`, `Proactivity Catalog`, and robot-side proactive trigger plumbing
|
||||
- [jibo test 13.txt](C:/Projects/JiboExperiments/artifact-output/jibo-test-13/jibo%20test%2013.txt) and its websocket artifacts show a proactive-style `I have something to share with you` offer and later proactive `TRIGGER` traffic
|
||||
- `@be/surprises`, `@be/surprises-date`, and `@be/surprises-ota` already exist as local robot-side building blocks
|
||||
- Questions to answer:
|
||||
- what minimum cloud-side selector we need for stock-OS-compatible surprise offers
|
||||
- how proactive `TRIGGER` traffic should map into a hosted OpenJibo proactivity service
|
||||
- whether `surprises-date/offer_date_fact` should be the first end-to-end proactive offer we intentionally support
|
||||
- Implementation notes:
|
||||
- model proactivity as its own orchestrator separate from ordinary conversational turn routing
|
||||
- include offer, constrained yes/no, fulfillment, and dismissal behavior in the design
|
||||
- preserve the artifact linkage to the original architecture diagram and `jibo-test-13`
|
||||
|
||||
### 10. Surprises Routing
|
||||
|
||||
- Status: `discovery`
|
||||
- Tags: `protocol`, `content`
|
||||
@@ -174,7 +202,40 @@ Parallel tags:
|
||||
- which categories still depend on cloud services versus fully local logic
|
||||
- whether stock OS `1.9` differs materially from the `3.1` source snapshot here
|
||||
|
||||
### 10. Personal Report, Calendar, And Commute
|
||||
### 11. History / Memory Layer
|
||||
|
||||
- Status: `discovery`
|
||||
- Tags: `content`, `docs`
|
||||
- Why later: the original architecture explicitly calls out `History`, and that likely maps to the kind of durable personal memory we want for names, preferences, and remembered facts.
|
||||
- Current evidence:
|
||||
- the attached original Jibo architecture diagram includes a dedicated `History` component in cloud storage
|
||||
- stock Jibo behavior historically included awareness of names, birthdays, holidays, and special dates
|
||||
- Questions to answer:
|
||||
- what data belongs in memory versus account/profile versus skill-specific storage
|
||||
- how much of the original behavior was robot-local versus cloud-backed
|
||||
- what the first safe OpenJibo memory slice should be
|
||||
- Implementation notes:
|
||||
- plan for person identity, preferred name, birthday, relationship facts, and notable dates
|
||||
- keep the first design privacy-aware and easy to host
|
||||
- treat this as shared infrastructure that other skills can consume rather than a standalone feature
|
||||
|
||||
### 12. Lasso / Knowledge And Event Aggregation
|
||||
|
||||
- Status: `discovery`
|
||||
- Tags: `content`
|
||||
- Why later: the original architecture diagram suggests `Lasso` sits between the hub and outside data sources, which likely explains how Jibo knew about news, calendar items, holidays, and other structured world events.
|
||||
- Current evidence:
|
||||
- the attached original Jibo architecture diagram shows `Lasso` connected to 3rd-party data such as AP News, Dark Sky, GCalendar, Wolfram, and other external sources
|
||||
- stock Jibo behavior historically covered holidays, birthdays, special events, and topical knowledge
|
||||
- Questions to answer:
|
||||
- whether `Lasso` should be recreated as a single aggregation service or as several focused providers behind a shared interface
|
||||
- which parts are needed for news, weather, calendar, commute, astrology/date facts, and holidays
|
||||
- what subset is practical for a hosted OpenJibo v1
|
||||
- Implementation notes:
|
||||
- treat holidays and special dates as first-class backlog scope here
|
||||
- use this item to drive future provider work for news, weather, calendar, commute, and event awareness
|
||||
|
||||
### 13. Personal Report, Calendar, And Commute
|
||||
|
||||
- Status: `discovery`
|
||||
- Tags: `protocol`, `content`
|
||||
@@ -185,10 +246,19 @@ Parallel tags:
|
||||
- Questions to answer:
|
||||
- should calendar and commute be independent feature paths or sections inside personal report
|
||||
- what minimum provider data shape lets Jibo present these naturally
|
||||
|
||||
### 14. Stop Command
|
||||
- Status: `ready`
|
||||
- Tags: `protocol`
|
||||
- Why later: Jibo can be interrupted by any command, but it would be nice to have a dedicated "stop" type of command.
|
||||
- Current evidence:
|
||||
- There is an Idle skill or subskill under @be so I think we can utilize it, but I am not sure if that was the default behavior.
|
||||
- Questions to answer:
|
||||
- Can we find in the original source evidence for this skill or stop word phrase?
|
||||
|
||||
## Support Tracks
|
||||
|
||||
### 11. Hosted Capture And Storage Plan
|
||||
### 15. Hosted Capture And Storage Plan
|
||||
|
||||
- Status: `ready`
|
||||
- Tags: `docs`
|
||||
@@ -197,7 +267,7 @@ Parallel tags:
|
||||
- define a clean boundary between local capture sinks and hosted archival/export
|
||||
- document how group testers should submit sessions without touching repo paths directly
|
||||
|
||||
### 12. STT Upgrade And Noise Screening
|
||||
### 16. STT Upgrade And Noise Screening
|
||||
|
||||
- Status: `ready`
|
||||
- Tags: `stt`
|
||||
@@ -219,6 +289,9 @@ Parallel tags:
|
||||
6. Clock family
|
||||
7. Photo family
|
||||
8. Weather
|
||||
9. Surprises
|
||||
10. Personal report, calendar, and commute
|
||||
11. Hosted capture/storage and STT improvements as parallel tracks
|
||||
9. Proactivity selector and surprise offers
|
||||
10. Surprises
|
||||
11. History / memory layer
|
||||
12. Lasso / knowledge and event aggregation
|
||||
13. Personal report, calendar, and commute
|
||||
14. Hosted capture/storage and STT improvements as parallel tracks
|
||||
|
||||
@@ -6,6 +6,13 @@
|
||||
|
||||
This is the production-oriented path for restoring device connectivity and creating a foundation for future runtime, AI, and OTA work.
|
||||
|
||||
Current spoken cloud version: `Open Jibo Cloud version 1.0.10.`
|
||||
|
||||
Release hygiene reminder:
|
||||
|
||||
- bump [OpenJiboCloudBuildInfo.cs](/C:/Projects/JiboExperiments/OpenJibo/src/Jibo.Cloud/dotnet/src/Jibo.Cloud.Application/Services/OpenJiboCloudBuildInfo.cs) whenever we ship a meaningful hosted-cloud update
|
||||
- keep the spoken version response and `/health` version field aligned from that single source of truth
|
||||
|
||||
## Architecture
|
||||
|
||||
The first implementation is a modular monolith:
|
||||
|
||||
@@ -125,7 +125,12 @@ app.Use(async (context, next) =>
|
||||
await telemetrySink.RecordConnectionClosedAsync(closeEnvelope, closeSession, $"socket-loop-ended{(isPrematureClose ? "-prematurely" : string.Empty)}", context.RequestAborted);
|
||||
});
|
||||
|
||||
app.MapGet("/health", () => Results.Json(new { ok = true, service = "OpenJibo Cloud Api" }));
|
||||
app.MapGet("/health", () => Results.Json(new
|
||||
{
|
||||
ok = true,
|
||||
service = "OpenJibo Cloud Api",
|
||||
version = OpenJiboCloudBuildInfo.Version
|
||||
}));
|
||||
|
||||
app.MapMethods("/{**path}", ["GET", "POST", "PUT"], async (HttpContext context, JiboCloudProtocolService service, IProtocolTelemetrySink telemetrySink, CancellationToken cancellationToken) =>
|
||||
{
|
||||
|
||||
@@ -17,5 +17,6 @@ public sealed class JiboExperienceCatalog
|
||||
public IReadOnlyList<string> CalendarReplies { get; init; } = [];
|
||||
public IReadOnlyList<string> CommuteReplies { get; init; } = [];
|
||||
public IReadOnlyList<string> NewsReplies { get; init; } = [];
|
||||
public IReadOnlyList<string> NewsBriefings { get; init; } = [];
|
||||
public IReadOnlyList<string> GenericFallbackReplies { get; init; } = [];
|
||||
}
|
||||
|
||||
@@ -72,6 +72,14 @@ public sealed class DemoConversationBroker(JiboInteractionService interactionSer
|
||||
{
|
||||
"word_of_the_day" => false,
|
||||
"word_of_the_day_guess" => false,
|
||||
"radio" => false,
|
||||
"radio_genre" => false,
|
||||
"clock_menu" => false,
|
||||
"timer_menu" => false,
|
||||
"alarm_menu" => false,
|
||||
"timer_value" => false,
|
||||
"alarm_value" => false,
|
||||
"news" => false,
|
||||
_ => true
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
using Jibo.Cloud.Application.Abstractions;
|
||||
using Jibo.Runtime.Abstractions;
|
||||
using System.Text.Json;
|
||||
using System.Text.RegularExpressions;
|
||||
|
||||
namespace Jibo.Cloud.Application.Services;
|
||||
|
||||
@@ -29,6 +30,15 @@ public sealed class JiboInteractionService(
|
||||
"dance" => BuildDanceDecision(catalog),
|
||||
"time" => new JiboInteractionDecision("time", $"It is {DateTime.Now:h:mm tt}."),
|
||||
"date" => new JiboInteractionDecision("date", $"Today is {DateTime.Now:dddd, MMMM d}."),
|
||||
"day" => new JiboInteractionDecision("day", $"Today is {DateTime.Now:dddd}."),
|
||||
"cloud_version" => new JiboInteractionDecision("cloud_version", OpenJiboCloudBuildInfo.SpokenVersion),
|
||||
"radio" => BuildRadioLaunchDecision(),
|
||||
"radio_genre" => BuildRadioGenreLaunchDecision(lowered),
|
||||
"clock_menu" => BuildClockLaunchDecision("clock", "Opening the clock."),
|
||||
"timer_menu" => BuildClockLaunchDecision("timer", "Opening the timer."),
|
||||
"alarm_menu" => BuildClockLaunchDecision("alarm", "Opening the alarm."),
|
||||
"timer_value" => BuildTimerValueDecision(lowered),
|
||||
"alarm_value" => BuildAlarmValueDecision(lowered),
|
||||
"hello" => new JiboInteractionDecision("hello", randomizer.Choose(catalog.GreetingReplies)),
|
||||
"how_are_you" => new JiboInteractionDecision("how_are_you", randomizer.Choose(catalog.HowAreYouReplies)),
|
||||
"yes" => new JiboInteractionDecision("yes", "Yes."),
|
||||
@@ -40,7 +50,7 @@ public sealed class JiboInteractionService(
|
||||
"weather" => new JiboInteractionDecision("weather", randomizer.Choose(catalog.WeatherReplies)),
|
||||
"calendar" => new JiboInteractionDecision("calendar", randomizer.Choose(catalog.CalendarReplies)),
|
||||
"commute" => new JiboInteractionDecision("commute", randomizer.Choose(catalog.CommuteReplies)),
|
||||
"news" => new JiboInteractionDecision("news", randomizer.Choose(catalog.NewsReplies)),
|
||||
"news" => BuildNewsDecision(catalog),
|
||||
_ => new JiboInteractionDecision("chat", BuildGenericReply(catalog, transcript, lowered))
|
||||
};
|
||||
}
|
||||
@@ -73,6 +83,22 @@ public sealed class JiboInteractionService(
|
||||
});
|
||||
}
|
||||
|
||||
private JiboInteractionDecision BuildNewsDecision(JiboExperienceCatalog catalog)
|
||||
{
|
||||
var briefing = randomizer.Choose(catalog.NewsBriefings);
|
||||
return new JiboInteractionDecision(
|
||||
"news",
|
||||
briefing,
|
||||
"news",
|
||||
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
["skillId"] = "news",
|
||||
["cloudSkill"] = "news",
|
||||
["mim_id"] = "runtime-news",
|
||||
["mim_type"] = "announcement"
|
||||
});
|
||||
}
|
||||
|
||||
private string BuildGenericReply(JiboExperienceCatalog catalog, string transcript, string lowered)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(transcript))
|
||||
@@ -130,6 +156,33 @@ public sealed class JiboInteractionService(
|
||||
return "date";
|
||||
}
|
||||
|
||||
if (string.Equals(clientIntent, "askForDay", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
return "day";
|
||||
}
|
||||
|
||||
if (string.Equals(clientIntent, "timerValue", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
return "timer_value";
|
||||
}
|
||||
|
||||
if (string.Equals(clientIntent, "alarmValue", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
return "alarm_value";
|
||||
}
|
||||
|
||||
if (string.Equals(clientIntent, "menu", StringComparison.OrdinalIgnoreCase) &&
|
||||
clientEntities.TryGetValue("domain", out var clockDomain))
|
||||
{
|
||||
return clockDomain.ToLowerInvariant() switch
|
||||
{
|
||||
"clock" => "clock_menu",
|
||||
"timer" => "timer_menu",
|
||||
"alarm" => "alarm_menu",
|
||||
_ => "chat"
|
||||
};
|
||||
}
|
||||
|
||||
if (MatchesAny(
|
||||
loweredTranscript,
|
||||
"word of the day",
|
||||
@@ -151,6 +204,53 @@ public sealed class JiboInteractionService(
|
||||
return "joke";
|
||||
}
|
||||
|
||||
if (MatchesAny(
|
||||
loweredTranscript,
|
||||
"cloud version",
|
||||
"open jibo cloud version",
|
||||
"openjibo cloud version",
|
||||
"what version is the cloud",
|
||||
"what s the cloud version",
|
||||
"what's the cloud version"))
|
||||
{
|
||||
return "cloud_version";
|
||||
}
|
||||
|
||||
if (TryResolveRadioGenre(loweredTranscript) is not null)
|
||||
{
|
||||
return "radio_genre";
|
||||
}
|
||||
|
||||
if (TryParseAlarmValue(loweredTranscript) is not null)
|
||||
{
|
||||
return "alarm_value";
|
||||
}
|
||||
|
||||
if (TryParseTimerValue(loweredTranscript) is not null)
|
||||
{
|
||||
return "timer_value";
|
||||
}
|
||||
|
||||
if (MatchesAny(loweredTranscript, "open the clock", "open clock", "show the clock", "show clock"))
|
||||
{
|
||||
return "clock_menu";
|
||||
}
|
||||
|
||||
if (MatchesAny(loweredTranscript, "open the timer", "open timer", "show the timer", "show timer"))
|
||||
{
|
||||
return "timer_menu";
|
||||
}
|
||||
|
||||
if (MatchesAny(loweredTranscript, "open the alarm", "open alarm", "show the alarm", "show alarm"))
|
||||
{
|
||||
return "alarm_menu";
|
||||
}
|
||||
|
||||
if (MatchesAny(loweredTranscript, "open the radio", "play the radio", "turn on the radio", "radio"))
|
||||
{
|
||||
return "radio";
|
||||
}
|
||||
|
||||
if (MatchesAny(loweredTranscript, "dance", "boogie"))
|
||||
{
|
||||
return "dance";
|
||||
@@ -212,6 +312,11 @@ public sealed class JiboInteractionService(
|
||||
return "time";
|
||||
}
|
||||
|
||||
if (MatchesAny(loweredTranscript, "what day is it", "what day is today"))
|
||||
{
|
||||
return "day";
|
||||
}
|
||||
|
||||
if (MatchesAny(loweredTranscript, "what day is it", "what is the date", "today s date", "today's date") ||
|
||||
loweredTranscript.Contains("date", StringComparison.Ordinal) ||
|
||||
loweredTranscript.Contains("day", StringComparison.Ordinal))
|
||||
@@ -235,6 +340,84 @@ public sealed class JiboInteractionService(
|
||||
});
|
||||
}
|
||||
|
||||
private static JiboInteractionDecision BuildRadioLaunchDecision()
|
||||
{
|
||||
return new JiboInteractionDecision(
|
||||
"radio",
|
||||
"Opening the radio.",
|
||||
"@be/radio",
|
||||
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
["skillId"] = "@be/radio"
|
||||
});
|
||||
}
|
||||
|
||||
private static JiboInteractionDecision BuildClockLaunchDecision(string domain, string replyText)
|
||||
{
|
||||
return new JiboInteractionDecision(
|
||||
$"{domain}_menu",
|
||||
replyText,
|
||||
"@be/clock",
|
||||
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
["skillId"] = "@be/clock",
|
||||
["domain"] = domain,
|
||||
["clockIntent"] = "menu"
|
||||
});
|
||||
}
|
||||
|
||||
private static JiboInteractionDecision BuildTimerValueDecision(string loweredTranscript)
|
||||
{
|
||||
var timer = TryParseTimerValue(loweredTranscript) ?? new ClockTimerValue("0", "1", "null");
|
||||
|
||||
return new JiboInteractionDecision(
|
||||
"timer_value",
|
||||
"Setting your timer.",
|
||||
"@be/clock",
|
||||
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
["skillId"] = "@be/clock",
|
||||
["domain"] = "timer",
|
||||
["clockIntent"] = "timerValue",
|
||||
["hours"] = timer.Hours,
|
||||
["minutes"] = timer.Minutes,
|
||||
["seconds"] = timer.Seconds
|
||||
});
|
||||
}
|
||||
|
||||
private static JiboInteractionDecision BuildAlarmValueDecision(string loweredTranscript)
|
||||
{
|
||||
var alarm = TryParseAlarmValue(loweredTranscript) ?? new ClockAlarmValue("7:00", "am");
|
||||
|
||||
return new JiboInteractionDecision(
|
||||
"alarm_value",
|
||||
"Setting your alarm.",
|
||||
"@be/clock",
|
||||
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
["skillId"] = "@be/clock",
|
||||
["domain"] = "alarm",
|
||||
["clockIntent"] = "alarmValue",
|
||||
["time"] = alarm.Time,
|
||||
["ampm"] = alarm.AmPm
|
||||
});
|
||||
}
|
||||
|
||||
private static JiboInteractionDecision BuildRadioGenreLaunchDecision(string loweredTranscript)
|
||||
{
|
||||
var station = TryResolveRadioGenre(loweredTranscript) ?? "Country";
|
||||
|
||||
return new JiboInteractionDecision(
|
||||
"radio_genre",
|
||||
$"Playing {FormatRadioGenreForSpeech(station)} on the radio.",
|
||||
"@be/radio",
|
||||
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
["skillId"] = "@be/radio",
|
||||
["station"] = station
|
||||
});
|
||||
}
|
||||
|
||||
private static JiboInteractionDecision BuildWordOfTheDayGuessDecision(
|
||||
IReadOnlyDictionary<string, string> clientEntities,
|
||||
string transcript,
|
||||
@@ -301,6 +484,7 @@ public sealed class JiboInteractionService(
|
||||
string.Equals(rule, "$YESNO", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "create/is_it_a_keeper", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "settings/download_now_later", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "surprises-date/offer_date_fact", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "surprises-ota/want_to_download_now", StringComparison.OrdinalIgnoreCase));
|
||||
}
|
||||
|
||||
@@ -417,6 +601,176 @@ public sealed class JiboInteractionService(
|
||||
{
|
||||
return candidates.Any(candidate => loweredTranscript.Contains(candidate, StringComparison.Ordinal));
|
||||
}
|
||||
|
||||
private static string? TryResolveRadioGenre(string loweredTranscript)
|
||||
{
|
||||
foreach (var (phrase, station) in RadioGenreAliases)
|
||||
{
|
||||
if (loweredTranscript.Contains(phrase, StringComparison.Ordinal))
|
||||
{
|
||||
return station;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
private static string FormatRadioGenreForSpeech(string station)
|
||||
{
|
||||
return station switch
|
||||
{
|
||||
"EightiesAndNinetiesHits" => "eighties and nineties hits",
|
||||
"ChristianAndGospel" => "Christian and gospel",
|
||||
"ClassicRock" => "classic rock",
|
||||
"CollegeRadio" => "college radio",
|
||||
"HipHop" => "hip hop",
|
||||
"NewsAndTalk" => "news and talk",
|
||||
"ReggaeAndIsland" => "reggae and island music",
|
||||
"SoftRock" => "soft rock",
|
||||
_ => station
|
||||
};
|
||||
}
|
||||
|
||||
private static ClockTimerValue? TryParseTimerValue(string loweredTranscript)
|
||||
{
|
||||
if (!loweredTranscript.Contains("timer", StringComparison.Ordinal))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
var hours = ExtractDurationValue(loweredTranscript, "hour");
|
||||
var minutes = ExtractDurationValue(loweredTranscript, "minute");
|
||||
var seconds = ExtractDurationValue(loweredTranscript, "second");
|
||||
|
||||
if (hours is null && minutes is null && seconds is null)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return new ClockTimerValue(
|
||||
(hours ?? 0).ToString(),
|
||||
(minutes ?? 0).ToString(),
|
||||
seconds is null ? "null" : seconds.Value.ToString());
|
||||
}
|
||||
|
||||
private static ClockAlarmValue? TryParseAlarmValue(string loweredTranscript)
|
||||
{
|
||||
if (!loweredTranscript.Contains("alarm", StringComparison.Ordinal))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
var match = AlarmPattern.Match(loweredTranscript);
|
||||
if (!match.Success)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
var hourToken = match.Groups["hour"].Value;
|
||||
var minuteToken = match.Groups["minute"].Success ? match.Groups["minute"].Value : "00";
|
||||
var hour = ParseNumberToken(hourToken);
|
||||
if (hour is null || hour is < 1 or > 12)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
if (!int.TryParse(minuteToken, out var minute) || minute is < 0 or > 59)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
var ampm = match.Groups["ampm"].Value.StartsWith("p", StringComparison.Ordinal) ? "pm" : "am";
|
||||
return new ClockAlarmValue($"{hour}:{minute:00}", ampm);
|
||||
}
|
||||
|
||||
private static int? ExtractDurationValue(string loweredTranscript, string unitStem)
|
||||
{
|
||||
var pattern = new Regex($@"\b(?<value>\d+|[a-z\-]+)\s+{unitStem}s?\b", RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);
|
||||
var match = pattern.Match(loweredTranscript);
|
||||
if (!match.Success)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return ParseNumberToken(match.Groups["value"].Value);
|
||||
}
|
||||
|
||||
private static int? ParseNumberToken(string token)
|
||||
{
|
||||
var normalized = token.Trim().ToLowerInvariant();
|
||||
if (int.TryParse(normalized, out var numeric))
|
||||
{
|
||||
return numeric;
|
||||
}
|
||||
|
||||
return normalized switch
|
||||
{
|
||||
"a" or "an" => 1,
|
||||
"one" => 1,
|
||||
"two" => 2,
|
||||
"three" => 3,
|
||||
"four" => 4,
|
||||
"five" => 5,
|
||||
"six" => 6,
|
||||
"seven" => 7,
|
||||
"eight" => 8,
|
||||
"nine" => 9,
|
||||
"ten" => 10,
|
||||
"eleven" => 11,
|
||||
"twelve" => 12,
|
||||
"thirteen" => 13,
|
||||
"fourteen" => 14,
|
||||
"fifteen" => 15,
|
||||
"sixteen" => 16,
|
||||
"seventeen" => 17,
|
||||
"eighteen" => 18,
|
||||
"nineteen" => 19,
|
||||
"twenty" => 20,
|
||||
"thirty" => 30,
|
||||
"forty" => 40,
|
||||
"fifty" => 50,
|
||||
_ => null
|
||||
};
|
||||
}
|
||||
|
||||
private sealed record ClockTimerValue(string Hours, string Minutes, string Seconds);
|
||||
|
||||
private sealed record ClockAlarmValue(string Time, string AmPm);
|
||||
|
||||
private static readonly Regex AlarmPattern = new(
|
||||
@"\b(?<hour>\d{1,2}|one|two|three|four|five|six|seven|eight|nine|ten|eleven|twelve)(?:[:\s](?<minute>\d{2}))?\s*(?<ampm>a\.?m\.?|p\.?m\.?)\b",
|
||||
RegexOptions.IgnoreCase | RegexOptions.CultureInvariant | RegexOptions.Compiled);
|
||||
|
||||
private static readonly (string Phrase, string Station)[] RadioGenreAliases =
|
||||
[
|
||||
("country music", "Country"),
|
||||
("country radio", "Country"),
|
||||
("country", "Country"),
|
||||
("classic rock", "ClassicRock"),
|
||||
("soft rock", "SoftRock"),
|
||||
("hip hop", "HipHop"),
|
||||
("hip-hop", "HipHop"),
|
||||
("news and talk", "NewsAndTalk"),
|
||||
("news talk", "NewsAndTalk"),
|
||||
("news radio", "NewsAndTalk"),
|
||||
("sports radio", "Sports"),
|
||||
("christian music", "ChristianAndGospel"),
|
||||
("gospel music", "ChristianAndGospel"),
|
||||
("oldies", "Oldies"),
|
||||
("pop music", "Pop"),
|
||||
("jazz", "Jazz"),
|
||||
("latin music", "Latin"),
|
||||
("dance music", "Dance"),
|
||||
("reggae", "ReggaeAndIsland"),
|
||||
("island music", "ReggaeAndIsland"),
|
||||
("alternative", "Alternative"),
|
||||
("blues", "Blues"),
|
||||
("classical music", "Classical"),
|
||||
("classical", "Classical"),
|
||||
("college radio", "CollegeRadio"),
|
||||
("comedy radio", "Comedy"),
|
||||
("npr", "NPR")
|
||||
];
|
||||
}
|
||||
|
||||
public sealed record JiboInteractionDecision(
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
namespace Jibo.Cloud.Application.Services;
|
||||
|
||||
public static class OpenJiboCloudBuildInfo
|
||||
{
|
||||
public const string Version = "1.0.10";
|
||||
|
||||
public static string SpokenVersion => $"Open Jibo Cloud version {Version}.";
|
||||
}
|
||||
@@ -23,10 +23,26 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
||||
string.Equals(plan.IntentName, "no", StringComparison.OrdinalIgnoreCase);
|
||||
var isWordOfDayLaunch = string.Equals(plan.IntentName, "word_of_the_day", StringComparison.OrdinalIgnoreCase);
|
||||
var isWordOfDayGuess = string.Equals(plan.IntentName, "word_of_the_day_guess", StringComparison.OrdinalIgnoreCase);
|
||||
var isRadioLaunch = string.Equals(plan.IntentName, "radio", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(plan.IntentName, "radio_genre", StringComparison.OrdinalIgnoreCase);
|
||||
var isClockSkillLaunch = string.Equals(skill?.SkillName, "@be/clock", StringComparison.OrdinalIgnoreCase);
|
||||
var clockIntent = ReadSkillPayloadString(skill, "clockIntent");
|
||||
var clockDomain = ReadSkillPayloadString(skill, "domain");
|
||||
var timerHours = ReadSkillPayloadString(skill, "hours");
|
||||
var timerMinutes = ReadSkillPayloadString(skill, "minutes");
|
||||
var timerSeconds = ReadSkillPayloadString(skill, "seconds");
|
||||
var alarmTime = ReadSkillPayloadString(skill, "time");
|
||||
var alarmAmPm = ReadSkillPayloadString(skill, "ampm");
|
||||
var radioStation = ReadSkillPayloadString(skill, "station");
|
||||
var cloudSkill = ReadSkillPayloadString(skill, "cloudSkill");
|
||||
var nluGuess = ReadClientEntity(turn, "guess");
|
||||
var wordOfDayGuess = ResolveWordOfDayGuess(turn, transcript, nluGuess);
|
||||
var outboundIntent = isWordOfDayLaunch
|
||||
? "menu"
|
||||
: isRadioLaunch
|
||||
? "menu"
|
||||
: isClockSkillLaunch && !string.IsNullOrWhiteSpace(clockIntent)
|
||||
? clockIntent
|
||||
: isWordOfDayGuess
|
||||
? "guess"
|
||||
: string.Equals(messageType, "CLIENT_NLU", StringComparison.OrdinalIgnoreCase) && !string.IsNullOrWhiteSpace(clientIntent)
|
||||
@@ -36,6 +52,10 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
||||
? wordOfDayGuess
|
||||
: isWordOfDayLaunch
|
||||
? string.Empty
|
||||
: isRadioLaunch
|
||||
? transcript
|
||||
: isClockSkillLaunch
|
||||
? transcript
|
||||
: string.Equals(clientIntent, "guess", StringComparison.OrdinalIgnoreCase) && !string.IsNullOrWhiteSpace(nluGuess)
|
||||
? nluGuess
|
||||
: isYesNoTurn && isYesNoIntent
|
||||
@@ -45,10 +65,31 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
||||
: transcript;
|
||||
var outboundRules = isWordOfDayLaunch
|
||||
? ["word-of-the-day/menu"]
|
||||
: isRadioLaunch
|
||||
? Array.Empty<string>()
|
||||
: isClockSkillLaunch
|
||||
? string.Equals(messageType, "CLIENT_NLU", StringComparison.OrdinalIgnoreCase) ? rules : Array.Empty<string>()
|
||||
: isWordOfDayGuess
|
||||
? ["word-of-the-day/puzzle"]
|
||||
: isYesNoTurn && isYesNoIntent ? [yesNoRule!] : rules;
|
||||
var entities = ReadEntities(turn, messageType, isYesNoTurn && isYesNoIntent, isWordOfDayLaunch, isWordOfDayGuess, wordOfDayGuess);
|
||||
var entities = ReadEntities(
|
||||
turn,
|
||||
messageType,
|
||||
isYesNoTurn && isYesNoIntent,
|
||||
ShouldIncludeCreateDomain(yesNoRule),
|
||||
isWordOfDayLaunch,
|
||||
isRadioLaunch,
|
||||
isWordOfDayGuess,
|
||||
wordOfDayGuess,
|
||||
radioStation,
|
||||
isClockSkillLaunch,
|
||||
clockDomain,
|
||||
clockIntent,
|
||||
timerHours,
|
||||
timerMinutes,
|
||||
timerSeconds,
|
||||
alarmTime,
|
||||
alarmAmPm);
|
||||
var listenMessage = new
|
||||
{
|
||||
type = "LISTEN",
|
||||
@@ -61,12 +102,20 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
||||
final = true,
|
||||
text = outboundAsrText
|
||||
},
|
||||
nlu = BuildNluPayload(outboundIntent, outboundRules, entities, isWordOfDayLaunch ? "@be/word-of-the-day" : null),
|
||||
nlu = BuildNluPayload(
|
||||
outboundIntent,
|
||||
outboundRules,
|
||||
entities,
|
||||
isWordOfDayLaunch ? "@be/word-of-the-day" :
|
||||
isRadioLaunch ? "@be/radio" :
|
||||
isClockSkillLaunch ? "@be/clock" :
|
||||
null),
|
||||
match = new
|
||||
{
|
||||
intent = outboundIntent,
|
||||
rule = outboundRules.FirstOrDefault() ?? string.Empty,
|
||||
score = 0.95
|
||||
score = 0.95,
|
||||
cloudSkill
|
||||
}
|
||||
}
|
||||
};
|
||||
@@ -100,6 +149,39 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
||||
DelayMs: 125));
|
||||
}
|
||||
|
||||
if (isRadioLaunch)
|
||||
{
|
||||
messages.Add(new SocketReplyPlan(
|
||||
JsonSerializer.Serialize(BuildSkillRedirectPayload(
|
||||
transId,
|
||||
"@be/radio",
|
||||
outboundIntent,
|
||||
outboundAsrText,
|
||||
outboundRules,
|
||||
entities)),
|
||||
DelayMs: 75));
|
||||
messages.Add(new SocketReplyPlan(
|
||||
JsonSerializer.Serialize(BuildCompletionOnlySkillPayload(transId, "@be/radio")),
|
||||
DelayMs: 125));
|
||||
}
|
||||
|
||||
if (isClockSkillLaunch &&
|
||||
!string.Equals(messageType, "CLIENT_NLU", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
messages.Add(new SocketReplyPlan(
|
||||
JsonSerializer.Serialize(BuildSkillRedirectPayload(
|
||||
transId,
|
||||
"@be/clock",
|
||||
outboundIntent,
|
||||
outboundAsrText,
|
||||
outboundRules,
|
||||
entities)),
|
||||
DelayMs: 75));
|
||||
messages.Add(new SocketReplyPlan(
|
||||
JsonSerializer.Serialize(BuildCompletionOnlySkillPayload(transId, "@be/clock")),
|
||||
DelayMs: 125));
|
||||
}
|
||||
|
||||
if (emitSkillActions && speak is not null)
|
||||
{
|
||||
messages.Add(new SocketReplyPlan(
|
||||
@@ -240,13 +322,29 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
||||
private static object ReadEntities(
|
||||
TurnContext turn,
|
||||
string? messageType,
|
||||
bool yesNoCreateTurn,
|
||||
bool yesNoTurn,
|
||||
bool includeCreateDomain,
|
||||
bool wordOfDayLaunch,
|
||||
bool radioLaunch,
|
||||
bool wordOfDayGuess,
|
||||
string? guess)
|
||||
string? guess,
|
||||
string? radioStation,
|
||||
bool clockSkillLaunch,
|
||||
string? clockDomain,
|
||||
string? clockIntent,
|
||||
string? timerHours,
|
||||
string? timerMinutes,
|
||||
string? timerSeconds,
|
||||
string? alarmTime,
|
||||
string? alarmAmPm)
|
||||
{
|
||||
if (yesNoCreateTurn)
|
||||
if (yesNoTurn)
|
||||
{
|
||||
if (!includeCreateDomain)
|
||||
{
|
||||
return new Dictionary<string, object?>();
|
||||
}
|
||||
|
||||
return new Dictionary<string, object?>
|
||||
{
|
||||
["domain"] = "create"
|
||||
@@ -261,6 +359,41 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
||||
};
|
||||
}
|
||||
|
||||
if (radioLaunch)
|
||||
{
|
||||
var entities = new Dictionary<string, object?>();
|
||||
if (!string.IsNullOrWhiteSpace(radioStation))
|
||||
{
|
||||
entities["station"] = radioStation;
|
||||
}
|
||||
|
||||
return entities;
|
||||
}
|
||||
|
||||
if (clockSkillLaunch)
|
||||
{
|
||||
var entities = new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase);
|
||||
if (!string.IsNullOrWhiteSpace(clockDomain))
|
||||
{
|
||||
entities["domain"] = clockDomain;
|
||||
}
|
||||
|
||||
if (string.Equals(clockIntent, "timerValue", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
entities["hours"] = timerHours ?? "0";
|
||||
entities["minutes"] = timerMinutes ?? "0";
|
||||
entities["seconds"] = timerSeconds ?? "null";
|
||||
}
|
||||
|
||||
if (string.Equals(clockIntent, "alarmValue", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
entities["time"] = alarmTime ?? string.Empty;
|
||||
entities["ampm"] = alarmAmPm ?? string.Empty;
|
||||
}
|
||||
|
||||
return entities;
|
||||
}
|
||||
|
||||
if (wordOfDayGuess)
|
||||
{
|
||||
return new Dictionary<string, object?>
|
||||
@@ -293,9 +426,16 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
||||
.FirstOrDefault(static rule =>
|
||||
string.Equals(rule, "create/is_it_a_keeper", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "settings/download_now_later", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "surprises-date/offer_date_fact", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "surprises-ota/want_to_download_now", StringComparison.OrdinalIgnoreCase));
|
||||
}
|
||||
|
||||
private static bool ShouldIncludeCreateDomain(string? yesNoRule)
|
||||
{
|
||||
return string.Equals(yesNoRule, "create/is_it_a_keeper", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(yesNoRule, "surprises-ota/want_to_download_now", StringComparison.OrdinalIgnoreCase);
|
||||
}
|
||||
|
||||
private static IEnumerable<string> ReadRuleValues(TurnContext turn)
|
||||
{
|
||||
return ReadRuleValues(turn, "listenRules").Concat(ReadRuleValues(turn, "clientRules"));
|
||||
@@ -677,8 +817,7 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
||||
.Replace("&", "&", StringComparison.Ordinal)
|
||||
.Replace("<", "<", StringComparison.Ordinal)
|
||||
.Replace(">", ">", StringComparison.Ordinal)
|
||||
.Replace("\"", """, StringComparison.Ordinal)
|
||||
.Replace("'", "'", StringComparison.Ordinal);
|
||||
.Replace("\"", """, StringComparison.Ordinal);
|
||||
}
|
||||
|
||||
private static string? ReadPayloadString(IDictionary<string, object?>? payload, string key)
|
||||
|
||||
@@ -493,6 +493,24 @@ public sealed class WebSocketTurnFinalizationService(
|
||||
turnState.FinalizeAttemptCount += 1;
|
||||
}
|
||||
|
||||
if (allowFallbackOnMissingTranscript &&
|
||||
turnState.BufferedAudioBytes >= AutoFinalizeMinBufferedAudioBytes &&
|
||||
IsYesNoTurn(finalizedTurn))
|
||||
{
|
||||
turnState.AwaitingTurnCompletion = false;
|
||||
session.LastTranscript = string.Empty;
|
||||
session.LastIntent = null;
|
||||
session.LastListenType = "no-input";
|
||||
var localRule = ReadPrimaryYesNoRule(finalizedTurn);
|
||||
var noInputReplies = ResponsePlanToSocketMessagesMapper.MapNoInput(
|
||||
turnState.TransId ?? session.LastTransId ?? string.Empty,
|
||||
string.IsNullOrWhiteSpace(localRule) ? turnState.ListenRules : [localRule])
|
||||
.Select(map => new WebSocketReply { Text = map.Text, DelayMs = map.DelayMs })
|
||||
.ToArray();
|
||||
ResetBufferedAudio(session);
|
||||
return noInputReplies;
|
||||
}
|
||||
|
||||
if (allowFallbackOnMissingTranscript &&
|
||||
turnState.BufferedAudioBytes >= AutoFinalizeMinBufferedAudioBytes &&
|
||||
string.IsNullOrWhiteSpace(turnState.LastSttError))
|
||||
@@ -544,6 +562,13 @@ public sealed class WebSocketTurnFinalizationService(
|
||||
: DateTimeOffset.UtcNow.Add(WebSocketTurnState.DefaultLateAudioIgnoreWindow);
|
||||
|
||||
var emitSkillActions = !string.Equals(plan.IntentName, "word_of_the_day", StringComparison.OrdinalIgnoreCase) &&
|
||||
!string.Equals(plan.IntentName, "radio", StringComparison.OrdinalIgnoreCase) &&
|
||||
!string.Equals(plan.IntentName, "radio_genre", StringComparison.OrdinalIgnoreCase) &&
|
||||
!string.Equals(plan.IntentName, "clock_menu", StringComparison.OrdinalIgnoreCase) &&
|
||||
!string.Equals(plan.IntentName, "timer_menu", StringComparison.OrdinalIgnoreCase) &&
|
||||
!string.Equals(plan.IntentName, "alarm_menu", StringComparison.OrdinalIgnoreCase) &&
|
||||
!string.Equals(plan.IntentName, "timer_value", StringComparison.OrdinalIgnoreCase) &&
|
||||
!string.Equals(plan.IntentName, "alarm_value", StringComparison.OrdinalIgnoreCase) &&
|
||||
(messageType != "CLIENT_NLU" ||
|
||||
string.Equals(plan.IntentName, "word_of_the_day_guess", StringComparison.OrdinalIgnoreCase));
|
||||
var replies = ResponsePlanToSocketMessagesMapper.Map(plan, finalizedTurn, session, emitSkillActions).Select(map => new WebSocketReply
|
||||
@@ -708,6 +733,18 @@ public sealed class WebSocketTurnFinalizationService(
|
||||
string.Equals(rule, "$YESNO", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "create/is_it_a_keeper", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "settings/download_now_later", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "surprises-date/offer_date_fact", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "surprises-ota/want_to_download_now", StringComparison.OrdinalIgnoreCase));
|
||||
}
|
||||
|
||||
private static string? ReadPrimaryYesNoRule(TurnContext turn)
|
||||
{
|
||||
return ReadRules(turn, "listenRules")
|
||||
.Concat(ReadRules(turn, "clientRules"))
|
||||
.FirstOrDefault(static rule =>
|
||||
string.Equals(rule, "create/is_it_a_keeper", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "settings/download_now_later", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "surprises-date/offer_date_fact", StringComparison.OrdinalIgnoreCase) ||
|
||||
string.Equals(rule, "surprises-ota/want_to_download_now", StringComparison.OrdinalIgnoreCase));
|
||||
}
|
||||
|
||||
|
||||
@@ -66,6 +66,11 @@ public sealed class InMemoryJiboExperienceContentRepository : IJiboExperienceCon
|
||||
"I heard your news request. That path is still a future cloud integration.",
|
||||
"News is recognized, but I do not have the full news service behind it yet."
|
||||
],
|
||||
NewsBriefings =
|
||||
[
|
||||
"Here are your headlines. Space missions are preparing for new launches, climate and weather systems are staying active across the country, and AI tools keep pushing into everyday products.",
|
||||
"Here is a quick news brief. Technology companies are still racing on AI, global leaders are trading policy updates, and science teams are sharing new research findings."
|
||||
],
|
||||
GenericFallbackReplies =
|
||||
[
|
||||
"Okay. You said, {transcript}.",
|
||||
|
||||
@@ -115,6 +115,26 @@ public sealed class JiboInteractionServiceTests
|
||||
Assert.Equal("No.", decision.ReplyText);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task BuildDecisionAsync_SurprisesDateOfferPrompt_MapsShortAffirmationToYesIntent()
|
||||
{
|
||||
var service = CreateService();
|
||||
|
||||
var decision = await service.BuildDecisionAsync(new TurnContext
|
||||
{
|
||||
RawTranscript = "Yes!",
|
||||
NormalizedTranscript = "Yes!",
|
||||
Attributes = new Dictionary<string, object?>
|
||||
{
|
||||
["listenRules"] = new[] { "surprises-date/offer_date_fact", "globals/global_commands_launch" },
|
||||
["listenAsrHints"] = new[] { "$YESNO" }
|
||||
}
|
||||
});
|
||||
|
||||
Assert.Equal("yes", decision.IntentName);
|
||||
Assert.Equal("Yes.", decision.ReplyText);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task BuildDecisionAsync_SkillPhraseVariant_MapsToKnownIntent()
|
||||
{
|
||||
@@ -129,6 +149,128 @@ public sealed class JiboInteractionServiceTests
|
||||
Assert.Equal("joke", decision.IntentName);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task BuildDecisionAsync_OpenTheRadio_MapsToRadioLaunchIntent()
|
||||
{
|
||||
var service = CreateService();
|
||||
|
||||
var decision = await service.BuildDecisionAsync(new TurnContext
|
||||
{
|
||||
RawTranscript = "open the radio",
|
||||
NormalizedTranscript = "open the radio"
|
||||
});
|
||||
|
||||
Assert.Equal("radio", decision.IntentName);
|
||||
Assert.Equal("@be/radio", decision.SkillName);
|
||||
Assert.Equal("@be/radio", decision.SkillPayload!["skillId"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task BuildDecisionAsync_PlayCountryMusic_MapsToRadioGenreLaunchIntent()
|
||||
{
|
||||
var service = CreateService();
|
||||
|
||||
var decision = await service.BuildDecisionAsync(new TurnContext
|
||||
{
|
||||
RawTranscript = "play country music",
|
||||
NormalizedTranscript = "play country music"
|
||||
});
|
||||
|
||||
Assert.Equal("radio_genre", decision.IntentName);
|
||||
Assert.Equal("@be/radio", decision.SkillName);
|
||||
Assert.Equal("Country", decision.SkillPayload!["station"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task BuildDecisionAsync_OpenTimer_MapsToLocalClockTimerMenu()
|
||||
{
|
||||
var service = CreateService();
|
||||
|
||||
var decision = await service.BuildDecisionAsync(new TurnContext
|
||||
{
|
||||
RawTranscript = "open timer",
|
||||
NormalizedTranscript = "open timer"
|
||||
});
|
||||
|
||||
Assert.Equal("timer_menu", decision.IntentName);
|
||||
Assert.Equal("@be/clock", decision.SkillName);
|
||||
Assert.Equal("timer", decision.SkillPayload!["domain"]);
|
||||
Assert.Equal("menu", decision.SkillPayload["clockIntent"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task BuildDecisionAsync_SetTimerForFiveMinutes_MapsToTimerValue()
|
||||
{
|
||||
var service = CreateService();
|
||||
|
||||
var decision = await service.BuildDecisionAsync(new TurnContext
|
||||
{
|
||||
RawTranscript = "set a timer for five minutes",
|
||||
NormalizedTranscript = "set a timer for five minutes"
|
||||
});
|
||||
|
||||
Assert.Equal("timer_value", decision.IntentName);
|
||||
Assert.Equal("@be/clock", decision.SkillName);
|
||||
Assert.Equal("timer", decision.SkillPayload!["domain"]);
|
||||
Assert.Equal("timerValue", decision.SkillPayload["clockIntent"]);
|
||||
Assert.Equal("0", decision.SkillPayload["hours"]);
|
||||
Assert.Equal("5", decision.SkillPayload["minutes"]);
|
||||
Assert.Equal("null", decision.SkillPayload["seconds"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task BuildDecisionAsync_SetAlarmForSevenThirtyAm_MapsToAlarmValue()
|
||||
{
|
||||
var service = CreateService();
|
||||
|
||||
var decision = await service.BuildDecisionAsync(new TurnContext
|
||||
{
|
||||
RawTranscript = "set an alarm for 7:30 am",
|
||||
NormalizedTranscript = "set an alarm for 7:30 am"
|
||||
});
|
||||
|
||||
Assert.Equal("alarm_value", decision.IntentName);
|
||||
Assert.Equal("@be/clock", decision.SkillName);
|
||||
Assert.Equal("alarm", decision.SkillPayload!["domain"]);
|
||||
Assert.Equal("alarmValue", decision.SkillPayload["clockIntent"]);
|
||||
Assert.Equal("7:30", decision.SkillPayload["time"]);
|
||||
Assert.Equal("am", decision.SkillPayload["ampm"]);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task BuildDecisionAsync_TellMeTheNews_UsesNimbusCloudSkillPath()
|
||||
{
|
||||
var service = CreateService();
|
||||
|
||||
var decision = await service.BuildDecisionAsync(new TurnContext
|
||||
{
|
||||
RawTranscript = "tell me the news",
|
||||
NormalizedTranscript = "tell me the news"
|
||||
});
|
||||
|
||||
Assert.Equal("news", decision.IntentName);
|
||||
Assert.Equal("news", decision.SkillName);
|
||||
Assert.Equal("news", decision.SkillPayload!["skillId"]);
|
||||
Assert.Equal("news", decision.SkillPayload["cloudSkill"]);
|
||||
Assert.Equal("runtime-news", decision.SkillPayload["mim_id"]);
|
||||
Assert.DoesNotContain("future cloud integration", decision.ReplyText, StringComparison.OrdinalIgnoreCase);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task BuildDecisionAsync_CloudVersion_UsesSharedBuildInfo()
|
||||
{
|
||||
var service = CreateService();
|
||||
|
||||
var decision = await service.BuildDecisionAsync(new TurnContext
|
||||
{
|
||||
RawTranscript = "what's the cloud version",
|
||||
NormalizedTranscript = "what's the cloud version"
|
||||
});
|
||||
|
||||
Assert.Equal("cloud_version", decision.IntentName);
|
||||
Assert.Equal(OpenJiboCloudBuildInfo.SpokenVersion, decision.ReplyText);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task BuildDecisionAsync_WordOfDayGuess_UsesStructuredClientNluGuess()
|
||||
{
|
||||
|
||||
@@ -4,6 +4,7 @@ using Jibo.Cloud.Domain.Models;
|
||||
using Jibo.Cloud.Infrastructure.Content;
|
||||
using Jibo.Cloud.Infrastructure.Persistence;
|
||||
using Jibo.Cloud.Tests.Fixtures;
|
||||
using Jibo.Runtime.Abstractions;
|
||||
|
||||
namespace Jibo.Cloud.Tests.WebSockets;
|
||||
|
||||
@@ -342,6 +343,77 @@ public sealed class JiboWebSocketServiceTests
|
||||
Assert.Equal("clock/clock_menu", listenPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("rule").GetString());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ClientAsr_SetTimerForFiveMinutes_RedirectsIntoClockSkillWithTimerEntities()
|
||||
{
|
||||
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-clock-timer-token",
|
||||
Text = """{"type":"LISTEN","transID":"trans-clock-timer","data":{"rules":["globals/global_commands_launch"]}}"""
|
||||
});
|
||||
|
||||
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-clock-timer-token",
|
||||
Text = """{"type":"CLIENT_ASR","transID":"trans-clock-timer","data":{"text":"set a timer for five minutes"}}"""
|
||||
});
|
||||
|
||||
Assert.Equal(4, replies.Count);
|
||||
Assert.Equal("LISTEN", ReadReplyType(replies[0]));
|
||||
Assert.Equal("EOS", ReadReplyType(replies[1]));
|
||||
Assert.Equal("SKILL_REDIRECT", ReadReplyType(replies[2]));
|
||||
Assert.Equal("SKILL_ACTION", ReadReplyType(replies[3]));
|
||||
|
||||
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
|
||||
Assert.Equal("timerValue", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||
Assert.Equal("@be/clock", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("skill").GetString());
|
||||
Assert.Equal("timer", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("domain").GetString());
|
||||
Assert.Equal("0", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("hours").GetString());
|
||||
Assert.Equal("5", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("minutes").GetString());
|
||||
Assert.Equal("null", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("seconds").GetString());
|
||||
|
||||
using var redirectPayload = JsonDocument.Parse(replies[2].Text!);
|
||||
Assert.Equal("@be/clock", redirectPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("skillID").GetString());
|
||||
Assert.Equal("timerValue", redirectPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ClientAsr_SetAlarmForSevenThirtyAm_RedirectsIntoClockSkillWithAlarmEntities()
|
||||
{
|
||||
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-clock-alarm-token",
|
||||
Text = """{"type":"LISTEN","transID":"trans-clock-alarm","data":{"rules":["globals/global_commands_launch"]}}"""
|
||||
});
|
||||
|
||||
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-clock-alarm-token",
|
||||
Text = """{"type":"CLIENT_ASR","transID":"trans-clock-alarm","data":{"text":"set an alarm for 7:30 am"}}"""
|
||||
});
|
||||
|
||||
Assert.Equal(4, replies.Count);
|
||||
|
||||
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
|
||||
Assert.Equal("alarmValue", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||
Assert.Equal("@be/clock", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("skill").GetString());
|
||||
Assert.Equal("alarm", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("domain").GetString());
|
||||
Assert.Equal("7:30", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("time").GetString());
|
||||
Assert.Equal("am", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("ampm").GetString());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ClientAsr_YesNoCreateFlow_PreservesCreateRuleAndDomain()
|
||||
{
|
||||
@@ -403,6 +475,265 @@ public sealed class JiboWebSocketServiceTests
|
||||
Assert.Equal("surprises-ota/want_to_download_now", listenPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("rule").GetString());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task BufferedAudio_YesNoPromptWithSttFailure_AutoFinalizesAsLocalNoInput()
|
||||
{
|
||||
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-yesno-noinput-token",
|
||||
Text = """{"type":"LISTEN","transID":"trans-yesno-noinput","data":{"rules":["surprises-ota/want_to_download_now","globals/gui_nav","globals/global_commands_launch"],"asr":{"hints":["$YESNO"]}}}"""
|
||||
});
|
||||
|
||||
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-yesno-noinput-token",
|
||||
Text = """{"type":"CONTEXT","transID":"trans-yesno-noinput","data":{"topic":"conversation"}}"""
|
||||
});
|
||||
|
||||
for (var index = 0; index < 4; index += 1)
|
||||
{
|
||||
var interimReplies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-yesno-noinput-token",
|
||||
Binary = new byte[3000]
|
||||
});
|
||||
|
||||
Assert.Single(interimReplies);
|
||||
Assert.Equal("OPENJIBO_AUDIO_RECEIVED", ReadReplyType(interimReplies[0]));
|
||||
}
|
||||
|
||||
var session = _store.FindSessionByToken("hub-yesno-noinput-token");
|
||||
Assert.NotNull(session);
|
||||
session.TurnState.FirstAudioReceivedUtc = DateTimeOffset.UtcNow - TimeSpan.FromSeconds(2);
|
||||
session.TurnState.LastSttError = "whisper.cpp returned no transcript";
|
||||
|
||||
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-yesno-noinput-token",
|
||||
Binary = new byte[3000]
|
||||
});
|
||||
|
||||
Assert.Equal(2, replies.Count);
|
||||
Assert.Equal("LISTEN", ReadReplyType(replies[0]));
|
||||
Assert.Equal("EOS", ReadReplyType(replies[1]));
|
||||
|
||||
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
|
||||
Assert.Equal(string.Empty, listenPayload.RootElement.GetProperty("data").GetProperty("asr").GetProperty("text").GetString());
|
||||
Assert.Equal(string.Empty, listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||
var rules = listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("rules");
|
||||
Assert.Single(rules.EnumerateArray());
|
||||
Assert.Equal("surprises-ota/want_to_download_now", rules[0].GetString());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ClientAsr_SurprisesDateOfferPrompt_MapsYesWithoutGlobalRuleLeak()
|
||||
{
|
||||
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-share-yesno-token",
|
||||
Text = """{"type":"LISTEN","transID":"trans-share-yes","data":{"rules":["surprises-date/offer_date_fact","globals/gui_nav","globals/mim_repeat","globals/global_commands_launch"],"asr":{"hints":["$YESNO"]}}}"""
|
||||
});
|
||||
|
||||
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-share-yesno-token",
|
||||
Text = """{"type":"CLIENT_ASR","transID":"trans-share-yes","data":{"text":"Yes!"}}"""
|
||||
});
|
||||
|
||||
Assert.Equal(3, replies.Count);
|
||||
|
||||
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
|
||||
Assert.Equal("yes", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||
var rules = listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("rules");
|
||||
Assert.Single(rules.EnumerateArray());
|
||||
Assert.Equal("surprises-date/offer_date_fact", rules[0].GetString());
|
||||
Assert.Equal("surprises-date/offer_date_fact", listenPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("rule").GetString());
|
||||
Assert.Equal(0, listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").EnumerateObject().Count());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ResponsePlanMapper_EscapesSpeechWithoutEncodingApostrophes()
|
||||
{
|
||||
var plan = new ResponsePlan
|
||||
{
|
||||
IntentName = "chat",
|
||||
Actions =
|
||||
{
|
||||
new SpeakAction
|
||||
{
|
||||
Sequence = 0,
|
||||
Text = "I'm glad you're here.",
|
||||
Voice = "griffin"
|
||||
},
|
||||
new InvokeNativeSkillAction
|
||||
{
|
||||
Sequence = 1,
|
||||
SkillName = "chitchat-skill",
|
||||
Payload = new Dictionary<string, object?>()
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
var turn = new TurnContext
|
||||
{
|
||||
Attributes = new Dictionary<string, object?>
|
||||
{
|
||||
["transID"] = "trans-apostrophe"
|
||||
}
|
||||
};
|
||||
|
||||
var replies = ResponsePlanToSocketMessagesMapper.Map(plan, turn, new CloudSession(), emitSkillActions: true);
|
||||
using var payload = JsonDocument.Parse(replies[2].Text);
|
||||
var esml = payload.RootElement
|
||||
.GetProperty("data")
|
||||
.GetProperty("action")
|
||||
.GetProperty("config")
|
||||
.GetProperty("jcp")
|
||||
.GetProperty("config")
|
||||
.GetProperty("play")
|
||||
.GetProperty("esml")
|
||||
.GetString();
|
||||
|
||||
Assert.Contains("I'm glad you're here.", esml, StringComparison.Ordinal);
|
||||
Assert.DoesNotContain("'", esml, StringComparison.Ordinal);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ClientAsr_TellMeTheNews_EmitsNimbusCloudSkillMatchAndNewsSkillAction()
|
||||
{
|
||||
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-news-token",
|
||||
Text = """{"type":"LISTEN","transID":"trans-news","data":{"hotphrase":true,"rules":["launch","globals/global_commands_launch"]}}"""
|
||||
});
|
||||
|
||||
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-news-token",
|
||||
Text = """{"type":"CLIENT_ASR","transID":"trans-news","data":{"text":"tell me the news"}}"""
|
||||
});
|
||||
|
||||
Assert.Equal(3, replies.Count);
|
||||
Assert.Equal("LISTEN", ReadReplyType(replies[0]));
|
||||
Assert.Equal("EOS", ReadReplyType(replies[1]));
|
||||
Assert.Equal("SKILL_ACTION", ReadReplyType(replies[2]));
|
||||
|
||||
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
|
||||
Assert.Equal("news", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||
Assert.Equal("news", listenPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("cloudSkill").GetString());
|
||||
|
||||
using var skillPayload = JsonDocument.Parse(replies[2].Text!);
|
||||
Assert.Equal("news", skillPayload.RootElement.GetProperty("data").GetProperty("skill").GetProperty("id").GetString());
|
||||
var meta = skillPayload.RootElement
|
||||
.GetProperty("data")
|
||||
.GetProperty("action")
|
||||
.GetProperty("config")
|
||||
.GetProperty("jcp")
|
||||
.GetProperty("config")
|
||||
.GetProperty("play")
|
||||
.GetProperty("meta");
|
||||
Assert.Equal("runtime-news", meta.GetProperty("mim_id").GetString());
|
||||
Assert.Equal("announcement", meta.GetProperty("mim_type").GetString());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ClientAsr_OpenTheRadio_EmitsRadioRedirectAndSilentCompletion()
|
||||
{
|
||||
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-radio-open-token",
|
||||
Text = """{"type":"LISTEN","transID":"trans-radio-open","data":{"hotphrase":true,"rules":["launch","globals/global_commands_launch"]}}"""
|
||||
});
|
||||
|
||||
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-radio-open-token",
|
||||
Text = """{"type":"CLIENT_ASR","transID":"trans-radio-open","data":{"text":"open the radio"}}"""
|
||||
});
|
||||
|
||||
Assert.Equal(4, replies.Count);
|
||||
Assert.Equal("LISTEN", ReadReplyType(replies[0]));
|
||||
Assert.Equal("EOS", ReadReplyType(replies[1]));
|
||||
Assert.Equal("SKILL_REDIRECT", ReadReplyType(replies[2]));
|
||||
Assert.Equal("SKILL_ACTION", ReadReplyType(replies[3]));
|
||||
|
||||
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
|
||||
Assert.Equal("menu", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||
Assert.Equal("@be/radio", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("skill").GetString());
|
||||
Assert.Equal(0, listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("rules").GetArrayLength());
|
||||
Assert.Equal(0, listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").EnumerateObject().Count());
|
||||
|
||||
using var redirectPayload = JsonDocument.Parse(replies[2].Text!);
|
||||
Assert.Equal("@be/radio", redirectPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("skillID").GetString());
|
||||
Assert.True(redirectPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("launch").GetBoolean());
|
||||
Assert.Equal("menu", redirectPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||
|
||||
using var completionPayload = JsonDocument.Parse(replies[3].Text!);
|
||||
Assert.Equal("@be/radio", completionPayload.RootElement.GetProperty("data").GetProperty("skill").GetProperty("id").GetString());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ClientAsr_PlayCountryMusic_EmitsRadioRedirectWithCountryStation()
|
||||
{
|
||||
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-radio-country-token",
|
||||
Text = """{"type":"LISTEN","transID":"trans-radio-country","data":{"hotphrase":true,"rules":["launch","globals/global_commands_launch"]}}"""
|
||||
});
|
||||
|
||||
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||
{
|
||||
HostName = "neo-hub.jibo.com",
|
||||
Path = "/listen",
|
||||
Kind = "neo-hub-listen",
|
||||
Token = "hub-radio-country-token",
|
||||
Text = """{"type":"CLIENT_ASR","transID":"trans-radio-country","data":{"text":"play country music"}}"""
|
||||
});
|
||||
|
||||
Assert.Equal(4, replies.Count);
|
||||
|
||||
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
|
||||
Assert.Equal("menu", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||
Assert.Equal("Country", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("station").GetString());
|
||||
|
||||
using var redirectPayload = JsonDocument.Parse(replies[2].Text!);
|
||||
Assert.Equal("Country", redirectPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("station").GetString());
|
||||
Assert.Equal("play country music", redirectPayload.RootElement.GetProperty("data").GetProperty("asr").GetProperty("text").GetString());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task ClientNlu_WordOfDayGuess_UsesGuessEntityAsAsrTextAndCompletesTurn()
|
||||
{
|
||||
|
||||
BIN
artifact-output/jibo-architecture.png
Normal file
BIN
artifact-output/jibo-architecture.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 176 KiB |
Reference in New Issue
Block a user