a couple more features for version 18

This commit is contained in:
Jacob Dubin
2026-04-26 20:19:16 -05:00
parent df78170aa2
commit acbba413db
9 changed files with 586 additions and 24 deletions

View File

@@ -91,6 +91,8 @@ The following behavior is present in source and covered by focused tests:
- apostrophes are no longer escaped to `&apos;` in spoken ESML, while `&`, `<`, `>`, and `"` remain escaped
- radio voice launch supports `open the radio` and genre launch such as `play country music`, using local `@be/radio` `menu` payloads, `SKILL_REDIRECT`, and silent completion
- news has a first Nimbus-shaped cloud path using `match.cloudSkill = news` and a `news` `SKILL_ACTION` with synthetic briefing content
- stop commands such as `stop that` and `never mind` emit stock `global_commands` `stop` NLU plus a local `@be/idle` redirect, without generic chat speech
- volume commands emit stock `global_commands` volume intents: `volumeUp`, `volumeDown`, and `volumeToValue` with `volumeLevel`; `show volume controls` redirects to `@be/settings` `volumeQuery`
- stock-shaped clock handoffs cover time, date, day, clock open, timer/alarm menu, timer/alarm value, timer/alarm clarification, and timer/alarm delete
- alarm parsing covers forms such as `7:30 am`, `830`, `8 30`, `7, 44`, `10-25`, `10:25 pm`, and `10 25 p m`
- ambiguous alarm times can prefer the next local occurrence when the robot context includes `runtime.location.iso`
@@ -147,6 +149,7 @@ Before calling `1.0.18` complete, prove or explicitly defer these:
- Regression test photo/gallery flows again after the `jibo test 24` fixes: open gallery, answer the stock `shared/yes_no` prompt with a transcript-bearing `yes`, hand into create, take one photo, keep it, and avoid blue-ring or `I heard you` stale turns.
- Live-test radio launch: `open the radio` passed in `jibo test 22`; re-run `play country music` if that exact phrase was not captured.
- Treat basic news as live-proven by `jibo test 23`; defer provider-backed or category-expanded news unless it is chosen as an optional feature slice.
- Regression test the added stop and volume slices: `stop that`, `never mind`, `turn it up`, `turn it down`, `set volume to six`, and `show volume controls`.
- Recheck constrained yes/no prompts for update/backup/share/gallery/alarm replacement without leaking global rules.
- Recheck that stock OS no longer logs OpenJibo-only websocket events such as synthetic pending/context/ack packets from the current build.
- Recheck backup/update behavior with explicit attention to robot-local `jibo.scheduler.backupStatus`, CPU/load, and whether the deployed cloud is involved at all.
@@ -164,13 +167,13 @@ These are not blockers for calling `1.0.18` complete unless the live test shows
- news content is synthetic; `jibo test 23` proved the path but not live provider-backed headlines
- gallery `shared/yes_no` still needs a successful transcript-bearing live `yes` pass
- weather, calendar, commute, personal report, identity, memory, and proactivity are still mostly discovery or placeholder content paths
- volume, stop, robot age, and command-versus-question personality routing are not implemented yet
- stop and volume are implemented but still need live stock-OS proof; robot age and command-versus-question personality routing are not implemented yet
## `1.0.19` Direction
After `1.0.18` is tested and tagged, `1.0.19` should move back into feature work:
- one lightweight device-control feature, most likely stop or volume
- harden whichever stop/volume behavior is not fully proven by the `1.0.18` live pass, or pick the next lightweight device/persona slice
- end-to-end update/backup/restore proof
- STT reliability improvements, including noise screening and a managed STT comparison
- provider-backed first content path, likely news or weather

View File

@@ -147,17 +147,26 @@ Current release theme:
### 5. Optional Small Feature Before `1.0.18` Freeze
- Status: `ready`
- Status: `implemented`
- Tags: `protocol`
- Why now: the user wants one or two features before `1.0.18` is called complete, but the release should not take on a risky subsystem.
- Preferred candidates:
- Selected slices:
- Stop command
- Volume up / volume down voice control
- How old are you / robot age persona
- Guidance:
- pick only one if the live regression pass finds bugs
- pick at most two if the current bug-fix paths stay stable
- keep the implementation source-backed and easy to revert or defer
- Volume up / volume down / set-to-value voice control
- Current code:
- `stop`, `stop that`, and `never mind` map to stock `global_commands` `stop` NLU plus local `@be/idle` redirect/completion
- `turn it up` and `turn it down` emit stock `global_commands` `volumeUp` / `volumeDown` with `volumeLevel = null` and no cloud speech
- `set volume to six` emits stock `global_commands` `volumeToValue` with `volumeLevel = 6` and no cloud speech
- `show volume controls` redirects into `@be/settings` with `volumeQuery`
- Evidence:
- Pegasus `globals/global_commands_launch.rule` defines `stop`, `volumeUp`, `volumeDown`, and `volumeToValue`
- stock Jibo `VolumePlugin` subscribes to global volume events and uses the same intent/entity names
- stock `@be/settings` exposes `volumeQuery` and opens the volume panel
- Exit criteria:
- live stop settles the robot without a generic chat reply
- live volume up/down audibly changes volume or logs a local volume event
- live volume-to-value changes the setting to the requested value or logs the expected stock local handling
- live volume controls opens the settings volume panel
## Implemented In Current Source
@@ -248,6 +257,19 @@ Current release theme:
- Follow-up:
- keep this in regression coverage because it shares turn-state machinery with gallery and alarm flows
### Stop And Volume First Pass
- Status: `implemented`
- Tags: `protocol`
- Result:
- global stop commands emit stock `global_commands` `stop` and redirect to `@be/idle`
- relative volume commands emit stock `global_commands` `volumeUp` / `volumeDown`
- absolute volume commands emit `volumeToValue` with a `volumeLevel` entity
- volume controls launch redirects to `@be/settings` `volumeQuery`
- websocket responses avoid generic chat speech for these local/global command paths
- Follow-up:
- live validation remains in the immediate queue because volume depends on stock robot local global-command handling
### Unknown OpenJibo Event Noise
- Status: `implemented`
@@ -272,7 +294,7 @@ Current release theme:
### 6. Stop Command
- Status: `ready`
- Status: `polish`
- Tags: `protocol`
- User goals:
- `stop`
@@ -280,15 +302,15 @@ Current release theme:
- `never mind`
- Evidence:
- `@be/idle` exists and is already used as a cleanup redirect target
- current `1.0.18` source emits stock `global_commands` `stop` plus local `@be/idle` redirect
- Questions:
- whether stock source has a dedicated stop/cancel intent beyond idle redirect
- whether stop should interrupt active local skills or only cloud speech paths in the first pass
- whether live stock OS treats the combined global stop plus idle redirect as cleanly as expected during active local skills
- Exit criteria:
- a spoken stop command settles the robot locally without a generic chat reply
### 7. Volume Up / Volume Down Voice Control
- Status: `ready`
- Status: `polish`
- Tags: `protocol`
- User goals:
- `turn it up`
@@ -296,10 +318,11 @@ Current release theme:
- `increase the volume`
- `decrease the volume`
- Evidence:
- stock Jibo exposes volume control through robot UX, so there should be a local control or settings path to mirror
- Pegasus global commands define `volumeUp`, `volumeDown`, and `volumeToValue`
- stock Jibo `VolumePlugin` listens for those global intents and `volumeLevel`
- current `1.0.18` source emits those stock NLU shapes and opens `@be/settings` `volumeQuery`
- Questions:
- exact local payload shape for relative volume changes
- whether first pass should support absolute values such as `set volume to 5`
- whether live stock OS applies the global volume event from the hosted cloud response without any additional local event payload
- Exit criteria:
- relative voice volume commands adjust volume without generic cloud speech
@@ -505,13 +528,13 @@ Before closing `1.0.18`:
2. Basic news regression, with provider-backed expansion deferred
3. Backup / OTA / share yes-no regression
4. Alarm and photo/gallery regression
5. Optional small feature only if the regression pass stays calm
5. Stop and volume first-pass validation
Use [regression-test-plan.md](regression-test-plan.md) as the detailed checklist for this sequence.
For `1.0.19`:
1. Stop command or volume control
1. Harden stop or volume if the `1.0.18` live pass exposes stock-OS quirks; otherwise pick robot age/persona or another lightweight slice
2. Update, backup, and restore proof
3. STT upgrade and noise screening
4. Hosted capture/storage plan

View File

@@ -168,6 +168,34 @@ Expected:
- no `ffmpeg` failure should become the dominant failure mode for non-Opus buffered audio
- short replies such as `yes`, `no`, `cancel`, and short alarm times should either map correctly or be classified as STT misses with evidence
### Stop And Volume
Goal: prove the added lightweight device-control slice before closing `1.0.18`.
Test these phrases:
- `stop`
- `stop that`
- `never mind`
- `turn it up`
- `turn it down`
- `set volume to six`
- `show volume controls`
Expected:
- stop commands settle the robot locally without generic chat speech
- `turn it up` and `turn it down` adjust volume or at least produce the stock local volume event/log
- `set volume to six` sets or attempts to set the local volume level to `6`
- `show volume controls` opens the settings volume panel
Capture check:
- stop emits `nlu.intent = stop`, `nlu.domain = global_commands`, then redirects to `@be/idle`
- relative volume emits `nlu.intent = volumeUp` or `volumeDown`, `nlu.domain = global_commands`, and `entities.volumeLevel = null`, with no `SKILL_ACTION` cloud speech
- absolute volume emits `nlu.intent = volumeToValue` and `entities.volumeLevel` matching the requested value, with no `SKILL_ACTION` cloud speech
- volume controls redirects to `@be/settings` with `nlu.intent = volumeQuery`
## Optional Feature Slice Checks
When a new feature is added before a release closes:
@@ -178,8 +206,6 @@ When a new feature is added before a release closes:
For the current candidate list, add cases here when implemented:
- stop command: `stop`, `stop that`, `never mind`
- volume: `turn it up`, `turn it down`, `increase the volume`, `decrease the volume`
- robot age/persona: `how old are you`
## After The Run

View File

@@ -74,6 +74,11 @@ public sealed class DemoConversationBroker(JiboInteractionService interactionSer
"word_of_the_day_guess" => false,
"radio" => false,
"radio_genre" => false,
"stop" => false,
"volume_up" => false,
"volume_down" => false,
"volume_to_value" => false,
"volume_query" => false,
"time" => false,
"date" => false,
"day" => false,

View File

@@ -51,6 +51,11 @@ public sealed class JiboInteractionService(
"cloud_version" => new JiboInteractionDecision("cloud_version", OpenJiboCloudBuildInfo.SpokenVersion),
"radio" => BuildRadioLaunchDecision(),
"radio_genre" => BuildRadioGenreLaunchDecision(lowered),
"stop" => BuildStopDecision(),
"volume_up" => BuildVolumeControlDecision("volume_up", "volumeUp", "null"),
"volume_down" => BuildVolumeControlDecision("volume_down", "volumeDown", "null"),
"volume_to_value" => BuildVolumeControlDecision("volume_to_value", "volumeToValue", ResolveVolumeLevel(lowered, clientEntities) ?? "7"),
"volume_query" => BuildSettingsVolumeDecision(),
"clock_open" => BuildClockLaunchDecision("clock_open", "clock", "askForTime", "Opening the clock."),
"clock_menu" => BuildClockLaunchDecision("clock_menu", "clock", "menu", "Opening the clock menu."),
"timer_menu" => BuildClockLaunchDecision("timer", "Opening the timer."),
@@ -309,6 +314,27 @@ public sealed class JiboInteractionService(
return "radio_genre";
}
if (TryResolveVolumeLevel(loweredTranscript) is not null ||
clientEntities.ContainsKey("volumeLevel"))
{
return "volume_to_value";
}
if (IsVolumeQueryRequest(loweredTranscript))
{
return "volume_query";
}
if (IsVolumeUpRequest(loweredTranscript))
{
return "volume_up";
}
if (IsVolumeDownRequest(loweredTranscript))
{
return "volume_down";
}
if (MatchesAny(loweredTranscript, "open the clock", "open clock", "show the clock", "show clock"))
{
return "clock_open";
@@ -346,6 +372,11 @@ public sealed class JiboInteractionService(
return "timer_delete";
}
if (IsGlobalStopRequest(loweredTranscript, clientIntent, clientEntities))
{
return "stop";
}
if (TryParseAlarmValue(loweredTranscript, isAlarmValueTurn, referenceLocalTime) is not null)
{
return "alarm_value";
@@ -535,6 +566,47 @@ public sealed class JiboInteractionService(
});
}
private static JiboInteractionDecision BuildStopDecision()
{
return new JiboInteractionDecision(
"stop",
"Stopping.",
"@be/idle",
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
{
["skillId"] = "@be/idle",
["globalIntent"] = "stop",
["nluDomain"] = "global_commands"
});
}
private static JiboInteractionDecision BuildVolumeControlDecision(string intentName, string globalIntent, string volumeLevel)
{
return new JiboInteractionDecision(
intentName,
"Adjusting volume.",
"global_commands",
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
{
["globalIntent"] = globalIntent,
["nluDomain"] = "global_commands",
["volumeLevel"] = volumeLevel
});
}
private static JiboInteractionDecision BuildSettingsVolumeDecision()
{
return new JiboInteractionDecision(
"volume_query",
"Opening volume controls.",
"@be/settings",
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
{
["skillId"] = "@be/settings",
["localIntent"] = "volumeQuery"
});
}
private static JiboInteractionDecision BuildClockLaunchDecision(string intentName, string domain, string clockIntent, string replyText)
{
return new JiboInteractionDecision(
@@ -1120,6 +1192,144 @@ public sealed class JiboInteractionService(
loweredTranscript is "cancel" or "stop" or "never mind" or "nevermind";
}
private static bool IsGlobalStopRequest(
string loweredTranscript,
string? clientIntent,
IReadOnlyDictionary<string, string> clientEntities)
{
if (string.Equals(clientIntent, "stop", StringComparison.OrdinalIgnoreCase) &&
IsGlobalCommandsDomain(clientEntities))
{
return true;
}
return loweredTranscript is "stop" or "stop it" or "stop that" or "stop talking" or "be quiet" or "never mind" or "nevermind" or "forget it" ||
MatchesAny(loweredTranscript, "that s enough", "that's enough", "that will do", "that ll do", "that'll do", "cut it out", "cut that out");
}
private static bool IsVolumeQueryRequest(string loweredTranscript)
{
return MatchesAny(
loweredTranscript,
"volume controls",
"volume control",
"volume menu",
"volume level",
"show volume",
"show the volume",
"open volume",
"open the volume",
"what is your volume",
"what's your volume",
"how is your volume",
"how s your volume");
}
private static bool IsVolumeUpRequest(string loweredTranscript)
{
return MatchesAny(
loweredTranscript,
"turn it up",
"turn this up",
"turn that up",
"turn up the volume",
"turn the volume up",
"turn volume up",
"turn your volume up",
"increase the volume",
"increase your volume",
"raise the volume",
"raise your volume",
"make it louder",
"make that louder",
"speak louder",
"talk louder",
"be louder",
"louder");
}
private static bool IsVolumeDownRequest(string loweredTranscript)
{
return MatchesAny(
loweredTranscript,
"turn it down",
"turn this down",
"turn that down",
"turn down the volume",
"turn the volume down",
"turn volume down",
"turn your volume down",
"decrease the volume",
"decrease your volume",
"lower the volume",
"lower your volume",
"make it quieter",
"make that quieter",
"make it softer",
"speak quieter",
"talk quieter",
"be quieter",
"quieter",
"softer");
}
private static string? ResolveVolumeLevel(string loweredTranscript, IReadOnlyDictionary<string, string> clientEntities)
{
if (clientEntities.TryGetValue("volumeLevel", out var entityValue) &&
TryNormalizeVolumeLevel(entityValue) is { } structuredLevel)
{
return structuredLevel;
}
return TryResolveVolumeLevel(loweredTranscript);
}
private static string? TryResolveVolumeLevel(string loweredTranscript)
{
if (!loweredTranscript.Contains("volume", StringComparison.Ordinal) &&
!loweredTranscript.Contains("loudness", StringComparison.Ordinal))
{
return null;
}
if (MatchesAny(loweredTranscript, "max volume", "maximum volume", "volume max", "volume maximum"))
{
return "10";
}
if (MatchesAny(loweredTranscript, "min volume", "minimum volume", "volume min", "volume minimum"))
{
return "1";
}
var match = VolumeLevelPattern.Match(loweredTranscript);
if (!match.Success)
{
return null;
}
return TryNormalizeVolumeLevel(match.Groups["value"].Value);
}
private static string? TryNormalizeVolumeLevel(string token)
{
if (string.Equals(token, "null", StringComparison.OrdinalIgnoreCase))
{
return "null";
}
var parsed = ParseNumberToken(token);
return parsed is >= 1 and <= 10
? parsed.Value.ToString()
: null;
}
private static bool IsGlobalCommandsDomain(IReadOnlyDictionary<string, string> clientEntities)
{
return clientEntities.TryGetValue("domain", out var domain) &&
string.Equals(domain, "global_commands", StringComparison.OrdinalIgnoreCase);
}
private static bool IsClockTimerValueTurn(
IReadOnlyList<string> clientRules,
IReadOnlyList<string> listenRules)
@@ -1235,6 +1445,10 @@ public sealed class JiboInteractionService(
@"\b(?<compact>\d{3,4})\s*(?<ampm>a[\s\.]*m\.?|p[\s\.]*m\.?)?\b",
RegexOptions.IgnoreCase | RegexOptions.CultureInvariant | RegexOptions.Compiled);
private static readonly Regex VolumeLevelPattern = new(
@"\b(?:volume|loudness)\s*(?:to|at|level|is)?\s*(?<value>10|\d|one|two|three|four|five|six|seven|eight|nine|ten)\b|\b(?:set|change|make|turn)\s+(?:the\s+|your\s+)?(?:volume|loudness)\s*(?:to|at)?\s*(?<value>10|\d|one|two|three|four|five|six|seven|eight|nine|ten)\b",
RegexOptions.IgnoreCase | RegexOptions.CultureInvariant | RegexOptions.Compiled);
private static readonly (string Phrase, string Station)[] RadioGenreAliases =
[
("country music", "Country"),

View File

@@ -25,6 +25,12 @@ public sealed class ResponsePlanToSocketMessagesMapper
var isWordOfDayGuess = string.Equals(plan.IntentName, "word_of_the_day_guess", StringComparison.OrdinalIgnoreCase);
var isRadioLaunch = string.Equals(plan.IntentName, "radio", StringComparison.OrdinalIgnoreCase) ||
string.Equals(plan.IntentName, "radio_genre", StringComparison.OrdinalIgnoreCase);
var isStopCommand = string.Equals(plan.IntentName, "stop", StringComparison.OrdinalIgnoreCase);
var isVolumeControl = string.Equals(plan.IntentName, "volume_up", StringComparison.OrdinalIgnoreCase) ||
string.Equals(plan.IntentName, "volume_down", StringComparison.OrdinalIgnoreCase) ||
string.Equals(plan.IntentName, "volume_to_value", StringComparison.OrdinalIgnoreCase);
var isSettingsLaunch = string.Equals(skill?.SkillName, "@be/settings", StringComparison.OrdinalIgnoreCase);
var isGlobalCommand = isStopCommand || isVolumeControl;
var isPhotoGalleryLaunch = string.Equals(plan.IntentName, "photo_gallery", StringComparison.OrdinalIgnoreCase);
var isPhotoCreateLaunch = string.Equals(plan.IntentName, "snapshot", StringComparison.OrdinalIgnoreCase) ||
string.Equals(plan.IntentName, "photobooth", StringComparison.OrdinalIgnoreCase);
@@ -39,12 +45,19 @@ public sealed class ResponsePlanToSocketMessagesMapper
var alarmAmPm = ReadSkillPayloadString(skill, "ampm");
var radioStation = ReadSkillPayloadString(skill, "station");
var cloudSkill = ReadSkillPayloadString(skill, "cloudSkill");
var globalIntent = ReadSkillPayloadString(skill, "globalIntent");
var nluDomain = ReadSkillPayloadString(skill, "nluDomain");
var volumeLevel = ReadSkillPayloadString(skill, "volumeLevel");
var nluGuess = ReadClientEntity(turn, "guess");
var wordOfDayGuess = ResolveWordOfDayGuess(turn, transcript, nluGuess);
var outboundIntent = isWordOfDayLaunch
var outboundIntent = isGlobalCommand && !string.IsNullOrWhiteSpace(globalIntent)
? globalIntent
: isWordOfDayLaunch
? "menu"
: isRadioLaunch
? "menu"
: isSettingsLaunch && !string.IsNullOrWhiteSpace(localIntent)
? localIntent
: (isPhotoGalleryLaunch || isPhotoCreateLaunch) && !string.IsNullOrWhiteSpace(localIntent)
? localIntent
: isClockSkillLaunch && !string.IsNullOrWhiteSpace(clockIntent)
@@ -58,8 +71,12 @@ public sealed class ResponsePlanToSocketMessagesMapper
? wordOfDayGuess
: isWordOfDayLaunch
? string.Empty
: isGlobalCommand
? transcript
: isRadioLaunch
? transcript
: isSettingsLaunch
? transcript
: isPhotoGalleryLaunch || isPhotoCreateLaunch
? transcript
: isClockSkillLaunch
@@ -73,8 +90,12 @@ public sealed class ResponsePlanToSocketMessagesMapper
: transcript;
var outboundRules = isWordOfDayLaunch
? ["word-of-the-day/menu"]
: isGlobalCommand
? BuildGlobalCommandRules(rules)
: isRadioLaunch
? Array.Empty<string>()
: isSettingsLaunch
? string.Equals(messageType, "CLIENT_NLU", StringComparison.OrdinalIgnoreCase) ? rules : Array.Empty<string>()
: isPhotoGalleryLaunch || isPhotoCreateLaunch
? string.Equals(messageType, "CLIENT_NLU", StringComparison.OrdinalIgnoreCase) ? rules : Array.Empty<string>()
: isClockSkillLaunch
@@ -88,6 +109,8 @@ public sealed class ResponsePlanToSocketMessagesMapper
isYesNoTurn && isYesNoIntent,
ShouldIncludeCreateDomain(yesNoRule),
isWordOfDayLaunch,
isGlobalCommand,
volumeLevel,
isRadioLaunch,
isWordOfDayGuess,
wordOfDayGuess,
@@ -118,10 +141,12 @@ public sealed class ResponsePlanToSocketMessagesMapper
entities,
isWordOfDayLaunch ? "@be/word-of-the-day" :
isRadioLaunch ? "@be/radio" :
isSettingsLaunch ? "@be/settings" :
isPhotoGalleryLaunch ? "@be/gallery" :
isPhotoCreateLaunch ? "@be/create" :
isClockSkillLaunch ? "@be/clock" :
null),
null,
isGlobalCommand ? nluDomain ?? "global_commands" : null),
match = new
{
intent = outboundIntent,
@@ -177,6 +202,39 @@ public sealed class ResponsePlanToSocketMessagesMapper
DelayMs: 125));
}
if (isStopCommand)
{
messages.Add(new SocketReplyPlan(
JsonSerializer.Serialize(BuildSkillRedirectPayload(
transId,
"@be/idle",
outboundIntent,
outboundAsrText,
outboundRules,
entities)),
DelayMs: 75));
messages.Add(new SocketReplyPlan(
JsonSerializer.Serialize(BuildCompletionOnlySkillPayload(transId, "@be/idle")),
DelayMs: 125));
}
if (isSettingsLaunch &&
!string.Equals(messageType, "CLIENT_NLU", StringComparison.OrdinalIgnoreCase))
{
messages.Add(new SocketReplyPlan(
JsonSerializer.Serialize(BuildSkillRedirectPayload(
transId,
"@be/settings",
outboundIntent,
outboundAsrText,
outboundRules,
entities)),
DelayMs: 75));
messages.Add(new SocketReplyPlan(
JsonSerializer.Serialize(BuildCompletionOnlySkillPayload(transId, "@be/settings")),
DelayMs: 125));
}
if (isClockSkillLaunch &&
!string.Equals(messageType, "CLIENT_NLU", StringComparison.OrdinalIgnoreCase))
{
@@ -355,6 +413,8 @@ public sealed class ResponsePlanToSocketMessagesMapper
bool yesNoTurn,
bool includeCreateDomain,
bool wordOfDayLaunch,
bool globalCommand,
string? volumeLevel,
bool radioLaunch,
bool wordOfDayGuess,
string? guess,
@@ -389,6 +449,17 @@ public sealed class ResponsePlanToSocketMessagesMapper
};
}
if (globalCommand)
{
var entities = new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase);
if (!string.IsNullOrWhiteSpace(volumeLevel))
{
entities["volumeLevel"] = volumeLevel;
}
return entities;
}
if (radioLaunch)
{
var entities = new Dictionary<string, object?>();
@@ -702,7 +773,8 @@ public sealed class ResponsePlanToSocketMessagesMapper
string outboundIntent,
IReadOnlyList<string> outboundRules,
object entities,
string? skillId)
string? skillId,
string? domain = null)
{
var payload = new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
{
@@ -717,9 +789,21 @@ public sealed class ResponsePlanToSocketMessagesMapper
payload["skill"] = skillId;
}
if (!string.IsNullOrWhiteSpace(domain))
{
payload["domain"] = domain;
}
return payload;
}
private static IReadOnlyList<string> BuildGlobalCommandRules(IReadOnlyList<string> rules)
{
return rules.Any(static rule => string.Equals(rule, "globals/global_commands_launch", StringComparison.OrdinalIgnoreCase))
? ["globals/global_commands_launch"]
: Array.Empty<string>();
}
private static object BuildGenericFallbackSkillPayload(string transId)
{
return new

View File

@@ -504,6 +504,11 @@ public sealed class WebSocketTurnFinalizationService(
var emitSkillActions = !string.Equals(plan.IntentName, "word_of_the_day", StringComparison.OrdinalIgnoreCase) &&
!string.Equals(plan.IntentName, "radio", StringComparison.OrdinalIgnoreCase) &&
!string.Equals(plan.IntentName, "radio_genre", StringComparison.OrdinalIgnoreCase) &&
!string.Equals(plan.IntentName, "stop", StringComparison.OrdinalIgnoreCase) &&
!string.Equals(plan.IntentName, "volume_up", StringComparison.OrdinalIgnoreCase) &&
!string.Equals(plan.IntentName, "volume_down", StringComparison.OrdinalIgnoreCase) &&
!string.Equals(plan.IntentName, "volume_to_value", StringComparison.OrdinalIgnoreCase) &&
!string.Equals(plan.IntentName, "volume_query", StringComparison.OrdinalIgnoreCase) &&
!string.Equals(plan.IntentName, "time", StringComparison.OrdinalIgnoreCase) &&
!string.Equals(plan.IntentName, "date", StringComparison.OrdinalIgnoreCase) &&
!string.Equals(plan.IntentName, "day", StringComparison.OrdinalIgnoreCase) &&

View File

@@ -258,6 +258,73 @@ public sealed class JiboInteractionServiceTests
Assert.Equal("Country", decision.SkillPayload!["station"]);
}
[Fact]
public async Task BuildDecisionAsync_StopThat_MapsToIdleStopCommand()
{
var service = CreateService();
var decision = await service.BuildDecisionAsync(new TurnContext
{
RawTranscript = "stop that",
NormalizedTranscript = "stop that"
});
Assert.Equal("stop", decision.IntentName);
Assert.Equal("@be/idle", decision.SkillName);
Assert.Equal("stop", decision.SkillPayload!["globalIntent"]);
Assert.Equal("global_commands", decision.SkillPayload["nluDomain"]);
}
[Fact]
public async Task BuildDecisionAsync_TurnItUp_MapsToGlobalVolumeUpCommand()
{
var service = CreateService();
var decision = await service.BuildDecisionAsync(new TurnContext
{
RawTranscript = "turn it up",
NormalizedTranscript = "turn it up"
});
Assert.Equal("volume_up", decision.IntentName);
Assert.Equal("global_commands", decision.SkillName);
Assert.Equal("volumeUp", decision.SkillPayload!["globalIntent"]);
Assert.Equal("null", decision.SkillPayload["volumeLevel"]);
}
[Fact]
public async Task BuildDecisionAsync_SetVolumeToSix_MapsToGlobalVolumeToValueCommand()
{
var service = CreateService();
var decision = await service.BuildDecisionAsync(new TurnContext
{
RawTranscript = "set volume to six",
NormalizedTranscript = "set volume to six"
});
Assert.Equal("volume_to_value", decision.IntentName);
Assert.Equal("global_commands", decision.SkillName);
Assert.Equal("volumeToValue", decision.SkillPayload!["globalIntent"]);
Assert.Equal("6", decision.SkillPayload["volumeLevel"]);
}
[Fact]
public async Task BuildDecisionAsync_ShowVolumeControls_MapsToSettingsVolumeQuery()
{
var service = CreateService();
var decision = await service.BuildDecisionAsync(new TurnContext
{
RawTranscript = "show volume controls",
NormalizedTranscript = "show volume controls"
});
Assert.Equal("volume_query", decision.IntentName);
Assert.Equal("@be/settings", decision.SkillName);
Assert.Equal("volumeQuery", decision.SkillPayload!["localIntent"]);
}
[Fact]
public async Task BuildDecisionAsync_OpenTimer_MapsToLocalClockTimerMenu()
{

View File

@@ -1583,6 +1583,141 @@ public sealed class JiboWebSocketServiceTests
Assert.Equal("play country music", redirectPayload.RootElement.GetProperty("data").GetProperty("asr").GetProperty("text").GetString());
}
[Fact]
public async Task ClientAsr_StopThat_EmitsGlobalStopAndIdleRedirect()
{
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
{
HostName = "neo-hub.jibo.com",
Path = "/listen",
Kind = "neo-hub-listen",
Token = "hub-stop-token",
Text = """{"type":"LISTEN","transID":"trans-stop","data":{"hotphrase":true,"rules":["launch","globals/global_commands_launch"]}}"""
});
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
{
HostName = "neo-hub.jibo.com",
Path = "/listen",
Kind = "neo-hub-listen",
Token = "hub-stop-token",
Text = """{"type":"CLIENT_ASR","transID":"trans-stop","data":{"text":"stop that"}}"""
});
Assert.Equal(4, replies.Count);
Assert.Equal("LISTEN", ReadReplyType(replies[0]));
Assert.Equal("EOS", ReadReplyType(replies[1]));
Assert.Equal("SKILL_REDIRECT", ReadReplyType(replies[2]));
Assert.Equal("SKILL_ACTION", ReadReplyType(replies[3]));
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
var nlu = listenPayload.RootElement.GetProperty("data").GetProperty("nlu");
Assert.Equal("stop", nlu.GetProperty("intent").GetString());
Assert.Equal("global_commands", nlu.GetProperty("domain").GetString());
Assert.Equal("globals/global_commands_launch", nlu.GetProperty("rules")[0].GetString());
using var redirectPayload = JsonDocument.Parse(replies[2].Text!);
Assert.Equal("@be/idle", redirectPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("skillID").GetString());
Assert.Equal("stop", redirectPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
}
[Fact]
public async Task ClientAsr_TurnItDown_EmitsGlobalVolumeDownWithoutCloudSpeech()
{
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
{
HostName = "neo-hub.jibo.com",
Path = "/listen",
Kind = "neo-hub-listen",
Token = "hub-volume-down-token",
Text = """{"type":"LISTEN","transID":"trans-volume-down","data":{"hotphrase":true,"rules":["launch","globals/global_commands_launch"]}}"""
});
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
{
HostName = "neo-hub.jibo.com",
Path = "/listen",
Kind = "neo-hub-listen",
Token = "hub-volume-down-token",
Text = """{"type":"CLIENT_ASR","transID":"trans-volume-down","data":{"text":"turn it down"}}"""
});
Assert.Equal(2, replies.Count);
Assert.Equal("LISTEN", ReadReplyType(replies[0]));
Assert.Equal("EOS", ReadReplyType(replies[1]));
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
var nlu = listenPayload.RootElement.GetProperty("data").GetProperty("nlu");
Assert.Equal("volumeDown", nlu.GetProperty("intent").GetString());
Assert.Equal("global_commands", nlu.GetProperty("domain").GetString());
Assert.Equal("null", nlu.GetProperty("entities").GetProperty("volumeLevel").GetString());
Assert.Equal("globals/global_commands_launch", nlu.GetProperty("rules")[0].GetString());
}
[Fact]
public async Task ClientAsr_SetVolumeToSix_EmitsGlobalVolumeToValue()
{
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
{
HostName = "neo-hub.jibo.com",
Path = "/listen",
Kind = "neo-hub-listen",
Token = "hub-volume-value-token",
Text = """{"type":"LISTEN","transID":"trans-volume-value","data":{"hotphrase":true,"rules":["launch","globals/global_commands_launch"]}}"""
});
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
{
HostName = "neo-hub.jibo.com",
Path = "/listen",
Kind = "neo-hub-listen",
Token = "hub-volume-value-token",
Text = """{"type":"CLIENT_ASR","transID":"trans-volume-value","data":{"text":"set volume to six"}}"""
});
Assert.Equal(2, replies.Count);
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
var nlu = listenPayload.RootElement.GetProperty("data").GetProperty("nlu");
Assert.Equal("volumeToValue", nlu.GetProperty("intent").GetString());
Assert.Equal("6", nlu.GetProperty("entities").GetProperty("volumeLevel").GetString());
Assert.Equal("global_commands", nlu.GetProperty("domain").GetString());
}
[Fact]
public async Task ClientAsr_ShowVolumeControls_RedirectsIntoSettingsVolumeQuery()
{
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
{
HostName = "neo-hub.jibo.com",
Path = "/listen",
Kind = "neo-hub-listen",
Token = "hub-volume-query-token",
Text = """{"type":"LISTEN","transID":"trans-volume-query","data":{"hotphrase":true,"rules":["launch","globals/global_commands_launch"]}}"""
});
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
{
HostName = "neo-hub.jibo.com",
Path = "/listen",
Kind = "neo-hub-listen",
Token = "hub-volume-query-token",
Text = """{"type":"CLIENT_ASR","transID":"trans-volume-query","data":{"text":"show volume controls"}}"""
});
Assert.Equal(4, replies.Count);
Assert.Equal("SKILL_REDIRECT", ReadReplyType(replies[2]));
Assert.Equal("SKILL_ACTION", ReadReplyType(replies[3]));
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
Assert.Equal("volumeQuery", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
Assert.Equal("@be/settings", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("skill").GetString());
using var redirectPayload = JsonDocument.Parse(replies[2].Text!);
Assert.Equal("@be/settings", redirectPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("skillID").GetString());
Assert.Equal("volumeQuery", redirectPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
}
[Fact]
public async Task ClientNlu_WordOfDayGuess_UsesGuessEntityAsAsrTextAndCompletesTurn()
{