jibo clock family skills and feature backlog updates
This commit is contained in:
@@ -177,6 +177,13 @@ Latest news discovery findings:
|
|||||||
- The first OpenJibo news pass should therefore use a real cloud-skill shape, not a generic placeholder chat reply.
|
- The first OpenJibo news pass should therefore use a real cloud-skill shape, not a generic placeholder chat reply.
|
||||||
- For now, the content can stay synthetic while the protocol is grounded: `match.cloudSkill = "news"` plus a supported `SLIM` announcement response is enough to validate the robot path before provider-backed headlines arrive later.
|
- For now, the content can stay synthetic while the protocol is grounded: `match.cloudSkill = "news"` plus a supported `SLIM` announcement response is enough to validate the robot path before provider-backed headlines arrive later.
|
||||||
|
|
||||||
|
Latest clock discovery findings:
|
||||||
|
|
||||||
|
- `@be/clock` is a real local skill with `clock`, `timer`, and `alarm` domains.
|
||||||
|
- Menu launches use `intent = "menu"` with `entities.domain` set to the target sub-area.
|
||||||
|
- Direct timer and alarm actions use `timerValue` and `alarmValue` utterances, not a generic chat path.
|
||||||
|
- A practical first OpenJibo slice is therefore: keep custom spoken time/date answers for now, but route `open clock`, `open timer`, `open alarm`, `set a timer ...`, and `set an alarm ...` through stock-shaped local `@be/clock` handoffs.
|
||||||
|
|
||||||
## Speech, Animation, And ESML
|
## Speech, Animation, And ESML
|
||||||
|
|
||||||
The current joke flow is only a small foothold into Jibo expressiveness.
|
The current joke flow is only a small foothold into Jibo expressiveness.
|
||||||
|
|||||||
@@ -120,15 +120,20 @@ Parallel tags:
|
|||||||
|
|
||||||
### 6. Clock Family Audit
|
### 6. Clock Family Audit
|
||||||
|
|
||||||
- Status: `ready`
|
- Status: `in_progress`
|
||||||
- Tags: `protocol`
|
- Tags: `protocol`
|
||||||
- Why now: clock, date, timer, and alarm menu hooks are already visible in captures and the robot repo has a real `@be/clock` skill.
|
- Why now: clock, date, timer, and alarm menu hooks are already visible in captures and the robot repo has a real `@be/clock` skill.
|
||||||
- Current evidence:
|
- Current evidence:
|
||||||
- [protocol-inventory.md](C:/Projects/JiboExperiments/OpenJibo/docs/protocol-inventory.md) already tracks menu intents for `askForTime`, `askForDate`, `timerValue`, and `alarmValue`
|
- [protocol-inventory.md](C:/Projects/JiboExperiments/OpenJibo/docs/protocol-inventory.md) already tracks menu intents for `askForTime`, `askForDate`, `timerValue`, and `alarmValue`
|
||||||
- `@be/clock` exists in the robot skill inventory
|
- `@be/clock` exists in the robot skill inventory
|
||||||
|
- `JiboOs` shows `@be/clock` branches on `entities.domain = clock | timer | alarm`, uses `intent = menu` for menu launches, and accepts direct `timerValue` / `alarmValue` utterances with structured entities
|
||||||
- Implementation notes:
|
- Implementation notes:
|
||||||
- compare our custom time/date path against actual menu payloads
|
- compare our custom time/date path against actual menu payloads
|
||||||
- decide whether timer and alarm should stay robot-local with cloud acknowledgement, or whether cloud needs to shape the launch and follow-up turns
|
- decide whether timer and alarm should stay robot-local with cloud acknowledgement, or whether cloud needs to shape the launch and follow-up turns
|
||||||
|
- Progress so far:
|
||||||
|
- voice `open clock`, `open timer`, and `open alarm` now synthesize stock-shaped local `@be/clock` launches
|
||||||
|
- voice `set a timer for five minutes` and `set an alarm for 7:30 am` now emit direct `timerValue` / `alarmValue` payloads with the domain and value entities the local skill expects
|
||||||
|
- time/date remain on the existing custom cloud reply path for now
|
||||||
- Exit criteria:
|
- Exit criteria:
|
||||||
- time/date behavior stays correct
|
- time/date behavior stays correct
|
||||||
- timer and alarm launch or set correctly from both menu and voice where applicable
|
- timer and alarm launch or set correctly from both menu and voice where applicable
|
||||||
@@ -242,9 +247,18 @@ Parallel tags:
|
|||||||
- should calendar and commute be independent feature paths or sections inside personal report
|
- should calendar and commute be independent feature paths or sections inside personal report
|
||||||
- what minimum provider data shape lets Jibo present these naturally
|
- what minimum provider data shape lets Jibo present these naturally
|
||||||
|
|
||||||
|
### 14. Stop Command
|
||||||
|
- Status: `ready`
|
||||||
|
- Tags: `protocol`
|
||||||
|
- Why later: Jibo can be interrupted by any command, but it would be nice to have a dedicated "stop" type of command.
|
||||||
|
- Current evidence:
|
||||||
|
- There is an Idle skill or subskill under @be so I think we can utilize it, but I am not sure if that was the default behavior.
|
||||||
|
- Questions to answer:
|
||||||
|
- Can we find in the original source evidence for this skill or stop word phrase?
|
||||||
|
|
||||||
## Support Tracks
|
## Support Tracks
|
||||||
|
|
||||||
### 14. Hosted Capture And Storage Plan
|
### 15. Hosted Capture And Storage Plan
|
||||||
|
|
||||||
- Status: `ready`
|
- Status: `ready`
|
||||||
- Tags: `docs`
|
- Tags: `docs`
|
||||||
@@ -253,7 +267,7 @@ Parallel tags:
|
|||||||
- define a clean boundary between local capture sinks and hosted archival/export
|
- define a clean boundary between local capture sinks and hosted archival/export
|
||||||
- document how group testers should submit sessions without touching repo paths directly
|
- document how group testers should submit sessions without touching repo paths directly
|
||||||
|
|
||||||
### 15. STT Upgrade And Noise Screening
|
### 16. STT Upgrade And Noise Screening
|
||||||
|
|
||||||
- Status: `ready`
|
- Status: `ready`
|
||||||
- Tags: `stt`
|
- Tags: `stt`
|
||||||
|
|||||||
@@ -74,6 +74,11 @@ public sealed class DemoConversationBroker(JiboInteractionService interactionSer
|
|||||||
"word_of_the_day_guess" => false,
|
"word_of_the_day_guess" => false,
|
||||||
"radio" => false,
|
"radio" => false,
|
||||||
"radio_genre" => false,
|
"radio_genre" => false,
|
||||||
|
"clock_menu" => false,
|
||||||
|
"timer_menu" => false,
|
||||||
|
"alarm_menu" => false,
|
||||||
|
"timer_value" => false,
|
||||||
|
"alarm_value" => false,
|
||||||
"news" => false,
|
"news" => false,
|
||||||
_ => true
|
_ => true
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
using Jibo.Cloud.Application.Abstractions;
|
using Jibo.Cloud.Application.Abstractions;
|
||||||
using Jibo.Runtime.Abstractions;
|
using Jibo.Runtime.Abstractions;
|
||||||
using System.Text.Json;
|
using System.Text.Json;
|
||||||
|
using System.Text.RegularExpressions;
|
||||||
|
|
||||||
namespace Jibo.Cloud.Application.Services;
|
namespace Jibo.Cloud.Application.Services;
|
||||||
|
|
||||||
@@ -29,9 +30,15 @@ public sealed class JiboInteractionService(
|
|||||||
"dance" => BuildDanceDecision(catalog),
|
"dance" => BuildDanceDecision(catalog),
|
||||||
"time" => new JiboInteractionDecision("time", $"It is {DateTime.Now:h:mm tt}."),
|
"time" => new JiboInteractionDecision("time", $"It is {DateTime.Now:h:mm tt}."),
|
||||||
"date" => new JiboInteractionDecision("date", $"Today is {DateTime.Now:dddd, MMMM d}."),
|
"date" => new JiboInteractionDecision("date", $"Today is {DateTime.Now:dddd, MMMM d}."),
|
||||||
|
"day" => new JiboInteractionDecision("day", $"Today is {DateTime.Now:dddd}."),
|
||||||
"cloud_version" => new JiboInteractionDecision("cloud_version", OpenJiboCloudBuildInfo.SpokenVersion),
|
"cloud_version" => new JiboInteractionDecision("cloud_version", OpenJiboCloudBuildInfo.SpokenVersion),
|
||||||
"radio" => BuildRadioLaunchDecision(),
|
"radio" => BuildRadioLaunchDecision(),
|
||||||
"radio_genre" => BuildRadioGenreLaunchDecision(lowered),
|
"radio_genre" => BuildRadioGenreLaunchDecision(lowered),
|
||||||
|
"clock_menu" => BuildClockLaunchDecision("clock", "Opening the clock."),
|
||||||
|
"timer_menu" => BuildClockLaunchDecision("timer", "Opening the timer."),
|
||||||
|
"alarm_menu" => BuildClockLaunchDecision("alarm", "Opening the alarm."),
|
||||||
|
"timer_value" => BuildTimerValueDecision(lowered),
|
||||||
|
"alarm_value" => BuildAlarmValueDecision(lowered),
|
||||||
"hello" => new JiboInteractionDecision("hello", randomizer.Choose(catalog.GreetingReplies)),
|
"hello" => new JiboInteractionDecision("hello", randomizer.Choose(catalog.GreetingReplies)),
|
||||||
"how_are_you" => new JiboInteractionDecision("how_are_you", randomizer.Choose(catalog.HowAreYouReplies)),
|
"how_are_you" => new JiboInteractionDecision("how_are_you", randomizer.Choose(catalog.HowAreYouReplies)),
|
||||||
"yes" => new JiboInteractionDecision("yes", "Yes."),
|
"yes" => new JiboInteractionDecision("yes", "Yes."),
|
||||||
@@ -149,6 +156,33 @@ public sealed class JiboInteractionService(
|
|||||||
return "date";
|
return "date";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (string.Equals(clientIntent, "askForDay", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
return "day";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (string.Equals(clientIntent, "timerValue", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
return "timer_value";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (string.Equals(clientIntent, "alarmValue", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
return "alarm_value";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (string.Equals(clientIntent, "menu", StringComparison.OrdinalIgnoreCase) &&
|
||||||
|
clientEntities.TryGetValue("domain", out var clockDomain))
|
||||||
|
{
|
||||||
|
return clockDomain.ToLowerInvariant() switch
|
||||||
|
{
|
||||||
|
"clock" => "clock_menu",
|
||||||
|
"timer" => "timer_menu",
|
||||||
|
"alarm" => "alarm_menu",
|
||||||
|
_ => "chat"
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
if (MatchesAny(
|
if (MatchesAny(
|
||||||
loweredTranscript,
|
loweredTranscript,
|
||||||
"word of the day",
|
"word of the day",
|
||||||
@@ -187,6 +221,31 @@ public sealed class JiboInteractionService(
|
|||||||
return "radio_genre";
|
return "radio_genre";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (TryParseAlarmValue(loweredTranscript) is not null)
|
||||||
|
{
|
||||||
|
return "alarm_value";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (TryParseTimerValue(loweredTranscript) is not null)
|
||||||
|
{
|
||||||
|
return "timer_value";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (MatchesAny(loweredTranscript, "open the clock", "open clock", "show the clock", "show clock"))
|
||||||
|
{
|
||||||
|
return "clock_menu";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (MatchesAny(loweredTranscript, "open the timer", "open timer", "show the timer", "show timer"))
|
||||||
|
{
|
||||||
|
return "timer_menu";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (MatchesAny(loweredTranscript, "open the alarm", "open alarm", "show the alarm", "show alarm"))
|
||||||
|
{
|
||||||
|
return "alarm_menu";
|
||||||
|
}
|
||||||
|
|
||||||
if (MatchesAny(loweredTranscript, "open the radio", "play the radio", "turn on the radio", "radio"))
|
if (MatchesAny(loweredTranscript, "open the radio", "play the radio", "turn on the radio", "radio"))
|
||||||
{
|
{
|
||||||
return "radio";
|
return "radio";
|
||||||
@@ -253,6 +312,11 @@ public sealed class JiboInteractionService(
|
|||||||
return "time";
|
return "time";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (MatchesAny(loweredTranscript, "what day is it", "what day is today"))
|
||||||
|
{
|
||||||
|
return "day";
|
||||||
|
}
|
||||||
|
|
||||||
if (MatchesAny(loweredTranscript, "what day is it", "what is the date", "today s date", "today's date") ||
|
if (MatchesAny(loweredTranscript, "what day is it", "what is the date", "today s date", "today's date") ||
|
||||||
loweredTranscript.Contains("date", StringComparison.Ordinal) ||
|
loweredTranscript.Contains("date", StringComparison.Ordinal) ||
|
||||||
loweredTranscript.Contains("day", StringComparison.Ordinal))
|
loweredTranscript.Contains("day", StringComparison.Ordinal))
|
||||||
@@ -288,6 +352,57 @@ public sealed class JiboInteractionService(
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private static JiboInteractionDecision BuildClockLaunchDecision(string domain, string replyText)
|
||||||
|
{
|
||||||
|
return new JiboInteractionDecision(
|
||||||
|
$"{domain}_menu",
|
||||||
|
replyText,
|
||||||
|
"@be/clock",
|
||||||
|
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
|
||||||
|
{
|
||||||
|
["skillId"] = "@be/clock",
|
||||||
|
["domain"] = domain,
|
||||||
|
["clockIntent"] = "menu"
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
private static JiboInteractionDecision BuildTimerValueDecision(string loweredTranscript)
|
||||||
|
{
|
||||||
|
var timer = TryParseTimerValue(loweredTranscript) ?? new ClockTimerValue("0", "1", "null");
|
||||||
|
|
||||||
|
return new JiboInteractionDecision(
|
||||||
|
"timer_value",
|
||||||
|
"Setting your timer.",
|
||||||
|
"@be/clock",
|
||||||
|
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
|
||||||
|
{
|
||||||
|
["skillId"] = "@be/clock",
|
||||||
|
["domain"] = "timer",
|
||||||
|
["clockIntent"] = "timerValue",
|
||||||
|
["hours"] = timer.Hours,
|
||||||
|
["minutes"] = timer.Minutes,
|
||||||
|
["seconds"] = timer.Seconds
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
private static JiboInteractionDecision BuildAlarmValueDecision(string loweredTranscript)
|
||||||
|
{
|
||||||
|
var alarm = TryParseAlarmValue(loweredTranscript) ?? new ClockAlarmValue("7:00", "am");
|
||||||
|
|
||||||
|
return new JiboInteractionDecision(
|
||||||
|
"alarm_value",
|
||||||
|
"Setting your alarm.",
|
||||||
|
"@be/clock",
|
||||||
|
new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase)
|
||||||
|
{
|
||||||
|
["skillId"] = "@be/clock",
|
||||||
|
["domain"] = "alarm",
|
||||||
|
["clockIntent"] = "alarmValue",
|
||||||
|
["time"] = alarm.Time,
|
||||||
|
["ampm"] = alarm.AmPm
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
private static JiboInteractionDecision BuildRadioGenreLaunchDecision(string loweredTranscript)
|
private static JiboInteractionDecision BuildRadioGenreLaunchDecision(string loweredTranscript)
|
||||||
{
|
{
|
||||||
var station = TryResolveRadioGenre(loweredTranscript) ?? "Country";
|
var station = TryResolveRadioGenre(loweredTranscript) ?? "Country";
|
||||||
@@ -516,6 +631,116 @@ public sealed class JiboInteractionService(
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private static ClockTimerValue? TryParseTimerValue(string loweredTranscript)
|
||||||
|
{
|
||||||
|
if (!loweredTranscript.Contains("timer", StringComparison.Ordinal))
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
var hours = ExtractDurationValue(loweredTranscript, "hour");
|
||||||
|
var minutes = ExtractDurationValue(loweredTranscript, "minute");
|
||||||
|
var seconds = ExtractDurationValue(loweredTranscript, "second");
|
||||||
|
|
||||||
|
if (hours is null && minutes is null && seconds is null)
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
return new ClockTimerValue(
|
||||||
|
(hours ?? 0).ToString(),
|
||||||
|
(minutes ?? 0).ToString(),
|
||||||
|
seconds is null ? "null" : seconds.Value.ToString());
|
||||||
|
}
|
||||||
|
|
||||||
|
private static ClockAlarmValue? TryParseAlarmValue(string loweredTranscript)
|
||||||
|
{
|
||||||
|
if (!loweredTranscript.Contains("alarm", StringComparison.Ordinal))
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
var match = AlarmPattern.Match(loweredTranscript);
|
||||||
|
if (!match.Success)
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
var hourToken = match.Groups["hour"].Value;
|
||||||
|
var minuteToken = match.Groups["minute"].Success ? match.Groups["minute"].Value : "00";
|
||||||
|
var hour = ParseNumberToken(hourToken);
|
||||||
|
if (hour is null || hour is < 1 or > 12)
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!int.TryParse(minuteToken, out var minute) || minute is < 0 or > 59)
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
var ampm = match.Groups["ampm"].Value.StartsWith("p", StringComparison.Ordinal) ? "pm" : "am";
|
||||||
|
return new ClockAlarmValue($"{hour}:{minute:00}", ampm);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static int? ExtractDurationValue(string loweredTranscript, string unitStem)
|
||||||
|
{
|
||||||
|
var pattern = new Regex($@"\b(?<value>\d+|[a-z\-]+)\s+{unitStem}s?\b", RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);
|
||||||
|
var match = pattern.Match(loweredTranscript);
|
||||||
|
if (!match.Success)
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
return ParseNumberToken(match.Groups["value"].Value);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static int? ParseNumberToken(string token)
|
||||||
|
{
|
||||||
|
var normalized = token.Trim().ToLowerInvariant();
|
||||||
|
if (int.TryParse(normalized, out var numeric))
|
||||||
|
{
|
||||||
|
return numeric;
|
||||||
|
}
|
||||||
|
|
||||||
|
return normalized switch
|
||||||
|
{
|
||||||
|
"a" or "an" => 1,
|
||||||
|
"one" => 1,
|
||||||
|
"two" => 2,
|
||||||
|
"three" => 3,
|
||||||
|
"four" => 4,
|
||||||
|
"five" => 5,
|
||||||
|
"six" => 6,
|
||||||
|
"seven" => 7,
|
||||||
|
"eight" => 8,
|
||||||
|
"nine" => 9,
|
||||||
|
"ten" => 10,
|
||||||
|
"eleven" => 11,
|
||||||
|
"twelve" => 12,
|
||||||
|
"thirteen" => 13,
|
||||||
|
"fourteen" => 14,
|
||||||
|
"fifteen" => 15,
|
||||||
|
"sixteen" => 16,
|
||||||
|
"seventeen" => 17,
|
||||||
|
"eighteen" => 18,
|
||||||
|
"nineteen" => 19,
|
||||||
|
"twenty" => 20,
|
||||||
|
"thirty" => 30,
|
||||||
|
"forty" => 40,
|
||||||
|
"fifty" => 50,
|
||||||
|
_ => null
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
private sealed record ClockTimerValue(string Hours, string Minutes, string Seconds);
|
||||||
|
|
||||||
|
private sealed record ClockAlarmValue(string Time, string AmPm);
|
||||||
|
|
||||||
|
private static readonly Regex AlarmPattern = new(
|
||||||
|
@"\b(?<hour>\d{1,2}|one|two|three|four|five|six|seven|eight|nine|ten|eleven|twelve)(?:[:\s](?<minute>\d{2}))?\s*(?<ampm>a\.?m\.?|p\.?m\.?)\b",
|
||||||
|
RegexOptions.IgnoreCase | RegexOptions.CultureInvariant | RegexOptions.Compiled);
|
||||||
|
|
||||||
private static readonly (string Phrase, string Station)[] RadioGenreAliases =
|
private static readonly (string Phrase, string Station)[] RadioGenreAliases =
|
||||||
[
|
[
|
||||||
("country music", "Country"),
|
("country music", "Country"),
|
||||||
|
|||||||
@@ -25,6 +25,14 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
|||||||
var isWordOfDayGuess = string.Equals(plan.IntentName, "word_of_the_day_guess", StringComparison.OrdinalIgnoreCase);
|
var isWordOfDayGuess = string.Equals(plan.IntentName, "word_of_the_day_guess", StringComparison.OrdinalIgnoreCase);
|
||||||
var isRadioLaunch = string.Equals(plan.IntentName, "radio", StringComparison.OrdinalIgnoreCase) ||
|
var isRadioLaunch = string.Equals(plan.IntentName, "radio", StringComparison.OrdinalIgnoreCase) ||
|
||||||
string.Equals(plan.IntentName, "radio_genre", StringComparison.OrdinalIgnoreCase);
|
string.Equals(plan.IntentName, "radio_genre", StringComparison.OrdinalIgnoreCase);
|
||||||
|
var isClockSkillLaunch = string.Equals(skill?.SkillName, "@be/clock", StringComparison.OrdinalIgnoreCase);
|
||||||
|
var clockIntent = ReadSkillPayloadString(skill, "clockIntent");
|
||||||
|
var clockDomain = ReadSkillPayloadString(skill, "domain");
|
||||||
|
var timerHours = ReadSkillPayloadString(skill, "hours");
|
||||||
|
var timerMinutes = ReadSkillPayloadString(skill, "minutes");
|
||||||
|
var timerSeconds = ReadSkillPayloadString(skill, "seconds");
|
||||||
|
var alarmTime = ReadSkillPayloadString(skill, "time");
|
||||||
|
var alarmAmPm = ReadSkillPayloadString(skill, "ampm");
|
||||||
var radioStation = ReadSkillPayloadString(skill, "station");
|
var radioStation = ReadSkillPayloadString(skill, "station");
|
||||||
var cloudSkill = ReadSkillPayloadString(skill, "cloudSkill");
|
var cloudSkill = ReadSkillPayloadString(skill, "cloudSkill");
|
||||||
var nluGuess = ReadClientEntity(turn, "guess");
|
var nluGuess = ReadClientEntity(turn, "guess");
|
||||||
@@ -33,6 +41,8 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
|||||||
? "menu"
|
? "menu"
|
||||||
: isRadioLaunch
|
: isRadioLaunch
|
||||||
? "menu"
|
? "menu"
|
||||||
|
: isClockSkillLaunch && !string.IsNullOrWhiteSpace(clockIntent)
|
||||||
|
? clockIntent
|
||||||
: isWordOfDayGuess
|
: isWordOfDayGuess
|
||||||
? "guess"
|
? "guess"
|
||||||
: string.Equals(messageType, "CLIENT_NLU", StringComparison.OrdinalIgnoreCase) && !string.IsNullOrWhiteSpace(clientIntent)
|
: string.Equals(messageType, "CLIENT_NLU", StringComparison.OrdinalIgnoreCase) && !string.IsNullOrWhiteSpace(clientIntent)
|
||||||
@@ -44,6 +54,8 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
|||||||
? string.Empty
|
? string.Empty
|
||||||
: isRadioLaunch
|
: isRadioLaunch
|
||||||
? transcript
|
? transcript
|
||||||
|
: isClockSkillLaunch
|
||||||
|
? transcript
|
||||||
: string.Equals(clientIntent, "guess", StringComparison.OrdinalIgnoreCase) && !string.IsNullOrWhiteSpace(nluGuess)
|
: string.Equals(clientIntent, "guess", StringComparison.OrdinalIgnoreCase) && !string.IsNullOrWhiteSpace(nluGuess)
|
||||||
? nluGuess
|
? nluGuess
|
||||||
: isYesNoTurn && isYesNoIntent
|
: isYesNoTurn && isYesNoIntent
|
||||||
@@ -55,6 +67,8 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
|||||||
? ["word-of-the-day/menu"]
|
? ["word-of-the-day/menu"]
|
||||||
: isRadioLaunch
|
: isRadioLaunch
|
||||||
? Array.Empty<string>()
|
? Array.Empty<string>()
|
||||||
|
: isClockSkillLaunch
|
||||||
|
? string.Equals(messageType, "CLIENT_NLU", StringComparison.OrdinalIgnoreCase) ? rules : Array.Empty<string>()
|
||||||
: isWordOfDayGuess
|
: isWordOfDayGuess
|
||||||
? ["word-of-the-day/puzzle"]
|
? ["word-of-the-day/puzzle"]
|
||||||
: isYesNoTurn && isYesNoIntent ? [yesNoRule!] : rules;
|
: isYesNoTurn && isYesNoIntent ? [yesNoRule!] : rules;
|
||||||
@@ -67,7 +81,15 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
|||||||
isRadioLaunch,
|
isRadioLaunch,
|
||||||
isWordOfDayGuess,
|
isWordOfDayGuess,
|
||||||
wordOfDayGuess,
|
wordOfDayGuess,
|
||||||
radioStation);
|
radioStation,
|
||||||
|
isClockSkillLaunch,
|
||||||
|
clockDomain,
|
||||||
|
clockIntent,
|
||||||
|
timerHours,
|
||||||
|
timerMinutes,
|
||||||
|
timerSeconds,
|
||||||
|
alarmTime,
|
||||||
|
alarmAmPm);
|
||||||
var listenMessage = new
|
var listenMessage = new
|
||||||
{
|
{
|
||||||
type = "LISTEN",
|
type = "LISTEN",
|
||||||
@@ -80,7 +102,14 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
|||||||
final = true,
|
final = true,
|
||||||
text = outboundAsrText
|
text = outboundAsrText
|
||||||
},
|
},
|
||||||
nlu = BuildNluPayload(outboundIntent, outboundRules, entities, isWordOfDayLaunch ? "@be/word-of-the-day" : isRadioLaunch ? "@be/radio" : null),
|
nlu = BuildNluPayload(
|
||||||
|
outboundIntent,
|
||||||
|
outboundRules,
|
||||||
|
entities,
|
||||||
|
isWordOfDayLaunch ? "@be/word-of-the-day" :
|
||||||
|
isRadioLaunch ? "@be/radio" :
|
||||||
|
isClockSkillLaunch ? "@be/clock" :
|
||||||
|
null),
|
||||||
match = new
|
match = new
|
||||||
{
|
{
|
||||||
intent = outboundIntent,
|
intent = outboundIntent,
|
||||||
@@ -136,6 +165,23 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
|||||||
DelayMs: 125));
|
DelayMs: 125));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (isClockSkillLaunch &&
|
||||||
|
!string.Equals(messageType, "CLIENT_NLU", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
messages.Add(new SocketReplyPlan(
|
||||||
|
JsonSerializer.Serialize(BuildSkillRedirectPayload(
|
||||||
|
transId,
|
||||||
|
"@be/clock",
|
||||||
|
outboundIntent,
|
||||||
|
outboundAsrText,
|
||||||
|
outboundRules,
|
||||||
|
entities)),
|
||||||
|
DelayMs: 75));
|
||||||
|
messages.Add(new SocketReplyPlan(
|
||||||
|
JsonSerializer.Serialize(BuildCompletionOnlySkillPayload(transId, "@be/clock")),
|
||||||
|
DelayMs: 125));
|
||||||
|
}
|
||||||
|
|
||||||
if (emitSkillActions && speak is not null)
|
if (emitSkillActions && speak is not null)
|
||||||
{
|
{
|
||||||
messages.Add(new SocketReplyPlan(
|
messages.Add(new SocketReplyPlan(
|
||||||
@@ -282,7 +328,15 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
|||||||
bool radioLaunch,
|
bool radioLaunch,
|
||||||
bool wordOfDayGuess,
|
bool wordOfDayGuess,
|
||||||
string? guess,
|
string? guess,
|
||||||
string? radioStation)
|
string? radioStation,
|
||||||
|
bool clockSkillLaunch,
|
||||||
|
string? clockDomain,
|
||||||
|
string? clockIntent,
|
||||||
|
string? timerHours,
|
||||||
|
string? timerMinutes,
|
||||||
|
string? timerSeconds,
|
||||||
|
string? alarmTime,
|
||||||
|
string? alarmAmPm)
|
||||||
{
|
{
|
||||||
if (yesNoTurn)
|
if (yesNoTurn)
|
||||||
{
|
{
|
||||||
@@ -316,6 +370,30 @@ public sealed class ResponsePlanToSocketMessagesMapper
|
|||||||
return entities;
|
return entities;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (clockSkillLaunch)
|
||||||
|
{
|
||||||
|
var entities = new Dictionary<string, object?>(StringComparer.OrdinalIgnoreCase);
|
||||||
|
if (!string.IsNullOrWhiteSpace(clockDomain))
|
||||||
|
{
|
||||||
|
entities["domain"] = clockDomain;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (string.Equals(clockIntent, "timerValue", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
entities["hours"] = timerHours ?? "0";
|
||||||
|
entities["minutes"] = timerMinutes ?? "0";
|
||||||
|
entities["seconds"] = timerSeconds ?? "null";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (string.Equals(clockIntent, "alarmValue", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
entities["time"] = alarmTime ?? string.Empty;
|
||||||
|
entities["ampm"] = alarmAmPm ?? string.Empty;
|
||||||
|
}
|
||||||
|
|
||||||
|
return entities;
|
||||||
|
}
|
||||||
|
|
||||||
if (wordOfDayGuess)
|
if (wordOfDayGuess)
|
||||||
{
|
{
|
||||||
return new Dictionary<string, object?>
|
return new Dictionary<string, object?>
|
||||||
|
|||||||
@@ -564,6 +564,11 @@ public sealed class WebSocketTurnFinalizationService(
|
|||||||
var emitSkillActions = !string.Equals(plan.IntentName, "word_of_the_day", StringComparison.OrdinalIgnoreCase) &&
|
var emitSkillActions = !string.Equals(plan.IntentName, "word_of_the_day", StringComparison.OrdinalIgnoreCase) &&
|
||||||
!string.Equals(plan.IntentName, "radio", StringComparison.OrdinalIgnoreCase) &&
|
!string.Equals(plan.IntentName, "radio", StringComparison.OrdinalIgnoreCase) &&
|
||||||
!string.Equals(plan.IntentName, "radio_genre", StringComparison.OrdinalIgnoreCase) &&
|
!string.Equals(plan.IntentName, "radio_genre", StringComparison.OrdinalIgnoreCase) &&
|
||||||
|
!string.Equals(plan.IntentName, "clock_menu", StringComparison.OrdinalIgnoreCase) &&
|
||||||
|
!string.Equals(plan.IntentName, "timer_menu", StringComparison.OrdinalIgnoreCase) &&
|
||||||
|
!string.Equals(plan.IntentName, "alarm_menu", StringComparison.OrdinalIgnoreCase) &&
|
||||||
|
!string.Equals(plan.IntentName, "timer_value", StringComparison.OrdinalIgnoreCase) &&
|
||||||
|
!string.Equals(plan.IntentName, "alarm_value", StringComparison.OrdinalIgnoreCase) &&
|
||||||
(messageType != "CLIENT_NLU" ||
|
(messageType != "CLIENT_NLU" ||
|
||||||
string.Equals(plan.IntentName, "word_of_the_day_guess", StringComparison.OrdinalIgnoreCase));
|
string.Equals(plan.IntentName, "word_of_the_day_guess", StringComparison.OrdinalIgnoreCase));
|
||||||
var replies = ResponsePlanToSocketMessagesMapper.Map(plan, finalizedTurn, session, emitSkillActions).Select(map => new WebSocketReply
|
var replies = ResponsePlanToSocketMessagesMapper.Map(plan, finalizedTurn, session, emitSkillActions).Select(map => new WebSocketReply
|
||||||
|
|||||||
@@ -181,6 +181,62 @@ public sealed class JiboInteractionServiceTests
|
|||||||
Assert.Equal("Country", decision.SkillPayload!["station"]);
|
Assert.Equal("Country", decision.SkillPayload!["station"]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task BuildDecisionAsync_OpenTimer_MapsToLocalClockTimerMenu()
|
||||||
|
{
|
||||||
|
var service = CreateService();
|
||||||
|
|
||||||
|
var decision = await service.BuildDecisionAsync(new TurnContext
|
||||||
|
{
|
||||||
|
RawTranscript = "open timer",
|
||||||
|
NormalizedTranscript = "open timer"
|
||||||
|
});
|
||||||
|
|
||||||
|
Assert.Equal("timer_menu", decision.IntentName);
|
||||||
|
Assert.Equal("@be/clock", decision.SkillName);
|
||||||
|
Assert.Equal("timer", decision.SkillPayload!["domain"]);
|
||||||
|
Assert.Equal("menu", decision.SkillPayload["clockIntent"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task BuildDecisionAsync_SetTimerForFiveMinutes_MapsToTimerValue()
|
||||||
|
{
|
||||||
|
var service = CreateService();
|
||||||
|
|
||||||
|
var decision = await service.BuildDecisionAsync(new TurnContext
|
||||||
|
{
|
||||||
|
RawTranscript = "set a timer for five minutes",
|
||||||
|
NormalizedTranscript = "set a timer for five minutes"
|
||||||
|
});
|
||||||
|
|
||||||
|
Assert.Equal("timer_value", decision.IntentName);
|
||||||
|
Assert.Equal("@be/clock", decision.SkillName);
|
||||||
|
Assert.Equal("timer", decision.SkillPayload!["domain"]);
|
||||||
|
Assert.Equal("timerValue", decision.SkillPayload["clockIntent"]);
|
||||||
|
Assert.Equal("0", decision.SkillPayload["hours"]);
|
||||||
|
Assert.Equal("5", decision.SkillPayload["minutes"]);
|
||||||
|
Assert.Equal("null", decision.SkillPayload["seconds"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task BuildDecisionAsync_SetAlarmForSevenThirtyAm_MapsToAlarmValue()
|
||||||
|
{
|
||||||
|
var service = CreateService();
|
||||||
|
|
||||||
|
var decision = await service.BuildDecisionAsync(new TurnContext
|
||||||
|
{
|
||||||
|
RawTranscript = "set an alarm for 7:30 am",
|
||||||
|
NormalizedTranscript = "set an alarm for 7:30 am"
|
||||||
|
});
|
||||||
|
|
||||||
|
Assert.Equal("alarm_value", decision.IntentName);
|
||||||
|
Assert.Equal("@be/clock", decision.SkillName);
|
||||||
|
Assert.Equal("alarm", decision.SkillPayload!["domain"]);
|
||||||
|
Assert.Equal("alarmValue", decision.SkillPayload["clockIntent"]);
|
||||||
|
Assert.Equal("7:30", decision.SkillPayload["time"]);
|
||||||
|
Assert.Equal("am", decision.SkillPayload["ampm"]);
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task BuildDecisionAsync_TellMeTheNews_UsesNimbusCloudSkillPath()
|
public async Task BuildDecisionAsync_TellMeTheNews_UsesNimbusCloudSkillPath()
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -343,6 +343,77 @@ public sealed class JiboWebSocketServiceTests
|
|||||||
Assert.Equal("clock/clock_menu", listenPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("rule").GetString());
|
Assert.Equal("clock/clock_menu", listenPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("rule").GetString());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task ClientAsr_SetTimerForFiveMinutes_RedirectsIntoClockSkillWithTimerEntities()
|
||||||
|
{
|
||||||
|
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||||
|
{
|
||||||
|
HostName = "neo-hub.jibo.com",
|
||||||
|
Path = "/listen",
|
||||||
|
Kind = "neo-hub-listen",
|
||||||
|
Token = "hub-clock-timer-token",
|
||||||
|
Text = """{"type":"LISTEN","transID":"trans-clock-timer","data":{"rules":["globals/global_commands_launch"]}}"""
|
||||||
|
});
|
||||||
|
|
||||||
|
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||||
|
{
|
||||||
|
HostName = "neo-hub.jibo.com",
|
||||||
|
Path = "/listen",
|
||||||
|
Kind = "neo-hub-listen",
|
||||||
|
Token = "hub-clock-timer-token",
|
||||||
|
Text = """{"type":"CLIENT_ASR","transID":"trans-clock-timer","data":{"text":"set a timer for five minutes"}}"""
|
||||||
|
});
|
||||||
|
|
||||||
|
Assert.Equal(4, replies.Count);
|
||||||
|
Assert.Equal("LISTEN", ReadReplyType(replies[0]));
|
||||||
|
Assert.Equal("EOS", ReadReplyType(replies[1]));
|
||||||
|
Assert.Equal("SKILL_REDIRECT", ReadReplyType(replies[2]));
|
||||||
|
Assert.Equal("SKILL_ACTION", ReadReplyType(replies[3]));
|
||||||
|
|
||||||
|
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
|
||||||
|
Assert.Equal("timerValue", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||||
|
Assert.Equal("@be/clock", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("skill").GetString());
|
||||||
|
Assert.Equal("timer", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("domain").GetString());
|
||||||
|
Assert.Equal("0", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("hours").GetString());
|
||||||
|
Assert.Equal("5", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("minutes").GetString());
|
||||||
|
Assert.Equal("null", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("seconds").GetString());
|
||||||
|
|
||||||
|
using var redirectPayload = JsonDocument.Parse(replies[2].Text!);
|
||||||
|
Assert.Equal("@be/clock", redirectPayload.RootElement.GetProperty("data").GetProperty("match").GetProperty("skillID").GetString());
|
||||||
|
Assert.Equal("timerValue", redirectPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task ClientAsr_SetAlarmForSevenThirtyAm_RedirectsIntoClockSkillWithAlarmEntities()
|
||||||
|
{
|
||||||
|
await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||||
|
{
|
||||||
|
HostName = "neo-hub.jibo.com",
|
||||||
|
Path = "/listen",
|
||||||
|
Kind = "neo-hub-listen",
|
||||||
|
Token = "hub-clock-alarm-token",
|
||||||
|
Text = """{"type":"LISTEN","transID":"trans-clock-alarm","data":{"rules":["globals/global_commands_launch"]}}"""
|
||||||
|
});
|
||||||
|
|
||||||
|
var replies = await _service.HandleMessageAsync(new WebSocketMessageEnvelope
|
||||||
|
{
|
||||||
|
HostName = "neo-hub.jibo.com",
|
||||||
|
Path = "/listen",
|
||||||
|
Kind = "neo-hub-listen",
|
||||||
|
Token = "hub-clock-alarm-token",
|
||||||
|
Text = """{"type":"CLIENT_ASR","transID":"trans-clock-alarm","data":{"text":"set an alarm for 7:30 am"}}"""
|
||||||
|
});
|
||||||
|
|
||||||
|
Assert.Equal(4, replies.Count);
|
||||||
|
|
||||||
|
using var listenPayload = JsonDocument.Parse(replies[0].Text!);
|
||||||
|
Assert.Equal("alarmValue", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("intent").GetString());
|
||||||
|
Assert.Equal("@be/clock", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("skill").GetString());
|
||||||
|
Assert.Equal("alarm", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("domain").GetString());
|
||||||
|
Assert.Equal("7:30", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("time").GetString());
|
||||||
|
Assert.Equal("am", listenPayload.RootElement.GetProperty("data").GetProperty("nlu").GetProperty("entities").GetProperty("ampm").GetString());
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task ClientAsr_YesNoCreateFlow_PreservesCreateRuleAndDomain()
|
public async Task ClientAsr_YesNoCreateFlow_PreservesCreateRuleAndDomain()
|
||||||
{
|
{
|
||||||
|
|||||||
Reference in New Issue
Block a user