forked from Jibo-Revival-Group/JiboOs
327 lines
12 KiB
JavaScript
327 lines
12 KiB
JavaScript
/**
|
|
* @class ListenBundle
|
|
* @memberof jibo.mim
|
|
* @description Internal bundle of listen specific data.
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Timeout duration in milliseconds. If NaN, do not timeout.
|
|
* @name jibo.mim.ListenBundle#timeout
|
|
* @type {Number}
|
|
*/
|
|
|
|
/**
|
|
* Beginning of timeout in epoch time. Timeout is triggered when the difference between this
|
|
* time and current time is greater than timeout.
|
|
* @name jibo.mim.ListenBundle#startTime
|
|
* @type {Number}
|
|
*/
|
|
|
|
/**
|
|
* Listener from jibo.mim#listenDelegate.
|
|
* @name jibo.mim.ListenBundle#listener
|
|
* @type {any}
|
|
*/
|
|
|
|
/**
|
|
* If we had failed to get a listener previously. Skill will be exited on another failure.
|
|
* @name jibo.mim.ListenBundle#failedToGetListener
|
|
* @type {boolean}
|
|
*/
|
|
|
|
/**
|
|
* @class MimStates
|
|
* @memberof jibo.mim
|
|
* @description Bundle all the Mim's state machine states, with strong typing. Each
|
|
* property of MimStates is a State from jibo-state-machine. All actions are performed
|
|
* by functions of the Mim class.
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Return prompt data for MIM. Properties are used as variables
|
|
* for MIM condition evaluation and prompt text insertion.
|
|
* @callback jibo.bt.behaviors.Mim~GetPromptData
|
|
* @returns {any}
|
|
*/
|
|
|
|
/**
|
|
* Called on listen complete during Optional Response or Question, before the MIM analyzes the
|
|
* result. Allows skills to implement their own check on the result to tell the MIM how to handle it.
|
|
* @callback jibo.bt.behaviors.Mim~CheckResult
|
|
* @param {jibo.jetstream.types.ListenResult} result Data from NLU service.
|
|
*/
|
|
|
|
/**
|
|
* Called when an Optional Response or Question MIM completes successfully.
|
|
* Optional Response MIMs always complete;
|
|
* check `options.state.lastResultState` to see if user responded.
|
|
* @callback jibo.bt.behaviors.Mim~OnSuccess
|
|
* @param {any} results Bundles data from MIM.
|
|
* @param {jibo.jetstream.types.ListenResult} results.asrResults Data from NLU service.
|
|
* @param {jibo.mim.SpeakerIds} results.speakerIds If available, the ID of the speaker.
|
|
* @param {jibo.mim.MimState} results.state Current state of the MIM.
|
|
* @param {String} results.firstGrammarTag (Flow use only) The value of the first rule slot (results.asrResults.entities[results.asrResults.slotActions[0]]). The default transition from a MIM. If a listen has multiple potential slots, this value should not be considered reliable.
|
|
*/
|
|
|
|
/**
|
|
* (Flow use only) Called when a Question MIM fails entirely. Either the user did not
|
|
* respond to Jibo or Jibo could not understand the user and has run out of error prompts.
|
|
* Return value finds an error handler within the flow. Unhandled
|
|
* errors use Jibo's standard MIM exception handling.
|
|
* @callback jibo.bt.behaviors.Mim~OnFailure
|
|
* @param {any} results Bundles data from MIM.
|
|
* @param {jibo.jetstream.types.ListenResult} results.asrResults Data from NLU service.
|
|
* @param {jibo.mim.SpeakerIds} results.speakerIds If available, the ID of the speaker.
|
|
* @param {jibo.mim.MimState} results.state Current state of the MIM.
|
|
* @param {String} results.exception Exception for this failure; either `~InteractionError.noMatch` or `~InteractionError.noInput`.
|
|
*/
|
|
|
|
/**
|
|
* (Internal to Menu behavior only) Called to confirm that a positional selection makes sense, allowing custom handling to select certain types of items, if desired.
|
|
* @callback jibo.bt.behaviors.Mim~OnPositionalSelect
|
|
* @param {jibo.jetstream.types.ListenResult} commandASR Data from NLU service.
|
|
* @param {Number} intendedIndex Index that the menu plans to use for selection.
|
|
* @param {jibo.face.views.MenuView} menu Created menu. Use to read button data, if you didn't keep track of it yourself.
|
|
* @returns {number} Return an item index to change what will be selected, or null to select nothing. Returning undefined will use the default behavior (selecting intendedIndex).
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* @class Mim
|
|
* @extends jibo.bt.Behavior
|
|
* @memberof jibo.bt.behaviors
|
|
* @description Perform a dialogue interaction. Can be an announcement, an
|
|
* announcement with optional user response, or a
|
|
* question and attempt to get answer from the user.
|
|
* @param {Object} options See {@link jibo.bt.Behavior|Behavior} for all options.
|
|
* @param {String} options.mimPath Path to the MIM file to use.
|
|
* @param {jibo.bt.behaviors.Mim~GetPromptData} options.getPromptData Returns the prompt variable data.
|
|
* @param {jibo.bt.behaviors.Mim~CheckResult} options.checkResult Called and passed data about what the Mim is hearing when a listen is complete. Optional Response and Question MIMs only.
|
|
* @param {jibo.bt.behaviors.Mim~OnSuccess} options.onSuccess Called and passed data about what the Mim heard when it completes. Optional Response and Question MIMs only.
|
|
* @param {jibo.bt.behaviors.Mim~OnFailure} options.onFailure Called and passed data about what the Mim heard or did not hear when it completes. Question MIMs only.
|
|
*/
|
|
|
|
/**
|
|
* Stops the MIM. The optional parameter here is considered private - this should be used
|
|
* like the stop() method it inherits from Behavior, with no parameters.
|
|
* @method jibo.bt.Mim#stop
|
|
* @param {boolean} [isMenuTap] If the stop() was due to tapping a menu button.
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* If this Mim has an active view, pops the top view from the list and destroys this Mim's
|
|
* active view.
|
|
* @method jibo.bt.Mim#removeView
|
|
* @private
|
|
* @param {string} intent Reason for removing the view - 'success', 'failure', or null if view should never be left blank.
|
|
*/
|
|
|
|
/**
|
|
* Remove combinedRuleHandle and ruleHandle from NLU memory.
|
|
* @method jibo.bt.Mim#unloadRules
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Cleans up global event listeners that the MIM added.
|
|
* @method jibo.bt.Mim#cleanUpEvents
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Opens the MIM's GUI if it is not already open.
|
|
* @method jibo.bt.Mim#openGUI
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Callback for events from `jibo.mim.openGUI`. Opens the GUI, if it is not already open.
|
|
* @method jibo.bt.Mim#onOpenGUIEvent
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Callback for events from `jibo.mim.handleSpeech`. Stops active speech or listen to parse a
|
|
* spoofed utterance.
|
|
* @method jibo.bt.Mim#onSpeechEvent
|
|
* @param {string} utterance The spoofed utterance in need of parsing.
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Callback for events from `jibo.mim.end`. Ends the MIM immediately.
|
|
* @method jibo.bt.Mim#onEndEvent
|
|
* @param {any} data Not currently used. In the future, data about how the MIM should end.
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Callback for events from `jibo.mim.heyJibo`. Handles barge in via "Hey Jibo".
|
|
* @method jibo.bt.Mim#onHeyJiboHeard
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.loadConfig. Load .mim file(s) and then go to states.loadRule for OR/Q
|
|
* mims or states.choosePrompt for AN mims.
|
|
* @method jibo.bt.Mim#loadMim
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.loadRule. Load .mim file(s) and then go to states.loadRule for OR/Q
|
|
* mims or states.choosePrompt for AN mims. Sets up loading of rule for mim and unionizing it
|
|
* with mim global rule as a Promise. Immediately go to states.choosePrompt.
|
|
* @method jibo.bt.Mim#loadRule
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.choosePrompt. Choose prompt text to be played. Go to states.listen
|
|
* if no prompt or silent prompt chosen. Go to states.speak to speak the prompt.
|
|
* @method jibo.bt.Mim#choosePrompt
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.speak. Speaks the chosen prompt through the TTS system. When finished,
|
|
* goes to states.reportResults for an AN mim or states.listen for an OR/Q mim.
|
|
* @method jibo.bt.Mim#speak
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State interuption for states.speak. Stops jibo.mim#speakDelegate playback.
|
|
* @method jibo.bt.Mim#stopSpeaking
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.listen. Starts a listener from jibo.mim#listenDelegate.
|
|
* If appropriate, starts timeout. If appropriate, displays the mim GUI. When listen completes,
|
|
* transitions to states.analyzeResults for mim results or states.analyzeMimGlobal for mim
|
|
* global commands.
|
|
* @method jibo.bt.Mim#startListen
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Callback for screen interaction. Resets timeouts during listen.
|
|
* @method jibo.bt.Mim#resetTimeout
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State interuption for states.listen. Stops listener and removes mim GUI if present.
|
|
* @method jibo.bt.Mim#stopListen
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State exit for states.listen. Stops listener and removes screen touch listeners.
|
|
* @method jibo.bt.Mim#listenExit
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State update for states.listen. Tracks timeout. If timeout triggered, change state to
|
|
* states.analyzeResults.
|
|
* @method jibo.bt.Mim#updateListen
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.analyzeMimGlobal. If global command is 'repeat', go to
|
|
* states.choosePrompt. Otherwise, redirect to states.analyzeResults.
|
|
* @method jibo.bt.Mim#analyzeMimGlobal
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.analyzeMenuGlobal. If global command is 'left' or right' and GUI is
|
|
* active, scroll list and return to states.listen. Otherwise, redirect to
|
|
* states.analyzeResults.
|
|
* @method jibo.bt.Mim#analyzeMenuGlobal
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.parseSpoof. Perform NLU parse on utterance text from GUI, followed * by going to states.reportResults.
|
|
* @method jibo.bt.Mim#parseSpoofedUtterance
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.analyzeResults. Analyze results to determine if result of listen is
|
|
* 'noMatch', 'noInput', or 'match'. If result is 'match' or mim has run out of error prompts,
|
|
* go to states.reportResults. Otherwise, go to states.choosePrompt.
|
|
* @method jibo.bt.Mim#analyzeResults
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.reportResults. If mim has failed due to noInput or noMatch errors,
|
|
* call onFailure to allow flow to handle exception. If exception not handled, go to
|
|
* states.handleException. If mim successful, call onSuccess and end Mim behavior.
|
|
* @method jibo.bt.Mim#reportResults
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.handleException. Begins global mim exception flow for either 'noInput'
|
|
* or 'noMatch' failures.
|
|
* @method jibo.bt.Mim#handleException
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State update for states.handleException. Update exception flow. If flow is complete act upon
|
|
* flow result. If flow result is 'exit', exit skill and return to be/idle. If flow result is
|
|
* 'restart', go to states.choosePrompt.
|
|
* @method jibo.bt.Mim#updateException
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State interuption for states.handleException. Stop and clean up exception flow. Remove any
|
|
* flow created view.
|
|
* @method jibo.bt.Mim#stopException
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Display text that mim rules could not match. Provided to noMatch exception flow as a
|
|
* callback.
|
|
* @method jibo.bt.Mim#showInputText
|
|
* @param {string} notMatchedText Text from speech to text that NLU could not match.
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Remove text display created during noMatch exception flow.
|
|
* @method jibo.bt.Mim#hideInputText
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State entry for states.waitForHJEnd. Begins waiting for the Hey Jibo listen to end in a way
|
|
* that the MIM should recover from.
|
|
* @method jibo.bt.Mim#waitForHJEnd
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* Standard method to tell the waitForHJEnd state to go back to prompt selection.
|
|
* @method jibo.bt.Mim#recoverFromHJ
|
|
* @private
|
|
*/
|
|
|
|
/**
|
|
* State interuption and exiting for states.waitForHJEnd. Removesevent listeners added in the
|
|
* state entry.
|
|
* @method jibo.bt.Mim#hjEnded
|
|
* @private
|
|
*/ |