67 lines
2.8 KiB
Markdown
67 lines
2.8 KiB
Markdown
|
# Jibo Hub-Shim Server: Architecture & Protocol
|
||
|
|
|
||
|
|
## Overview
|
||
|
|
The hub-shim server emulates the Jibo cloud hub's `/v1/listen` WebSocket protocol, allowing the robot to run skills and menu flows entirely offline. It acts as a bridge between the robot's jetstream service (binary) and local ASR/NLU logic, providing the expected protocol and responses for seamless skill operation.
|
||
|
|
|
||
|
|
## Key Features
|
||
|
|
- Listens on a configurable port (default: 9000) and path (`/v1/listen`)
|
||
|
|
- Accepts WebSocket connections from the robot's jetstream service
|
||
|
|
- Handles CONTEXT, LISTEN, CLIENT_ASR, and CLIENT_NLU messages
|
||
|
|
- Runs local ASR (via HTTP) and NLU (via rule-based inference)
|
||
|
|
- Sends TURN_RESULT responses that resolve the robot's local turn promise
|
||
|
|
- Fully mimics the cloud hub protocol, including transID/requestID echo
|
||
|
|
|
||
|
|
## Protocol Flow
|
||
|
|
1. **Connection**: Jetstream binary opens a WebSocket to `/v1/listen`.
|
||
|
|
2. **CONTEXT**: Robot sends context (skill, runtime info).
|
||
|
|
3. **LISTEN**: Robot requests a listen turn, providing rules and mode.
|
||
|
|
4. **ASR**: hub-shim runs local ASR (or accepts client-supplied text).
|
||
|
|
5. **NLU**: hub-shim infers intent/entities from text and rules.
|
||
|
|
6. **TURN_RESULT**: hub-shim sends a TURN_RESULT with status `SUCCEEDED`, echoing the transID as requestID, and including asr/nlu/match objects.
|
||
|
|
7. **Skill Advances**: The robot's local turn promise resolves, and the skill/menu flow continues.
|
||
|
|
|
||
|
|
## Message Types
|
||
|
|
- **CONTEXT**: Sets skill/runtime context for the turn.
|
||
|
|
- **LISTEN**: Initiates a listen turn; includes rules and mode.
|
||
|
|
- **CLIENT_ASR/CLIENT_NLU**: (Optional) Supplies ASR text or NLU result directly.
|
||
|
|
- **TURN_RESULT**: Final result; must have `status: 'SUCCEEDED'`, `global: false`, and correct asr/nlu/match.
|
||
|
|
|
||
|
|
## Key Implementation Details
|
||
|
|
- **requestID**: Always set to the incoming transID for local turns.
|
||
|
|
- **asr/nlu/match**: Structured to match the robot's expected ListenResult format.
|
||
|
|
- **No global-only results**: All results are sent as local TURN_RESULTs to resolve the skill's local turn.
|
||
|
|
- **Skill-specific NLU**: Rule-based inference for menu/intro/yes-no flows.
|
||
|
|
|
||
|
|
## Configuration
|
||
|
|
- `config.json` controls port, ASR service URL, and logging.
|
||
|
|
- ASR service must be running and reachable by hub-shim.
|
||
|
|
|
||
|
|
## Debugging
|
||
|
|
- Logs all connections, requests, and results.
|
||
|
|
- Use robot-logger on the robot to trace skill flow and intent resolution.
|
||
|
|
|
||
|
|
## Example TURN_RESULT
|
||
|
|
```
|
||
|
|
{
|
||
|
|
"type": "TURN_RESULT",
|
||
|
|
"msgID": "...",
|
||
|
|
"transID": "...",
|
||
|
|
"ts": 1234567890,
|
||
|
|
"requestID": "...", // matches transID
|
||
|
|
"data": {
|
||
|
|
"status": "SUCCEEDED",
|
||
|
|
"global": false,
|
||
|
|
"result": {
|
||
|
|
"asr": { "text": "face", "confidence": 1 },
|
||
|
|
"nlu": { "intent": "face", ... },
|
||
|
|
"match": { "onRobot": true }
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"final": true
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# See also: hub-shim source code for full protocol details.
|