talking works & menus work plus ai documentation : )

V3.1/build/hub-shim/HUB-SHIM_DEPLOYMENT.md

@@ -0,0 +1,57 @@
+# Jibo Hub-Shim Server: Deployment & Usage Guide
+
+## Quick Start
+1. **Install dependencies** (if needed):
+   - Node.js v16+ (recommended v18+)
+   - `npm install` (if package.json present)
+2. **Configure**:
+   - Edit `config.json` (or use `config.example.json`):
+     - `listen.port`: Port to listen on (default: 9000)
+     - `listen.path`: WebSocket path (default: `/v1/listen`)
+     - `asr.baseUrl`: Local ASR HTTP endpoint
+     - `logging.level`: `info`, `debug`, etc.
+3. **Run the server**:
+   ```sh
+   node index.js config.json
+   ```
+   - Or set `JIBO_HUB_SHIM_CONFIG` env var.
+
+## Integration with Jibo Robot
+- The robot's `jibo-jetstream-service` binary must be configured to connect to the hub-shim:
+  - `hub_port`: 9000 (or your chosen port)
+  - `hub_hostname`: IP of the machine running hub-shim
+  - `listen_url`: `/v1/listen`
+- Confirm with logs: hub-shim should show `ws connected` and `listen result` for each turn.
+
+## How It Works
+- The robot opens a WebSocket to hub-shim for every listen turn.
+- hub-shim receives CONTEXT and LISTEN messages, runs ASR/NLU, and sends a TURN_RESULT.
+- The TURN_RESULT is always local (`global: false`), with `status: 'SUCCEEDED'` and the correct intent/entities.
+- This resolves the skill's local turn and allows menu/intro flows to advance by voice.
+
+## Troubleshooting
+- **No skill advancement?**
+  - Check that TURN_RESULT is sent with `status: 'SUCCEEDED'`, `global: false`, and correct `requestID` (should match transID).
+  - Use robot-logger (`/tmp/jibo-skills.log` or web panel) to trace intent and state.
+- **ASR timeouts?**
+  - Ensure ASR service is running and reachable from hub-shim.
+- **NLU not matching?**
+  - Edit `inferNluFromText()` in `index.js` to adjust rule patterns.
+
+## Advanced
+- **Custom NLU**: You can extend `inferNluFromText()` for more complex intent/entity extraction.
+- **Client-supplied ASR/NLU**: Send CLIENT_ASR or CLIENT_NLU messages before LISTEN to inject results.
+- **Logging**: Adjust `logging.level` in config for more/less verbosity.
+
+## File Locations
+- Main server: `index.js`
+- Config: `config.json` or `config.example.json`
+- Protocol doc: `HUB-SHIM_PROTOCOL.md`
+
+## Support
+- For protocol or integration issues, see HUB-SHIM_PROTOCOL.md and the source code comments.
+- For robot-side debugging, use robot-logger and check `/tmp/jibo-skills.log`.
+
+---
+
+# Enjoy fully offline Jibo skills and menus!

V3.1/build/hub-shim/HUB-SHIM_PROTOCOL.md

@@ -0,0 +1,66 @@
+# Jibo Hub-Shim Server: Architecture & Protocol
+
+## Overview
+The hub-shim server emulates the Jibo cloud hub's `/v1/listen` WebSocket protocol, allowing the robot to run skills and menu flows entirely offline. It acts as a bridge between the robot's jetstream service (binary) and local ASR/NLU logic, providing the expected protocol and responses for seamless skill operation.
+
+## Key Features
+- Listens on a configurable port (default: 9000) and path (`/v1/listen`)
+- Accepts WebSocket connections from the robot's jetstream service
+- Handles CONTEXT, LISTEN, CLIENT_ASR, and CLIENT_NLU messages
+- Runs local ASR (via HTTP) and NLU (via rule-based inference)
+- Sends TURN_RESULT responses that resolve the robot's local turn promise
+- Fully mimics the cloud hub protocol, including transID/requestID echo
+
+## Protocol Flow
+1. **Connection**: Jetstream binary opens a WebSocket to `/v1/listen`.
+2. **CONTEXT**: Robot sends context (skill, runtime info).
+3. **LISTEN**: Robot requests a listen turn, providing rules and mode.
+4. **ASR**: hub-shim runs local ASR (or accepts client-supplied text).
+5. **NLU**: hub-shim infers intent/entities from text and rules.
+6. **TURN_RESULT**: hub-shim sends a TURN_RESULT with status `SUCCEEDED`, echoing the transID as requestID, and including asr/nlu/match objects.
+7. **Skill Advances**: The robot's local turn promise resolves, and the skill/menu flow continues.
+
+## Message Types
+- **CONTEXT**: Sets skill/runtime context for the turn.
+- **LISTEN**: Initiates a listen turn; includes rules and mode.
+- **CLIENT_ASR/CLIENT_NLU**: (Optional) Supplies ASR text or NLU result directly.
+- **TURN_RESULT**: Final result; must have `status: 'SUCCEEDED'`, `global: false`, and correct asr/nlu/match.
+
+## Key Implementation Details
+- **requestID**: Always set to the incoming transID for local turns.
+- **asr/nlu/match**: Structured to match the robot's expected ListenResult format.
+- **No global-only results**: All results are sent as local TURN_RESULTs to resolve the skill's local turn.
+- **Skill-specific NLU**: Rule-based inference for menu/intro/yes-no flows.
+
+## Configuration
+- `config.json` controls port, ASR service URL, and logging.
+- ASR service must be running and reachable by hub-shim.
+
+## Debugging
+- Logs all connections, requests, and results.
+- Use robot-logger on the robot to trace skill flow and intent resolution.
+
+## Example TURN_RESULT
+```
+{
+  "type": "TURN_RESULT",
+  "msgID": "...",
+  "transID": "...",
+  "ts": 1234567890,
+  "requestID": "...", // matches transID
+  "data": {
+    "status": "SUCCEEDED",
+    "global": false,
+    "result": {
+      "asr": { "text": "face", "confidence": 1 },
+      "nlu": { "intent": "face", ... },
+      "match": { "onRobot": true }
+    }
+  },
+  "final": true
+}
+```
+
+---
+
+# See also: hub-shim source code for full protocol details.