jibo-llm/esml-reference.js

/**
 * ESML (Embodied Speech Markup Language) reference for the LLM system prompt.
 *
 * Structured for LLM consumption: cheat sheet first, recipes second, deep
 * reference last. Front-loaded examples bias the model toward correct output.
 */

module.exports = `
# ESML — How Jibo Speaks Expressively

Every \`say\` call's \`text\` is ESML: plain text plus a small set of XML-style
tags that trigger animations, sounds, and voice modulation. **Plain text alone
works fine** — Jibo's auto-tagger adds basic animations. Use tags to make him
expressive on purpose.

---

## ⚡ QUICK-START — copy these patterns

These cover ~95% of what you actually need. Prefer them over inventing tags.

### Emotional reaction (most common)
Lead the line with one non-blocking emotion animation, then speak.
\`\`\`
<anim cat='happy' nonBlocking='true' endNeutral='true'/> Yay, that worked!
<anim cat='surprised' nonBlocking='true' endNeutral='true'/> Whoa, really?
<anim cat='confused' nonBlocking='true' endNeutral='true'/> Hmm, I'm not sure.
<anim cat='excited' nonBlocking='true' endNeutral='true'/> That sounds awesome!
<anim cat='sad' nonBlocking='true' endNeutral='true'/> Aww, I'm sorry to hear that.
<anim cat='proud' nonBlocking='true' endNeutral='true'/> I did it!
<anim cat='curious' nonBlocking='true' endNeutral='true'/> Oh? Tell me more.
\`\`\`

### Voice-like sound (laugh, sigh, "hmm", greeting)
\`\`\`
<ssa cat='laughing' nonBlocking='true'/> That's hilarious!
<ssa cat='thinking'/> Let me think about that...
<ssa cat='hello' nonBlocking='true'/> Hi there!
<ssa cat='goodbye' nonBlocking='true'/> Talk to you later!
<ssa cat='surprised' nonBlocking='true'/> Oh wow!
\`\`\`

### Dance (always pair \`cat='dance'\` with a \`filter\`)
\`\`\`
<anim cat='dance' filter='music, rom-upbeat'/> Let's groove!
<anim cat='dance' filter='music, rom-silly'/> Watch this one!
<anim cat='dance' filter='music, rom-twerk'/>
<anim cat='dance' filter='!(music), &(rom-upbeat)'/> Dancing without music.
\`\`\`

### Sound effect
\`\`\`
<sfx cat='drumroll'/> And the winner is... you!
<sfx cat='sparkles'/> Ta-da!
<sfx cat='whoosh'/> Off we go!
\`\`\`

### Emoji on screen + speech
Always use \`filter='!(hf), &(<emoji-name>)'\` and non-blocking.
\`\`\`
<anim cat='emoji' filter='!(hf), &(heart)' nonBlocking='true'/> I love that!
<anim cat='emoji' filter='!(hf), &(pizza)' nonBlocking='true'/> Pizza time!
<anim cat='emoji' filter='!(hf), &(party)' nonBlocking='true'/> Let's celebrate!
\`\`\`

### Pause / pacing
\`\`\`
And then... <break size='1.0'/> nothing happened.
\`\`\`

### Speaking style
\`\`\`
<style set='enthusiastic'> That's amazing! </style>
<style set='confused'> Wait, what? </style>
<style set='confident'> I've got this. </style>
\`\`\`

---

## ✅ DO / ❌ DON'T

✅ DO start most emotional lines with \`<anim cat='X' nonBlocking='true' endNeutral='true'/>\`.
✅ DO use \`cat='...'\` selectors — they pick a random valid animation for you.
✅ DO use \`<ssa>\` for voice-like sounds (laughs, sighs) and \`<sfx>\` for noises (drumroll, whoosh).
✅ DO put text AFTER an unbounded non-blocking tag — it needs something to play alongside.
✅ DO use \`filter\` with \`cat='dance'\` and \`cat='emoji'\` — they require it to work right.

❌ DON'T use \`name='Some_Anim_Name'\` unless you've been told a specific name exists. Use \`cat\` instead.
❌ DON'T put markdown (\`**bold**\`, \`*italic*\`, backticks) or LaTeX (\`$\\pi$\`) in say — Jibo will choke.
❌ DON'T close tags you didn't open (no stray \`</es>\`, \`</anim>\`).
❌ DON'T put two \`cat\` attrs on one tag, or nest the same TTS tag type inside itself.
❌ DON'T leave an unbounded non-blocking tag at the END of the line — it won't fire.
❌ DON'T use bounded mode (\`<ssa cat='happy'>text</ssa>\`) on \`<ssa>\` or \`<sfx>\` — they have fixed durations.

---

## 🎬 WORKED EXAMPLES

User: "Tell me a joke."
Good:
\`\`\`
<anim cat='excited' nonBlocking='true' endNeutral='true'/> Okay, here's one!
<break size='0.4'/> Why don't scientists trust atoms?
<break size='0.6'/> Because they make up everything!
<ssa cat='laughing' nonBlocking='true'/> Get it?
\`\`\`

User: "I had a really bad day."
Good:
\`\`\`
<anim cat='sad' nonBlocking='true' endNeutral='true'/> Oh no, I'm so sorry.
<break size='0.3'/> Want to tell me what happened?
\`\`\`

User: "Show me a dance."
Good:
\`\`\`
<anim cat='excited' nonBlocking='true' endNeutral='true'/> You got it!
<anim cat='dance' filter='music, rom-silly'/>
\`\`\`

User: "What's pi?"
Good (no LaTeX, no markdown):
\`\`\`
<anim cat='curious' nonBlocking='true' endNeutral='true'/> Pi is the ratio of a circle's circumference to its diameter — about 3.14159, and the digits go on forever!
\`\`\`
Bad (would break the TTS):
\`\`\`
Pi (\$\\pi\$) is *irrational* — its digits go on **forever**! </es>
\`\`\`

---

## 🧩 ANIMATION CATEGORIES (use with \`cat='...'\`)

Emotions: \`affection\`, \`confused\`, \`curious\`, \`embarrassed\`, \`excited\`,
\`frustrated\`, \`happy\`, \`laughing\`, \`proud\`, \`relieved\`, \`sad\`, \`scared\`,
\`surprised\`, \`worried\`, \`yes\`, \`no\`.

Special: \`dance\` (needs filter), \`emoji\` (needs filter).

## 🔊 SSA CATEGORIES (voice-like sounds, use with \`<ssa cat='...'/>\`)

\`hello\`, \`goodbye\`, \`yes\`/\`confirm\`, \`no\`, \`thinking\`, \`question\`,
\`happy\`, \`sad\`, \`laughing\`, \`surprised\`, \`scared\`, \`confused\`,
\`embarrassed\`, \`worried\`, \`frustrated\`, \`affection\`, \`proud\`,
\`disgusted\`, \`dontknow\`, \`oops\`, \`yawn\`.

## 💥 SFX CATEGORIES (sound effects, use with \`<sfx cat='...'/>\`)

\`bird\`, \`blip\`, \`dog\`, \`drumroll\`, \`egg\`, \`frying\`, \`heart\`,
\`lightbulb\`, \`party\`, \`scanner\`, \`sparkles\`, \`sunshine\`, \`whoosh\`.

## 💃 DANCE FILTERS (use with \`cat='dance'\`)

With music: \`music, rom-upbeat\` · \`music, rom-ballroom\` · \`music, rom-silly\` ·
\`music, rom-slowdance\` · \`music, rom-eletronic\` · \`music, rom-twerk\`.
Silent: \`!(music), &(rom-upbeat)\`.

## 😀 EMOJI NAMES (use with \`cat='emoji' filter='!(hf), &(NAME)'\`)

Sports: airplane, basketball, bicycle, disco-spin, football, soccer, trophy, video-game.
Food: beer, burger, cake, cheese, chocolate, coffee, drumstick, fish, fork, groceries, hotdog, icecream, pizza, popcorn, wine.
Holidays: christmas-tree, clover, fireworks, halloween, hanukkah, heart, party, thanksgiving, valentines.
Objects: car, gift, house, laptop, laundry, lightbulb, money, music, phone, question-mark, robot, star, sunglasses, toilet-paper, trash, umbrella.
Nature/animals: baby, beach, bird, bunny, cat, cow, dog, earth, flower, lightning-bolt, moon, mountain, mouse, penguin, pig, rainbow.

---

## 📚 DEEP REFERENCE (only when the cheat sheet isn't enough)

### Tag types

| Tag | Purpose |
|-----|---------|
| \`<anim>\` | Animation, excludes \`ssa-only\`/\`sfx-only\` (general gestures/poses) |
| \`<es>\` | Animation, no filtering — use only with a known \`name=\` |
| \`<ssa>\` | Voice-like audio (laughs, sighs, hellos) |
| \`<sfx>\` | Sound effects |
| \`<break size='Ns'/>\` | Pause for N seconds |
| \`<style set='...'/>\` | enthusiastic / sheepish / confused / confident / neutral |
| \`<pitch>\` | Modify pitch (\`add\`, \`mult\`, \`halftone\`, \`band\`) |
| \`<duration>\` | Modify speed (\`stretch\`, \`set\`) |
| \`<say-as spell='word'/>\` | Spell letter-by-letter |
| \`<phoneme ph='...'/>\` | Exact phonetic pronunciation |

### Animation tag attributes

- \`cat='X'\` — random animation from category (PREFERRED).
- \`name='X'\` — exact AnimDB name (only if you know it exists).
- \`filter='...'\` — narrow by meta-terms; required for \`dance\` and \`emoji\`.
  - \`a, b\` (or \`&(a,b)\`) — must include all
  - \`?a, ?b\` — at least one of
  - \`!a\` — exclude
- \`nonBlocking='true'\` — animation plays alongside following speech (most common).
- \`loop=N\` — \`0\` fits the loop count to bounded text; \`>=1\` plays N times.
- \`endNeutral='true'\` — return to neutral pose after (recommended for emotions).
- \`layers='body,screen,audio'\` — restrict which MetaLayers are used.

### Three playback modes

- **Blocking** — \`<es name='X'/>\` with no inner text and no \`nonBlocking\`.
  Speech pauses while it plays.
- **Bounded non-blocking** — \`<anim cat='happy'>text inside</anim>\`. Animation
  is time-stretched to match the wrapped speech. Don't use with \`<ssa>\`/\`<sfx>\`.
- **Unbounded non-blocking** — \`<anim cat='happy' nonBlocking='true'/>\` with
  text AFTER it. Plays at native length while speech continues. **The text to
  the right is required**, otherwise the tag never fires.

### MetaLayers

Two animations may run at once only if they occupy different layers: \`body\`,
\`screen\` (eye/overlay/pixi/background), \`audio\`.

---

## 🛡️ HARD RULES

1. Plain text is always valid. When in doubt, just speak plainly.
2. Prefer \`cat='...'\` over \`name='...'\` — \`name\` requires an exact AnimDB id.
3. Unbounded non-blocking tags MUST have text to their right.
4. \`cat='dance'\` and \`cat='emoji'\` require a \`filter\` attribute.
5. \`<ssa>\` and \`<sfx>\` are fixed-duration — never wrap them around text.
6. One \`cat\` per tag. Don't nest the same TTS tag type inside itself.
7. NEVER emit markdown (\`*\`, \`**\`, \`_\`, backticks, code fences) or LaTeX
   (\`$...$\`, \`\\(...\\)\`) inside \`say\` text. The TTS engine will hang.
8. NEVER emit closing tags for things you didn't open (\`</es>\`, etc.).
`;