Stub in framework for new .net Open Jibo cloud

This commit is contained in:
Jacob Dubin
2026-04-11 07:12:57 -05:00
parent 0c040d1348
commit 8f838787a0
54 changed files with 1933 additions and 897 deletions

View File

@@ -1,204 +1,65 @@
# Jibo.Cloud.DotNet
## Overview
## Summary
**Jibo.Cloud.DotNet** is the long-term, production-focused implementation of the OpenJibo cloud layer.
`Jibo.Cloud.DotNet` is the stable hosted implementation of the OpenJibo cloud.
While the Node.js implementation was used to rapidly explore and validate Jibos cloud behavior, this project represents the future direction: a clean, maintainable, and scalable cloud platform built on **C# and .NET**.
This is the production-oriented path for restoring device connectivity and creating a foundation for future runtime, AI, and OTA work.
This is where the OpenJibo cloud becomes real.
## Architecture
---
The first implementation is a modular monolith:
## Vision
The goal of this project is not just to replicate the original Jibo cloud, but to build something better:
* A stable and secure cloud platform for Jibo devices
* A bridge between the physical robot and modern AI-driven systems
* A foundation for new capabilities beyond what Jibo originally supported
This is the backbone of the OpenJibo ecosystem.
---
## Design Principles
### 1. Clean Architecture
This implementation is designed around clear separation of concerns:
* Transport (HTTP, WebSocket)
* Application logic (routing, orchestration)
* Domain models (robot, session, capabilities)
* Integration layers (AI, storage, external services)
---
### 2. Compatibility First
The system will:
* Emulate required Jibo cloud endpoints
* Support existing device expectations
* Preserve OTA update compatibility
This ensures existing devices can reconnect without invasive changes.
---
### 3. Extensibility
The platform is being designed to support:
* New skills and capabilities
* AI-driven conversation and planning
* External integrations (APIs, services, tools)
* Multi-agent orchestration (future CoffeeBreak integration)
---
### 4. Cloud-Native Deployment
The target deployment model includes:
* Azure-hosted services
* Real domains (`openjibo.com`, `openjibo.ai`)
* Proper TLS / certificate chains
* Scalable service architecture
---
## Role in the OpenJibo Architecture
Jibo.Cloud sits between the robot and the runtime:
```plaintext id="l3tq9n"
Jibo Device
Jibo.Cloud (this project)
OpenJibo Runtime (.NET)
Capabilities / AI / Services
```text
Api -> Application -> Domain -> Infrastructure
```
Responsibilities include:
This keeps deployment simple while preserving clean boundaries.
* Handling device communication (HTTPS + WebSockets)
* Managing identity, sessions, and tokens
* Routing requests to runtime services
* Delivering OTA updates
* Acting as the central coordination layer
## Azure Direction
---
The target Azure footprint is:
## OTA Update Strategy
- Azure App Service for HTTP and WebSocket traffic
- Azure SQL for relational persistence
- Azure Blob Storage for uploads and update artifacts
- Azure Key Vault for secrets and certificates
- Application Insights for observability
A key part of this implementation is full support for Jibos OTA update mechanism.
Azure SQL is the primary system of record for:
This enables:
- accounts
- devices
- sessions
- update metadata
- host mappings
- bootstrap and provisioning records
* Delivery of updated skills
* Deployment of new capabilities
* Gradual rollout of OpenJibo runtime components
* A path toward restoring devices without manual intervention
## Compatibility Goal
The goal is to make recovery and updates feel native to the device.
The first compatibility milestone is `core revive`.
---
That means the .NET cloud should handle:
## Planned Features
- token and session issuance
- account and robot identity flows needed for startup
- core `X-Amz-Target` dispatch
- listen and proactive WebSocket paths
- basic media and update metadata responses
- handoff into normalized `TurnContext` and `ResponsePlan` contracts
This project will evolve to support:
## Relationship To The Node Prototype
### Core Platform
The Node server remains the discovery harness and fixture source.
* HTTPS API endpoints compatible with Jibo
* WebSocket communication layer
* Authentication and token services
* Device and user management
The .NET implementation should:
### Runtime Integration
- copy observed behavior where needed
- use fixtures captured from Node and real robots
- avoid speculative protocol design
* Conversation orchestration
* Capability routing
* Response planning
* Integration with OpenJibo runtime abstractions
## Current State
### AI Integration
* Speech-to-text and text-to-speech
* Intent recognition and planning
* External AI providers (pluggable)
### Update System
* OTA update orchestration
* Skill delivery pipeline
* Versioning and rollout control
---
## Project Structure
This folder will contain one or more .NET projects, likely including:
```plaintext id="2qk0a7"
dotnet/
Jibo.Cloud.Api/ # HTTP + WebSocket endpoints
Jibo.Cloud.Application/ # Application logic and orchestration
Jibo.Cloud.Domain/ # Core models and contracts
Jibo.Cloud.Infrastructure/ # External integrations (storage, AI, etc.)
```
Structure may evolve as the system matures.
---
## Relationship to Node Implementation
The Node.js implementation remains valuable as:
* A reference for endpoint behavior
* A rapid testing environment
* A discovery tool for protocol details
However, this .NET implementation is the intended long-term solution.
---
## Status
This project is in early development.
* Core abstractions are being defined
* Endpoint behavior is being mapped from the Node implementation
* Initial service scaffolding is planned
---
## Contributing
If you're interested in building the future of Jibo, this is the place to do it.
Areas where help is especially valuable:
* API design and endpoint mapping
* WebSocket protocol handling
* OTA update workflows
* AI and conversation systems
* Cloud infrastructure and deployment
---
## Notes
This project is part of the OpenJibo initiative and is not affiliated with the original Jibo company.
The mission is simple:
Bring Jibo back for everyone, technical or not.
Make him what he was meant to be.
Then we make him better.
This folder now contains the first hosted scaffold, not just a README.
The intent is to grow from a runnable dev monolith into the real Azure deployment target without abandoning the existing abstractions work.

View File

@@ -0,0 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<ItemGroup>
<ProjectReference Include="..\Jibo.Cloud.Application\Jibo.Cloud.Application.csproj" />
<ProjectReference Include="..\Jibo.Cloud.Infrastructure\Jibo.Cloud.Infrastructure.csproj" />
<ProjectReference Include="..\..\..\..\Jibo.Runtime.Abstractions\Jibo.Runtime.Abstractions.csproj" />
</ItemGroup>
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
</Project>

View File

@@ -0,0 +1,170 @@
using System.Net.WebSockets;
using System.Text;
using Jibo.Cloud.Application.Services;
using Jibo.Cloud.Domain.Models;
using Jibo.Cloud.Infrastructure.DependencyInjection;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddOpenJiboCloud();
var app = builder.Build();
app.UseWebSockets();
app.Use(async (context, next) =>
{
if (!context.WebSockets.IsWebSocketRequest)
{
await next();
return;
}
var webSocketService = context.RequestServices.GetRequiredService<JiboWebSocketService>();
using var socket = await context.WebSockets.AcceptWebSocketAsync();
var kind = ResolveSocketKind(context.Request.Host.Host, context.Request.Path);
var token = ResolveToken(context.Request);
while (socket.State == WebSocketState.Open)
{
var received = await ReceiveAsync(socket, context.RequestAborted);
if (received.MessageType == WebSocketMessageType.Close)
{
await socket.CloseAsync(WebSocketCloseStatus.NormalClosure, "bye", context.RequestAborted);
break;
}
var envelope = new WebSocketMessageEnvelope
{
ConnectionId = Guid.NewGuid().ToString("N"),
HostName = context.Request.Host.Host,
Path = context.Request.Path.Value ?? "/",
Kind = kind,
Token = token,
Text = received.MessageType == WebSocketMessageType.Text ? Encoding.UTF8.GetString(received.Buffer) : null,
Binary = received.MessageType == WebSocketMessageType.Binary ? received.Buffer : null
};
var replies = await webSocketService.HandleMessageAsync(envelope, context.RequestAborted);
foreach (var reply in replies)
{
if (string.IsNullOrWhiteSpace(reply.Text))
{
continue;
}
var payload = Encoding.UTF8.GetBytes(reply.Text);
await socket.SendAsync(payload, WebSocketMessageType.Text, true, context.RequestAborted);
}
}
});
app.MapGet("/health", () => Results.Json(new { ok = true, service = "OpenJibo Cloud Api" }));
app.MapMethods("/{**path}", ["GET", "POST", "PUT"], async (HttpContext context, JiboCloudProtocolService service, CancellationToken cancellationToken) =>
{
var envelope = await BuildEnvelopeAsync(context, cancellationToken);
var result = await service.DispatchAsync(envelope, cancellationToken);
context.Response.StatusCode = result.StatusCode;
context.Response.ContentType = result.ContentType;
foreach (var header in result.Headers)
{
context.Response.Headers[header.Key] = header.Value;
}
if (!string.IsNullOrEmpty(result.BodyText))
{
await context.Response.WriteAsync(result.BodyText, cancellationToken);
}
});
app.Run();
return;
static async Task<ReceivedSocketMessage> ReceiveAsync(WebSocket socket, CancellationToken cancellationToken)
{
var buffer = new byte[8192];
using var ms = new MemoryStream();
WebSocketReceiveResult result;
do
{
result = await socket.ReceiveAsync(buffer, cancellationToken);
ms.Write(buffer, 0, result.Count);
}
while (!result.EndOfMessage);
return new ReceivedSocketMessage(result.MessageType, ms.ToArray());
}
static async Task<ProtocolEnvelope> BuildEnvelopeAsync(HttpContext context, CancellationToken cancellationToken)
{
context.Request.EnableBuffering();
using var reader = new StreamReader(context.Request.Body, Encoding.UTF8, detectEncodingFromByteOrderMarks: false, leaveOpen: true);
var bodyText = await reader.ReadToEndAsync(cancellationToken);
context.Request.Body.Position = 0;
var target = context.Request.Headers["X-Amz-Target"].ToString();
var targetParts = target.Split('.', 2, StringSplitOptions.RemoveEmptyEntries);
return new ProtocolEnvelope
{
RequestId = Guid.NewGuid().ToString("N"),
Transport = "http",
Method = context.Request.Method,
HostName = context.Request.Host.Host,
Path = context.Request.Path.Value ?? "/",
ServicePrefix = targetParts.Length > 0 ? targetParts[0] : null,
Operation = targetParts.Length > 1 ? targetParts[1] : null,
DeviceId = context.Request.Headers["X-Jibo-RobotId"].ToString(),
CorrelationId = context.TraceIdentifier,
FirmwareVersion = context.Request.Headers["X-OpenJibo-Firmware"].ToString(),
ApplicationVersion = context.Request.Headers["X-OpenJibo-AppVersion"].ToString(),
BodyText = bodyText,
Headers = context.Request.Headers.ToDictionary(pair => pair.Key, pair => pair.Value.ToString(), StringComparer.OrdinalIgnoreCase)
};
}
static string ResolveSocketKind(string host, PathString path)
{
if (host.Equals("api-socket.jibo.com", StringComparison.OrdinalIgnoreCase))
{
return "api-socket";
}
if (host.Equals("neo-hub.jibo.com", StringComparison.OrdinalIgnoreCase) &&
path.StartsWithSegments("/v1/proactive"))
{
return "neo-hub-proactive";
}
if (host.Equals("neo-hub.jibo.com", StringComparison.OrdinalIgnoreCase))
{
return "neo-hub-listen";
}
return "openjibo";
}
static string? ResolveToken(HttpRequest request)
{
var auth = request.Headers.Authorization.ToString();
if (auth.StartsWith("Bearer ", StringComparison.OrdinalIgnoreCase))
{
return auth["Bearer ".Length..].Trim();
}
var path = request.Path.Value;
if (!string.IsNullOrWhiteSpace(path) && path.Length > 1)
{
return path.Trim('/');
}
return null;
}
internal sealed record ReceivedSocketMessage(WebSocketMessageType MessageType, byte[] Buffer);

View File

@@ -0,0 +1,12 @@
{
"profiles": {
"Jibo.Cloud.Api": {
"commandName": "Project",
"launchBrowser": true,
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"applicationUrl": "https://localhost:24604;http://localhost:24605"
}
}
}

View File

@@ -0,0 +1,17 @@
using Jibo.Cloud.Domain.Models;
namespace Jibo.Cloud.Application.Abstractions;
public interface ICloudStateStore
{
AccountProfile GetAccount();
DeviceRegistration GetRobot();
DeviceRegistration GetOrCreateDevice(string deviceId, string? firmwareVersion, string? applicationVersion);
string IssueHubToken();
string IssueRobotToken(string deviceId);
CloudSession OpenSession(string kind, string? deviceId, string? token, string? hostName, string? path);
CloudSession? FindSessionByToken(string token);
IReadOnlyList<UpdateManifest> ListUpdates(string? subsystem = null, string? filter = null);
UpdateManifest GetUpdateFrom(string? subsystem, string? fromVersion, string? filter);
void UpdateRobot(DeviceRegistration registration);
}

View File

@@ -0,0 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk">
<ItemGroup>
<ProjectReference Include="..\Jibo.Cloud.Domain\Jibo.Cloud.Domain.csproj" />
<ProjectReference Include="..\..\..\..\Jibo.Runtime.Abstractions\Jibo.Runtime.Abstractions.csproj" />
</ItemGroup>
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
</Project>

View File

@@ -0,0 +1,62 @@
using Jibo.Runtime.Abstractions;
namespace Jibo.Cloud.Application.Services;
public sealed class DemoConversationBroker : IConversationBroker
{
public Task<ResponsePlan> HandleTurnAsync(TurnContext turn, CancellationToken cancellationToken = default)
{
var transcript = (turn.NormalizedTranscript ?? turn.RawTranscript ?? string.Empty).Trim();
var lowered = transcript.ToLowerInvariant();
var reply = transcript.Length == 0
? "I am listening."
: lowered.Contains("time")
? $"It is {DateTime.Now:hh:mm tt}."
: lowered.Contains("hello") || lowered.Contains("hi")
? "Hello from the OpenJibo cloud."
: lowered.Contains("joke")
? "Why did the robot bring a ladder? Because it wanted to reach the cloud."
: $"I heard: {transcript}";
var plan = new ResponsePlan
{
SessionId = turn.SessionId,
Status = ResponseStatus.Succeeded,
IntentName = lowered.Contains("joke") ? "joke" : lowered.Contains("time") ? "time" : "chat",
Topic = "conversation",
DeviceId = turn.DeviceId,
TargetHost = turn.HostName,
DebugRoute = "demo-broker",
Actions =
{
new SpeakAction
{
Sequence = 0,
Text = reply,
Voice = "griffin"
},
new ListenAction
{
Sequence = 1,
Timeout = TimeSpan.FromSeconds(12),
Mode = "follow-up"
}
},
FollowUp = new FollowUpPolicy
{
KeepMicOpen = true,
Timeout = TimeSpan.FromSeconds(12),
ExpectedTopic = "conversation"
},
ProtocolMetadata = new Dictionary<string, object?>
{
["host"] = turn.HostName,
["service"] = turn.ProtocolService,
["operation"] = turn.ProtocolOperation
}
};
return Task.FromResult(plan);
}
}

View File

@@ -0,0 +1,251 @@
using System.Text.Json;
using Jibo.Cloud.Application.Abstractions;
using Jibo.Cloud.Domain.Models;
namespace Jibo.Cloud.Application.Services;
public sealed class JiboCloudProtocolService(ICloudStateStore stateStore)
{
private static readonly string[] AcceptedHosts =
[
"api.jibo.com",
"openjibo.com",
"openjibo.ai",
"localhost"
];
public Task<ProtocolDispatchResult> DispatchAsync(ProtocolEnvelope envelope, CancellationToken cancellationToken = default)
{
if (envelope.Method.Equals("GET", StringComparison.OrdinalIgnoreCase) &&
envelope.Path == "/" &&
string.IsNullOrWhiteSpace(envelope.ServicePrefix))
{
return Task.FromResult(ProtocolDispatchResult.NoContent());
}
if (envelope.Method.Equals("GET", StringComparison.OrdinalIgnoreCase) &&
envelope.Path.Equals("/health", StringComparison.OrdinalIgnoreCase))
{
return Task.FromResult(ProtocolDispatchResult.Ok(new { ok = true, host = envelope.HostName }));
}
if (envelope.Method.Equals("PUT", StringComparison.OrdinalIgnoreCase) &&
(envelope.Path.Equals("/upload/asr-binary", StringComparison.OrdinalIgnoreCase) ||
envelope.Path.Equals("/upload/log-events", StringComparison.OrdinalIgnoreCase) ||
envelope.Path.Equals("/upload/log-binary", StringComparison.OrdinalIgnoreCase)))
{
return Task.FromResult(ProtocolDispatchResult.Raw(200, string.Empty));
}
if (!AcceptedHosts.Contains(envelope.HostName, StringComparer.OrdinalIgnoreCase))
{
return Task.FromResult(ProtocolDispatchResult.Ok(new
{
ok = true,
accepted = false,
host = envelope.HostName
}));
}
var servicePrefix = envelope.ServicePrefix ?? string.Empty;
var operation = envelope.Operation ?? string.Empty;
if (servicePrefix.StartsWith("Account_", StringComparison.OrdinalIgnoreCase))
{
return Task.FromResult(HandleAccount(operation));
}
if (servicePrefix.StartsWith("Notification_", StringComparison.OrdinalIgnoreCase))
{
return Task.FromResult(HandleNotification(operation, envelope));
}
if (servicePrefix.StartsWith("Loop_", StringComparison.OrdinalIgnoreCase))
{
return Task.FromResult(HandleLoop(operation));
}
if (servicePrefix.StartsWith("Robot_", StringComparison.OrdinalIgnoreCase))
{
return Task.FromResult(HandleRobot(operation, envelope));
}
if (servicePrefix.StartsWith("Update_", StringComparison.OrdinalIgnoreCase))
{
return Task.FromResult(HandleUpdate(operation, envelope));
}
return Task.FromResult(ProtocolDispatchResult.Ok(new
{
ok = true,
service = servicePrefix,
operation
}));
}
private ProtocolDispatchResult HandleAccount(string operation)
{
var account = stateStore.GetAccount();
return operation switch
{
"CreateHubToken" => ProtocolDispatchResult.Ok(new
{
token = stateStore.IssueHubToken(),
expires = DateTimeOffset.UtcNow.AddHours(1).ToUnixTimeMilliseconds()
}),
"CreateAccessToken" => ProtocolDispatchResult.Ok(new
{
token = $"access-{account.AccountId}-{DateTimeOffset.UtcNow.ToUnixTimeMilliseconds()}",
expires = DateTimeOffset.UtcNow.AddHours(1).ToUnixTimeMilliseconds()
}),
"Get" => ProtocolDispatchResult.Ok(new[]
{
new
{
id = account.AccountId,
email = account.Email,
firstName = account.FirstName,
lastName = account.LastName,
accessKeyId = account.AccessKeyId,
secretAccessKey = account.SecretAccessKey
}
}),
_ => ProtocolDispatchResult.Ok(new
{
id = account.AccountId,
email = account.Email,
firstName = account.FirstName,
lastName = account.LastName
})
};
}
private ProtocolDispatchResult HandleNotification(string operation, ProtocolEnvelope envelope)
{
if (!operation.Equals("NewRobotToken", StringComparison.OrdinalIgnoreCase))
{
return ProtocolDispatchResult.Ok(new { ok = true, operation });
}
var body = envelope.TryParseBody();
var deviceId = envelope.DeviceId
?? ReadString(body, "deviceId")
?? ReadString(body, "serialNumber")
?? "unknown-device";
stateStore.GetOrCreateDevice(deviceId, envelope.FirmwareVersion, envelope.ApplicationVersion);
return ProtocolDispatchResult.Ok(new
{
token = stateStore.IssueRobotToken(deviceId)
});
}
private ProtocolDispatchResult HandleLoop(string operation)
{
if (operation is not ("List" or "ListLoops"))
{
return ProtocolDispatchResult.Ok(Array.Empty<object>());
}
var robot = stateStore.GetRobot();
var account = stateStore.GetAccount();
return ProtocolDispatchResult.Ok(new[]
{
new
{
id = "openjibo-default-loop",
name = "OpenJibo Default Loop",
owner = account.AccountId,
robot = robot.RobotId,
robotFriendlyId = robot.DeviceId,
members = Array.Empty<object>(),
isSuspended = false,
created = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(),
updated = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds()
}
});
}
private ProtocolDispatchResult HandleRobot(string operation, ProtocolEnvelope envelope)
{
var robot = stateStore.GetRobot();
if (operation.Equals("UpdateRobot", StringComparison.OrdinalIgnoreCase))
{
var updated = new DeviceRegistration
{
DeviceId = robot.DeviceId,
RobotId = robot.RobotId,
FriendlyName = robot.FriendlyName,
FirmwareVersion = envelope.FirmwareVersion ?? robot.FirmwareVersion,
ApplicationVersion = envelope.ApplicationVersion ?? robot.ApplicationVersion,
HostMappings = robot.HostMappings
};
stateStore.UpdateRobot(updated);
robot = updated;
}
return ProtocolDispatchResult.Ok(new
{
id = robot.RobotId,
friendlyId = robot.DeviceId,
name = robot.FriendlyName,
firmwareVersion = robot.FirmwareVersion,
applicationVersion = robot.ApplicationVersion
});
}
private ProtocolDispatchResult HandleUpdate(string operation, ProtocolEnvelope envelope)
{
var body = envelope.TryParseBody();
var subsystem = ReadString(body, "subsystem");
var filter = ReadString(body, "filter");
var fromVersion = ReadString(body, "fromVersion");
return operation switch
{
"ListUpdates" => ProtocolDispatchResult.Ok(stateStore.ListUpdates(subsystem, filter).Select(MapUpdate).ToArray()),
"ListUpdatesFrom" => ProtocolDispatchResult.Ok(stateStore.ListUpdates(subsystem, filter).Select(MapUpdate).ToArray()),
"GetUpdateFrom" => ProtocolDispatchResult.Ok(MapUpdate(stateStore.GetUpdateFrom(subsystem, fromVersion, filter))),
_ => ProtocolDispatchResult.Ok(Array.Empty<object>())
};
}
private static object MapUpdate(UpdateManifest update)
{
return new
{
_id = update.UpdateId,
created = update.CreatedUtc.ToUnixTimeMilliseconds(),
fromVersion = update.FromVersion,
toVersion = update.ToVersion,
changes = update.Changes,
url = update.Url,
shaHash = update.ShaHash,
length = update.Length,
subsystem = update.Subsystem,
filter = update.Filter
};
}
private static string? ReadString(JsonElement? element, string propertyName)
{
if (element is null)
{
return null;
}
if (!element.Value.TryGetProperty(propertyName, out var property))
{
return null;
}
return property.ValueKind == JsonValueKind.String
? property.GetString()
: property.ToString();
}
}

View File

@@ -0,0 +1,91 @@
using System.Text.Json;
using Jibo.Cloud.Application.Abstractions;
using Jibo.Cloud.Domain.Models;
using Jibo.Runtime.Abstractions;
namespace Jibo.Cloud.Application.Services;
public sealed class JiboWebSocketService(
ICloudStateStore stateStore,
ProtocolToTurnContextMapper turnContextMapper,
IConversationBroker conversationBroker,
ResponsePlanToSocketMessagesMapper replyMapper)
{
public async Task<IReadOnlyList<WebSocketReply>> HandleMessageAsync(WebSocketMessageEnvelope envelope, CancellationToken cancellationToken = default)
{
var session = stateStore.FindSessionByToken(envelope.Token ?? string.Empty) ??
stateStore.OpenSession(envelope.Kind, null, envelope.Token, envelope.HostName, envelope.Path);
if (envelope.IsBinary)
{
return
[
new WebSocketReply
{
Text = JsonSerializer.Serialize(new
{
type = "OPENJIBO_AUDIO_RECEIVED",
data = new
{
bytes = envelope.Binary?.Length ?? 0,
sessionId = session.SessionId
}
})
}
];
}
var parsedType = ReadMessageType(envelope.Text);
session.LastListenType = parsedType;
if (parsedType is "LISTEN" or "CLIENT_NLU" or "CLIENT_ASR")
{
var turn = turnContextMapper.MapListenMessage(envelope, session);
var plan = await conversationBroker.HandleTurnAsync(turn, cancellationToken);
return replyMapper.Map(plan).Select(text => new WebSocketReply
{
Text = text
}).ToArray();
}
return
[
new WebSocketReply
{
Text = JsonSerializer.Serialize(new
{
type = "OPENJIBO_ACK",
data = new
{
messageType = parsedType,
sessionId = session.SessionId
}
})
}
];
}
private static string ReadMessageType(string? text)
{
if (string.IsNullOrWhiteSpace(text))
{
return "UNKNOWN";
}
try
{
using var document = JsonDocument.Parse(text);
if (document.RootElement.TryGetProperty("type", out var type) && type.ValueKind == JsonValueKind.String)
{
return type.GetString() ?? "UNKNOWN";
}
}
catch
{
return "TEXT";
}
return "UNKNOWN";
}
}

View File

@@ -0,0 +1,63 @@
using System.Text.Json;
using Jibo.Cloud.Domain.Models;
using Jibo.Runtime.Abstractions;
namespace Jibo.Cloud.Application.Services;
public sealed class ProtocolToTurnContextMapper
{
public TurnContext MapListenMessage(WebSocketMessageEnvelope envelope, CloudSession session)
{
var text = ExtractTranscript(envelope.Text);
return new TurnContext
{
SessionId = session.SessionId,
InputMode = session.LastListenType == "follow-up" ? TurnInputMode.FollowUp : TurnInputMode.DirectText,
SourceKind = TurnSourceKind.Api,
RawTranscript = text,
NormalizedTranscript = text?.Trim(),
DeviceId = session.DeviceId,
HostName = envelope.HostName,
RequestId = envelope.ConnectionId,
ProtocolService = "neo-hub",
ProtocolOperation = "listen",
FirmwareVersion = session.Metadata.TryGetValue("firmwareVersion", out var firmwareVersion) ? firmwareVersion as string : null,
ApplicationVersion = session.Metadata.TryGetValue("applicationVersion", out var applicationVersion) ? applicationVersion as string : null,
IsFollowUpEligible = true
};
}
private static string? ExtractTranscript(string? text)
{
if (string.IsNullOrWhiteSpace(text))
{
return null;
}
try
{
using var document = JsonDocument.Parse(text);
var root = document.RootElement;
if (root.TryGetProperty("data", out var data))
{
if (data.TryGetProperty("text", out var transcript) && transcript.ValueKind == JsonValueKind.String)
{
return transcript.GetString();
}
if (data.TryGetProperty("intent", out var intent) && intent.ValueKind == JsonValueKind.String)
{
return intent.GetString();
}
}
}
catch
{
return text;
}
return text;
}
}

View File

@@ -0,0 +1,39 @@
using System.Text.Json;
using Jibo.Runtime.Abstractions;
namespace Jibo.Cloud.Application.Services;
public sealed class ResponsePlanToSocketMessagesMapper
{
public IReadOnlyList<string> Map(ResponsePlan plan)
{
var speak = plan.Actions.OfType<SpeakAction>().FirstOrDefault();
var messages = new List<string>();
if (speak is not null)
{
messages.Add(JsonSerializer.Serialize(new
{
type = "OPENJIBO_RESPONSE",
data = new
{
intent = plan.IntentName,
text = speak.Text,
followUpOpen = plan.FollowUp.KeepMicOpen,
timeoutMs = (int)plan.FollowUp.Timeout.TotalMilliseconds
}
}));
}
messages.Add(JsonSerializer.Serialize(new
{
type = "EOS",
data = new
{
sessionId = plan.SessionId
}
}));
return messages;
}
}

View File

@@ -0,0 +1,9 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
</Project>

View File

@@ -0,0 +1,11 @@
namespace Jibo.Cloud.Domain.Models;
public sealed class AccountProfile
{
public string AccountId { get; init; } = "usr_openjibo_owner";
public string Email { get; init; } = "owner@openjibo.local";
public string FirstName { get; init; } = "Jibo";
public string LastName { get; init; } = "Owner";
public string AccessKeyId { get; init; } = "openjibo-access-key";
public string SecretAccessKey { get; init; } = "openjibo-secret-access-key";
}

View File

@@ -0,0 +1,11 @@
namespace Jibo.Cloud.Domain.Models;
public sealed class CapturedExchange
{
public string CaptureId { get; init; } = Guid.NewGuid().ToString("N");
public DateTimeOffset CapturedUtc { get; init; } = DateTimeOffset.UtcNow;
public ProtocolEnvelope Request { get; init; } = new();
public ProtocolDispatchResult Response { get; init; } = ProtocolDispatchResult.Ok();
public string Confidence { get; init; } = "observed";
public IDictionary<string, string> Tags { get; init; } = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
}

View File

@@ -0,0 +1,17 @@
namespace Jibo.Cloud.Domain.Models;
public sealed class CloudSession
{
public string SessionId { get; init; } = Guid.NewGuid().ToString("N");
public string Kind { get; init; } = "http";
public string? AccountId { get; init; }
public string? DeviceId { get; init; }
public string? Token { get; init; }
public string? HostName { get; init; }
public string? Path { get; init; }
public DateTimeOffset CreatedUtc { get; init; } = DateTimeOffset.UtcNow;
public DateTimeOffset LastSeenUtc { get; set; } = DateTimeOffset.UtcNow;
public string? LastListenType { get; set; }
public string? LastTranscript { get; set; }
public IDictionary<string, object?> Metadata { get; init; } = new Dictionary<string, object?>();
}

View File

@@ -0,0 +1,12 @@
namespace Jibo.Cloud.Domain.Models;
public sealed class DeviceRegistration
{
public string DeviceId { get; init; } = "my-robot-serial-number";
public string RobotId { get; init; } = "my-robot-name";
public string FriendlyName { get; init; } = "OpenJibo Dev Robot";
public string? FirmwareVersion { get; init; }
public string? ApplicationVersion { get; init; }
public bool IsActive { get; init; } = true;
public IDictionary<string, string> HostMappings { get; init; } = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
}

View File

@@ -0,0 +1,40 @@
using System.Text.Json;
namespace Jibo.Cloud.Domain.Models;
public sealed class ProtocolDispatchResult
{
public int StatusCode { get; init; } = 200;
public string ContentType { get; init; } = "application/x-amz-json-1.1";
public string BodyText { get; init; } = "{}";
public IDictionary<string, string> Headers { get; init; } = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
public static ProtocolDispatchResult Ok(object? body = null)
{
return new ProtocolDispatchResult
{
StatusCode = 200,
BodyText = JsonSerializer.Serialize(body ?? new { ok = true })
};
}
public static ProtocolDispatchResult NoContent()
{
return new ProtocolDispatchResult
{
StatusCode = 204,
BodyText = string.Empty,
ContentType = "text/plain"
};
}
public static ProtocolDispatchResult Raw(int statusCode, string bodyText, string contentType = "text/plain")
{
return new ProtocolDispatchResult
{
StatusCode = statusCode,
BodyText = bodyText,
ContentType = contentType
};
}
}

View File

@@ -0,0 +1,39 @@
using System.Text.Json;
namespace Jibo.Cloud.Domain.Models;
public sealed class ProtocolEnvelope
{
public string RequestId { get; init; } = Guid.NewGuid().ToString("N");
public DateTimeOffset ReceivedUtc { get; init; } = DateTimeOffset.UtcNow;
public string Transport { get; init; } = "http";
public string Method { get; init; } = "POST";
public string HostName { get; init; } = "api.jibo.com";
public string Path { get; init; } = "/";
public string? ServicePrefix { get; init; }
public string? Operation { get; init; }
public string? DeviceId { get; init; }
public string? CorrelationId { get; init; }
public string? FirmwareVersion { get; init; }
public string? ApplicationVersion { get; init; }
public string BodyText { get; init; } = string.Empty;
public IDictionary<string, string> Headers { get; init; } = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
public JsonElement? TryParseBody()
{
if (string.IsNullOrWhiteSpace(BodyText))
{
return null;
}
try
{
using var document = JsonDocument.Parse(BodyText);
return document.RootElement.Clone();
}
catch
{
return null;
}
}
}

View File

@@ -0,0 +1,15 @@
namespace Jibo.Cloud.Domain.Models;
public sealed class UpdateManifest
{
public string UpdateId { get; init; } = "noop-update";
public DateTimeOffset CreatedUtc { get; init; } = DateTimeOffset.UtcNow;
public string FromVersion { get; init; } = "unknown";
public string ToVersion { get; init; } = "unknown";
public string Changes { get; init; } = "No update available";
public string Url { get; init; } = "https://api.jibo.com/update/noop";
public string ShaHash { get; init; } = "noop";
public long Length { get; init; }
public string Subsystem { get; init; } = "robot";
public string? Filter { get; init; }
}

View File

@@ -0,0 +1,10 @@
namespace Jibo.Cloud.Domain.Models;
public sealed class UploadReference
{
public string UploadId { get; init; } = Guid.NewGuid().ToString("N");
public string Path { get; init; } = string.Empty;
public string ContentType { get; init; } = "application/octet-stream";
public long Length { get; init; }
public DateTimeOffset CreatedUtc { get; init; } = DateTimeOffset.UtcNow;
}

View File

@@ -0,0 +1,13 @@
namespace Jibo.Cloud.Domain.Models;
public sealed class WebSocketMessageEnvelope
{
public string ConnectionId { get; init; } = Guid.NewGuid().ToString("N");
public string HostName { get; init; } = string.Empty;
public string Path { get; init; } = "/";
public string Kind { get; init; } = "unknown";
public string? Token { get; init; }
public string? Text { get; init; }
public byte[]? Binary { get; init; }
public bool IsBinary => Binary is { Length: > 0 };
}

View File

@@ -0,0 +1,7 @@
namespace Jibo.Cloud.Domain.Models;
public sealed class WebSocketReply
{
public string? Text { get; init; }
public bool Close { get; init; }
}

View File

@@ -0,0 +1,22 @@
using Jibo.Cloud.Application.Abstractions;
using Jibo.Cloud.Application.Services;
using Jibo.Cloud.Infrastructure.Persistence;
using Jibo.Runtime.Abstractions;
using Microsoft.Extensions.DependencyInjection;
namespace Jibo.Cloud.Infrastructure.DependencyInjection;
public static class ServiceCollectionExtensions
{
public static IServiceCollection AddOpenJiboCloud(this IServiceCollection services)
{
services.AddSingleton<ICloudStateStore, InMemoryCloudStateStore>();
services.AddSingleton<IConversationBroker, DemoConversationBroker>();
services.AddSingleton<ProtocolToTurnContextMapper>();
services.AddSingleton<ResponsePlanToSocketMessagesMapper>();
services.AddSingleton<JiboCloudProtocolService>();
services.AddSingleton<JiboWebSocketService>();
return services;
}
}

View File

@@ -0,0 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk">
<ItemGroup>
<ProjectReference Include="..\Jibo.Cloud.Application\Jibo.Cloud.Application.csproj" />
<ProjectReference Include="..\Jibo.Cloud.Domain\Jibo.Cloud.Domain.csproj" />
<FrameworkReference Include="Microsoft.AspNetCore.App" />
</ItemGroup>
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
</Project>

View File

@@ -0,0 +1,150 @@
using System.Collections.Concurrent;
using Jibo.Cloud.Application.Abstractions;
using Jibo.Cloud.Domain.Models;
namespace Jibo.Cloud.Infrastructure.Persistence;
public sealed class InMemoryCloudStateStore : ICloudStateStore
{
private readonly AccountProfile _account = new();
private readonly ConcurrentDictionary<string, DeviceRegistration> _devices = new(StringComparer.OrdinalIgnoreCase);
private readonly ConcurrentDictionary<string, CloudSession> _sessionsByToken = new(StringComparer.OrdinalIgnoreCase);
private readonly List<UpdateManifest> _updates;
private DeviceRegistration _robot;
public InMemoryCloudStateStore()
{
_robot = new DeviceRegistration
{
HostMappings = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
{
["api.jibo.com"] = "openjibo.com",
["api-socket.jibo.com"] = "openjibo.com",
["neo-hub.jibo.com"] = "openjibo.com",
["neohub.jibo.com"] = "openjibo.com"
}
};
_devices[_robot.DeviceId] = _robot;
_updates =
[
new UpdateManifest
{
UpdateId = "noop-update-robot",
FromVersion = "unknown",
ToVersion = "unknown",
Changes = "No update available",
Url = "https://api.jibo.com/update/noop",
ShaHash = "noop",
Subsystem = "robot"
}
];
}
public AccountProfile GetAccount() => _account;
public DeviceRegistration GetRobot() => _robot;
public DeviceRegistration GetOrCreateDevice(string deviceId, string? firmwareVersion, string? applicationVersion)
{
return _devices.AddOrUpdate(
deviceId,
_ => new DeviceRegistration
{
DeviceId = deviceId,
RobotId = $"robot-{deviceId}",
FriendlyName = "OpenJibo Registered Robot",
FirmwareVersion = firmwareVersion,
ApplicationVersion = applicationVersion
},
(_, current) => new DeviceRegistration
{
DeviceId = current.DeviceId,
RobotId = current.RobotId,
FriendlyName = current.FriendlyName,
FirmwareVersion = firmwareVersion ?? current.FirmwareVersion,
ApplicationVersion = applicationVersion ?? current.ApplicationVersion,
HostMappings = current.HostMappings
});
}
public string IssueHubToken()
{
var token = $"hub-{_account.AccountId}-{DateTimeOffset.UtcNow.ToUnixTimeMilliseconds()}";
_sessionsByToken[token] = new CloudSession
{
Kind = "hub",
AccountId = _account.AccountId,
Token = token,
DeviceId = _robot.DeviceId
};
return token;
}
public string IssueRobotToken(string deviceId)
{
var token = $"token-{deviceId}-{DateTimeOffset.UtcNow.ToUnixTimeMilliseconds()}";
_sessionsByToken[token] = new CloudSession
{
Kind = "robot",
AccountId = _account.AccountId,
Token = token,
DeviceId = deviceId
};
return token;
}
public CloudSession OpenSession(string kind, string? deviceId, string? token, string? hostName, string? path)
{
var session = new CloudSession
{
Kind = kind,
AccountId = _account.AccountId,
DeviceId = deviceId ?? _robot.DeviceId,
Token = token,
HostName = hostName,
Path = path
};
if (!string.IsNullOrWhiteSpace(token))
{
_sessionsByToken[token] = session;
}
return session;
}
public CloudSession? FindSessionByToken(string token)
{
return _sessionsByToken.GetValueOrDefault(token);
}
public IReadOnlyList<UpdateManifest> ListUpdates(string? subsystem = null, string? filter = null)
{
return _updates
.Where(update => subsystem is null || update.Subsystem.Equals(subsystem, StringComparison.OrdinalIgnoreCase))
.Where(update => filter is null || string.Equals(update.Filter, filter, StringComparison.OrdinalIgnoreCase))
.ToArray();
}
public UpdateManifest GetUpdateFrom(string? subsystem, string? fromVersion, string? filter)
{
return ListUpdates(subsystem, filter).FirstOrDefault() ?? new UpdateManifest
{
UpdateId = $"noop-update-{subsystem ?? "robot"}-{fromVersion ?? "unknown"}",
FromVersion = fromVersion ?? "unknown",
ToVersion = fromVersion ?? "unknown",
Filter = filter,
Subsystem = subsystem ?? "robot"
};
}
public void UpdateRobot(DeviceRegistration registration)
{
_robot = registration;
_devices[registration.DeviceId] = registration;
}
}