190 lines
6.2 KiB
Markdown
190 lines
6.2 KiB
Markdown
# DynaLab Core connector protocol
|
|
|
|
> [!IMPORTANT]
|
|
> **Packet model change.** Unprefixed packet classes (`Heartbeat`, `ConnectorHello`,
|
|
> `ValueDescriptor`, and so on) are lightweight slotted dataclasses for the
|
|
> internal data path. They do not perform Pydantic validation. Every packet that
|
|
> crosses the JSON-over-TCP boundary uses its Pydantic `Json*` counterpart:
|
|
> `JsonHeartbeat`, `JsonConnectorHello`, `JsonValueDescriptor`, etc. This
|
|
> separation avoids validating packets repeatedly under load. Convert at the
|
|
> boundary with `to_pure()` (received JSON) and `to_json()` (outgoing internal
|
|
> packet).
|
|
|
|
> [!WARNING]
|
|
> The protocol is alpha. Validate a connector against the Core version it will
|
|
> run with.
|
|
|
|
## Transport and framing
|
|
|
|
The current transport is JSON over TCP. A frame is one UTF-8 JSON object
|
|
followed by a newline (`\n`). The maximum incoming frame is 65,536 bytes,
|
|
including its newline delimiter. A missing delimiter, oversized frame,
|
|
invalid JSON, or a JSON object that does not match a `Json*` packet closes the
|
|
connection.
|
|
|
|
Use `JsonProtocolMessage` with the JSON wire helpers:
|
|
|
|
```python
|
|
from dynalab_core.protocols.json.wire import read_message, write_message
|
|
from dynalab_core.protocols.packets import JsonConnectorHello
|
|
|
|
# Receive JSON/TCP data (validated once), then enter the internal data path.
|
|
json_packet = await read_message(reader)
|
|
packet = json_packet.to_pure()
|
|
|
|
# Send a JSON packet directly, or convert an internal packet first.
|
|
await write_message(writer, JsonConnectorHello(...))
|
|
# await write_message(writer, packet.to_json())
|
|
```
|
|
|
|
## Packet representations
|
|
|
|
| Purpose | Internal type | JSON/TCP type |
|
|
| --- | --- | --- |
|
|
| Signal metadata | `SignalDescriptor` | `JsonSignalDescriptor` |
|
|
| Server hello | `DynaLabHello` | `JsonDynaLabHello` |
|
|
| Connector hello | `ConnectorHello` | `JsonConnectorHello` |
|
|
| Accepted / rejected | `HandshakeAccepted` / `HandshakeRejected` | `JsonHandshakeAccepted` / `JsonHandshakeRejected` |
|
|
| Heartbeat | `Heartbeat` | `JsonHeartbeat` |
|
|
| One value | `ValueDescriptor` | `JsonValueDescriptor` |
|
|
| Value batch | `ValueBatch` | `JsonValueBatch` |
|
|
|
|
`JsonProtocolMessage` is the discriminated union used to parse JSON frames;
|
|
`ProtocolMessage` is the corresponding union for the internal packet types.
|
|
The JSON model includes the discriminator field `type`; pure packets do not.
|
|
|
|
JSON models validate and serialize UUIDs and nested fields. Pure packets accept
|
|
the annotated values as supplied, so construct them only from trusted or
|
|
already-validated data.
|
|
|
|
## Timestamps
|
|
|
|
DynaLab uses the host monotonic clock:
|
|
|
|
- Heartbeat timestamps are milliseconds: `time.monotonic_ns() // 1_000_000`.
|
|
- Data timestamps are nanoseconds: `time.monotonic_ns()`.
|
|
|
|
Do not use wall-clock (`time.time()`) timestamps. The core compares heartbeat
|
|
times and plays data against this monotonic timebase.
|
|
|
|
## Handshake
|
|
|
|
All examples below are **wire JSON** and therefore correspond to `Json*`
|
|
packet classes.
|
|
|
|
1. Core sends `JsonDynaLabHello` immediately after the TCP connection opens.
|
|
2. The connector replies with `JsonConnectorHello` within five seconds.
|
|
3. Core returns `JsonHandshakeAccepted` or `JsonHandshakeRejected`.
|
|
4. On acceptance, the connector can send values and must return heartbeats.
|
|
|
|
### 1. `JsonDynaLabHello`
|
|
|
|
```json
|
|
{
|
|
"type": "dynalab_hello",
|
|
"instance_id": "2f3c7f62-0a45-4d66-9f32-7fbeb92f9d36",
|
|
"core_version": {"type": "alpha", "major": 0, "minor": 2, "patch": 1},
|
|
"protocol_version": {"type": "alpha", "major": 0, "minor": 0, "patch": 1},
|
|
"heartbeat_interval_ms": 1000,
|
|
"heartbeat_timeout_ms": 5000
|
|
}
|
|
```
|
|
|
|
Store the `protocol_version`, interval, and timeout. The connector must use
|
|
the exact protocol version in its reply.
|
|
|
|
### 2. `JsonConnectorHello`
|
|
|
|
```json
|
|
{
|
|
"type": "connector_hello",
|
|
"connector_uuid": "7b3d9a25-2d54-4cd3-b8ef-9c0c44d2dd81",
|
|
"protocol_version": {"type": "alpha", "major": 0, "minor": 0, "patch": 1},
|
|
"connector_name": "DynaLab Test Connector",
|
|
"connector_version": "0.1.0",
|
|
"signals": [{
|
|
"id": "2d60a378-f37c-4c37-867b-c68f5116598b",
|
|
"name": "Engine RPM",
|
|
"type": "number",
|
|
"min_value": 0.0,
|
|
"max_value": 6000.0,
|
|
"unit": "rpm",
|
|
"timeout_ms": 2000,
|
|
"origin": "source"
|
|
}]
|
|
}
|
|
```
|
|
|
|
Each connection needs a unique connector UUID and at least one signal. Signal
|
|
UUIDs should be stable across connector restarts. `origin` defaults to
|
|
`"source"`; connectors should not advertise derived signals.
|
|
|
|
### 3. Result
|
|
|
|
Accepted:
|
|
|
|
```json
|
|
{
|
|
"type": "handshake_accepted",
|
|
"accepted_signals": ["2d60a378-f37c-4c37-867b-c68f5116598b"]
|
|
}
|
|
```
|
|
|
|
Rejected:
|
|
|
|
```json
|
|
{"type": "handshake_rejected", "reason": "Protocol versions mismatch"}
|
|
```
|
|
|
|
Core rejects a protocol-version mismatch, a hello without signals, and a
|
|
connector UUID already registered on another active endpoint. Do not send data
|
|
until `handshake_accepted` arrives. Only send values for accepted signal IDs.
|
|
|
|
## Heartbeats
|
|
|
|
Core periodically sends `JsonHeartbeat`; return the same `sequence` and
|
|
`send_timestamp`, adding a local monotonic-millisecond `return_timestamp`.
|
|
|
|
```json
|
|
{"type": "heartbeat", "sequence": 42, "send_timestamp": 1938475621, "return_timestamp": null}
|
|
```
|
|
|
|
```json
|
|
{"type": "heartbeat", "sequence": 42, "send_timestamp": 1938475621, "return_timestamp": 1938475624}
|
|
```
|
|
|
|
Return heartbeats before `heartbeat_timeout_ms` elapses. A connector that does
|
|
not reply is disconnected.
|
|
|
|
## Data packets
|
|
|
|
Send `JsonValueDescriptor` for one sample:
|
|
|
|
```json
|
|
{
|
|
"type": "value_descriptor",
|
|
"signal_id": "2d60a378-f37c-4c37-867b-c68f5116598b",
|
|
"value": 3247.6,
|
|
"timestamp": 1938475621000000
|
|
}
|
|
```
|
|
|
|
For higher rates, send `JsonValueBatch` to amortize framing and scheduling
|
|
overhead:
|
|
|
|
```json
|
|
{
|
|
"type": "value_batch",
|
|
"values": [
|
|
{"type": "value_descriptor", "signal_id": "2d60a378-f37c-4c37-867b-c68f5116598b", "value": 3247.6, "timestamp": 1938475621000000},
|
|
{"type": "value_descriptor", "signal_id": "5097265d-cdf4-4872-a0e2-b9515ae66b19", "value": 6.42, "timestamp": 1938475621000100}
|
|
]
|
|
}
|
|
```
|
|
|
|
Every value timestamp is in monotonic nanoseconds. The batch has no separate
|
|
configured item limit, but its complete newline-delimited JSON frame must fit
|
|
within 65,536 bytes. Core silently ignores values for a signal that was not
|
|
declared by that connector, and connector data is routed only while Core is in
|
|
`"realtime"` mode.
|