diff --git a/docs/API.md b/docs/API.md new file mode 100644 index 0000000..e69de29 diff --git a/docs/Internal/README.md b/docs/Internal/README.md new file mode 100644 index 0000000..e69de29 diff --git a/docs/Protocol.md b/docs/Protocol.md new file mode 100644 index 0000000..eb15eed --- /dev/null +++ b/docs/Protocol.md @@ -0,0 +1,184 @@ +# DynaLab Protocol + +> [!NOTE] +> All protocol documentation uses JSON packets as examples, however it applies to any form of communication protocol used for data ingress + +> !NOTE +> Any transport dependent specificities will be noted using this format, look out for any specificities to the transport you are using when building connectors + +## Transports + +DynaLab currently only has one transport option: +- JSON over TCP + +More will be added as the project develops + +## Layers + +Internally DynaLab Core works in three layers, it is important to understand this architecture to understand how the handshake procedure works. + +The layers are as follows: +- Server - layer 1 +- Endpoint - layer 2 +- Core - layer 3 + +**Layer 1** - The server, this is what any connectors connect to, it is responsible for the initial `dynalab_hello` and `connector_hello` handshake before elevating to the next layer. Once elevated the server sends incoming deserialized packets to Layer 2 + +**Layer 2** - The endpoint, this is a *connector mapping* of sorts, each connector gets its own endpoint, the endpoint handles packet filtering and the connector heartbeat. Once the handshake is accepted this layer send all data oriented packets to Layer 3 + +**Layer 3** - The core, this is the heart of DynaLab, it handles live value storage, as well as all of the recording and processing aspects of DynaLab + + + + + +## Handshake +> [!TIP] +> All handshake relevant packets are located in the `dynalab_core.protocols.packets.handshake` module + +### Step 1 +The first step to establishing a communication with the DynaLab Core is proceeding with the initialization handshake. + +With most transports upon connection DynaLab Core will send the first packet, this is know as the `dynalab_hello` packet. + +```json +{ + "type": "dynalab_hello", + "instance_id": "2f3c7f62-0a45-4d66-9f32-7fbeb92f9d36", + "core_version": { + "type": "alpha", + "major": 0, + "minor": 1, + "patch": 0 + }, + "protocol_version": { + "type": "alpha", + "major": 0, + "minor": 0, + "patch": 1 + }, + "heartbeat_interval_ms": 1000, + "heartbeat_timeout_ms": 5000 +} +``` + +It is recommended to store this packet for potential later use even if it is not critical for a simple connector + +### Step 2 +Once the `dynalab_hello` has been received it is time to return a `connector_hello` packet, this can be found in the `handshake` module as `ConnectorHello` + +```json +{ + "type": "connector_hello", + "connector_uuid": "7b3d9a25-2d54-4cd3-b8ef-9c0c44d2dd81", + "protocol_version": { + "type": "alpha", + "major": 0, + "minor": 0, + "patch": 1 + }, + "connector_name": "DynaLab Test Connector", + "connector_version": "0.1.0", + "signals": [ + { + "id": "2d60a378-f37c-4c37-867b-c68f5116598b", + "name": "Engine RPM", + "type": "number", + "min_value": 0.0, + "max_value": 6000.0, + "unit": "rpm", + "timeout_ms": 2000 + }, + { + "id": "5097265d-cdf4-4872-a0e2-b9515ae66b19", + "name": "Torque", + "type": "number", + "min_value": 0.0, + "max_value": 10.0, + "unit": "Nm", + "timeout_ms": 2000 + }, + { + "id": "d6e633cd-7ca3-4974-a164-44482ddb2a8e", + "name": "Emergency Stop", + "type": "binary", + "min_value": null, + "max_value": null, + "unit": null, + "timeout_ms": 500 + } + ] +} +``` + +This packet will tell DynaLab Core information about itself and the available signals. + +> [!IMPORTANT] +> `protocol_version` must match the one sent in `dynalab_hello` otherwise the connector will be rejected + +> [!IMPORTANT] +> All id's are UUID's and should be generated using the v4 UUID generation algorithm if possible, they can be stored in config files for repeatable UUID's across restarts but should never be set with human created values + +### Step 3 +Once the DynaLab Core has received this `connector_hello` packet it will elevate the connection internally to layer 2. At this point DynaLab Core will respond with one of two packets: +- `handshake_accepted` +- `handshake_declined` + +**`handshake_accepted`** - This means that DynaLab Core has accepted the handshake and that it is ready for data +```json +{ + "type": "handshake_accepted", + "accepted_signals": [ + "2d60a378-f37c-4c37-867b-c68f5116598b", + "5097265d-cdf4-4872-a0e2-b9515ae66b19", + "d6e633cd-7ca3-4974-a164-44482ddb2a8e" + ] +} +``` +> [!NOTE] +> It is recommended to check the accepted signals list as not all signals may be accepted. DynaLab Core will drop any incoming values that it doesn't recognize however this is not recommended as it can hinder performance especially when a signals frequency is very high + +**`handshake_declined`** - This means that DynaLab Core has rejected the handshake for some reason, usually this will be a protocol mismatch or an empty list of signals, after this packet is sent, the connection will be closed by DynaLab Core +```json +{ + "type": "handshake_rejected", + "reason": "The connector uses an unsupported protocol version." +} +``` + +### Step 4 +The handshake is now complete, and either accepted or rejected + +The next order of business is handling the heartbeat which is a key part of the DynaLab Core - Connector protocol + +## Heartbeat +Heartbeats are sent out from DynaLab Core at the interval specified by the `dynalab_hello` packet during the handshake procedure. These heartbeats need to be returned to DynaLab Core with a return timestamp. + +A typical incoming heartbeat looks like so: +```json +{ + "type": "heartbeat", + "sequence": 42, + "send_timestamp": 1938475621, + "return_timestamp": null +} +``` +It must then be returned with a timestamp generated by the connector, in python: +```python +return_timestamp = round(monotonic() * 1000) +``` +This gives a millisecond timestamp that is OS wide to avoid any synchronization issues between DynaLab Core and connectors. + +A typical returned heartbeat looks like so: +```json +{ + "type": "heartbeat", + "sequence": 42, + "send_timestamp": 1938475621, + "return_timestamp": 1938475624 +} +``` + +You may have noticed that in the `dynalab_hello` packet there was a timeout value, if DynaLab Core receives no returned heartbeats from the connector for a longer than this timeout, the connector is deemed dead and the connection is closed + +## Data diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..c83b93d --- /dev/null +++ b/docs/README.md @@ -0,0 +1,12 @@ +# DynaLab Core Documentation + +This documentation is ordered in the following sections +- API +- Protocol +- Internal + +[**API**](API.md) - Documents the user side API of the `Core` class, this goes through setting up a new core, and how to interact with it + +[**Protocol**](Protocol.md) - Documents the data ingress protocol, this is useful for developing connectors to pipe data into the DynaLab Core + +[**Internal**](Internal/README.md) - Documents the inner working of DynaLab Core including data flow, internal API's, custom file types and much more