dynalab-core/docs/API.md

# DynaLab Core API

`Core` runs a DynaLab Core instance: it accepts connector data, retains live
values, records data, prepares DLPak archives, plays recordings, and manages
derive units. Connector authors should also read the [protocol guide](Protocol.md).

> [!IMPORTANT]
> **Packet model change.** The default packet and signal classes are lightweight
> `@dataclass(slots=True)` objects. They intentionally do not validate or coerce
> input at construction time. Pydantic models are retained only at a JSON
> boundary and are named with the `Json` prefix, such as `JsonHeartbeat` and
> `JsonConnectorHello`. Use the `Json*` classes with JSON-over-TCP and convert
> with `to_pure()`; use the unprefixed classes inside Core, connectors, and
> derive code. See [Packet representations](Protocol.md#packet-representations).

> [!WARNING]
> DynaLab Core is in alpha. Its API and protocol can change between releases.

## Quick start

```python
from dynalab_core import Core
from dynalab_core.config import CoreConfig

core = Core(CoreConfig(host="127.0.0.1", port=58763))
try:
    core.start()
    core.wait()
finally:
    core.stop()
```

Constructing `Core` starts its internal workers. `start()` starts the TCP
server. Always call `stop()` during shutdown; a stopped core cannot be started
again.

## Configuration

`CoreConfig` is a Pydantic configuration model in `dynalab_core.config`.

| Field | Type | Default | Meaning |
| --- | --- | --- | --- |
| `host` | `str` | `"127.0.0.1"` | TCP bind address. |
| `port` | `int` | `58763` | TCP bind port; validated from 0 through 65535. |
| `thread_debug` | `bool` | `False` | Enables periodic per-thread CPU debug logging. |
| `playback_mode` | `"reprocess" \| "read"` | `"reprocess"` | Whether playback recalculates bound derived signals or reads their recorded values. Offline `reprocess()` requires `"read"`. |

`CoreConfig.bind_str() -> str` returns `"host:port"`.

## Lifecycle

### `Core(config: CoreConfig)`

Creates a core, its input, playback, and derive workers, plus the JSON/TCP
server object. It does not bind the network port until `start()`.

### `start() -> None`

Starts the configured JSON/TCP server. It may be called only once, while Core
is initialised; otherwise it raises `CoreError(code="invalid_state")`. A start
timeout or bind failure raises `JsonServerError` with code `"start_timeout"` or
`"startup_failed"`.

### `wait(timeout: float | None = None) -> None`

Blocks until `stop()` is called or the optional timeout elapses.

### `stop() -> bool`

Stops the server, connector registry, derive registry, input worker, and
playback worker. Returns `True` only when all required components stop within
their time limits; it returns `False` if shutdown is incomplete.

## Modes and buffers

`CoreMode` is `"realtime" | "playback" | "offline"`; new instances start in
`"realtime"`. `set_mode()` clears cached live values and cannot run during an
active recording (`CoreError(code="recording_active")`).

| Mode | Source | Purpose |
| --- | --- | --- |
| `"realtime"` | Connected connectors | Live values and recording. |
| `"playback"` | Loaded DLPak | Time-based replay. |
| `"offline"` | Recorded or loaded DLPak | Prepare a processing buffer without replaying it. |

Endpoints remain connected in non-realtime modes, but only realtime connector
values are routed into Core.

### `set_mode(mode: CoreMode) -> None` / `get_mode() -> CoreMode`

Sets or returns the current mode. `CoreMode` is a type contract; callers must
pass one of the three values above. Changing mode does not automatically pause
playback, so pause it before leaving playback mode.

### `load_playback_engine(input: Path | None = None) -> None`

Available only in `"playback"` mode (`CoreError(code="invalid_mode")`). Pass a
`.dlpak` path, or omit `input` to load the current processing buffer. With no
processing buffer, omitting it raises `CoreError(code="no_playback_data")`.
Loading does not start playback; use `play()`.

### `load_processing_buffer(input: Path | None = None) -> None`

Available only in `"offline"` mode (`CoreError(code="invalid_mode")`). Pass a
`.dlpak` path to load it, or omit `input` to turn the completed recording into
a processing buffer and create its manifest. Omitting `input` before a
recording is available raises `CoreError(code="no_playback_data")`.

### `reprocess() -> None`

Recalculates derived signals in the current processing buffer using the bound
derive units. Call it in `"offline"` mode after `load_processing_buffer()`, with
`CoreConfig(playback_mode="read")`. Units with source inputs run first; units
that depend on their derived outputs run in dependency order. The operation
removes previously recorded derived samples and descriptors, then merges the
new outputs into the processing buffer. Calling it again replaces those
outputs. It does not write an archive or start playback; use `write()` to save
the result, or switch to `"playback"` mode, call `load_playback_engine()`, and
then `play()`.

The mode, configuration, and buffer checks raise `CoreError` with codes
`"invalid_mode"`, `"invalid_config"`, and `"empty_buffer"`, respectively.
Unresolved derive dependencies raise `DeriveError(code="unresolved_dependencies")`;
a unit that produces no values raises `DeriveError(code="no_values")`. Errors
from a derive function propagate to the caller. Reprocessing changes the
in-memory processing buffer before all units finish, so a failed call may
leave its old derived data removed. Reload an unchanged archive before retrying
if that data is needed.

## Signals and live values

`SignalDescriptor` is the lightweight internal signal type in
`dynalab_core.protocols.packets.handshake`. Its fields are `id`, `name`,
`type` (`"number"` or `"binary"`), `min_value`, `max_value`, `unit`,
`timeout_ms` (default `2000`), and `origin` (`"source"` or `"derived"`, default
`"source"`). Use `JsonSignalDescriptor` only when serializing a descriptor.

### `get_signal_descriptor(signal_id: UUID) -> SignalDescriptor | None`

Returns a copy of the matching descriptor or `None`. Realtime descriptors come
from connectors and bound derive units. Playback descriptors come from the
loaded archive; derived descriptors are recalculated only when
`playback_mode="reprocess"`.

### `get_all_signal_descriptors() -> list[SignalDescriptor]`

Returns descriptors available in the current mode. In `"read"` playback mode,
recorded derived descriptors are included; in `"reprocess"` mode, bound derive
units supply them instead.

### `get_live_value(signal_id: UUID) -> float | None`

Returns the most recent non-stale value for a signal, or `None` when unknown,
not yet received, or past the signal's `timeout_ms`.

### `get_all_live_values() -> dict[UUID, float]`

Returns a snapshot of current non-stale values. Signals without a live value
are omitted. Ingress timestamps use `time.monotonic_ns()`; see the protocol
guide for details.

## Recording and DLPak archives

### `start_recording() -> None` / `stop_recording() -> None`

Recording is available only in `"realtime"` mode. Starting clears the prior
in-memory buffer; stopping normalizes and sorts samples and creates an
offload buffer with a manifest. Errors use `"incorrect_mode"`,
`"recording_active"`, or `"not_recording"` as appropriate.

### `get_recording_state() -> bool`

Returns whether a recording is active.

### `write(dir: str | Path, filename: str) -> None`

Writes the current **processing** buffer as a `.dlpak` archive; the extension
is appended when absent. After recording, select `"offline"` mode and call
`load_processing_buffer()` before writing. If no processing buffer exists,
this raises `CoreError(code="no_recording")`. Archive failures raise
`DLPakError` with a stable code such as `"write_failed"`.

## Playback

### `play() -> None` / `pause() -> None`

Start or resume, and pause, a loaded playback engine. Both require
`"playback"` mode (`CoreError(code="invalid_mode")`). `play()` before loading
data raises `PlaybackError(code="no_data")`. Playback returns to timestamp zero
after it reaches the end.

### `seek(timestamp_ns: int) -> None`

Sets the playback position relative to the recording's first sample. Requires
`"playback"` mode; a negative timestamp raises
`PlaybackError(code="invalid_timestamp")`.

### `get_playback_state() -> bool`

Returns `True` while the playback engine is emitting data.

## Derived signals

### `bind_derive_unit(dlunit: DLUnit | None = None, file: Path | None = None) -> UUID`

Binds an in-memory `DLUnit` or loads one from a `.dlunit` path, then returns
its ID. Provide exactly one source. Missing or competing sources raise
`CoreError(code="bind_derive_failed")`; invalid source or function shape raises
`DeriveError` (for example `"invalid_source"` or `"invalid_signature"`).

Derive source operates on internal `ValueDescriptor` and `SignalDescriptor`
dataclasses, not their `Json*` equivalents. Source is executed with `exec()`;
only load units from trusted sources.

### `unbind_derive_unit(unit_id: UUID) -> None`

Stops and removes a bound derive unit. An unknown ID has no effect.

## Exceptions

Expected library errors inherit from `DynaLabError` in `dynalab_core.errors`
and expose a stable `code` attribute. Catch a specific subsystem class for
recovery, or `DynaLabError` at an application boundary. Operating-system and
validation causes are retained through `error.__cause__` where applicable.

| Exception | Area |
| --- | --- |
| `CoreError` | Lifecycle, modes, recording, buffer preparation, derive binding. |
| `DLPakError` | DLPak archive read/write and completeness. |
| `DLUnitError` | DLUnit archive read/write and completeness. |
| `DeriveError` | Derive source, signature, arguments, registration. |
| `PlaybackError` | Playback data, state, and timestamps. |
| `ConnectorError` | Connector queues and registration. |
| `JsonServerError` | JSON/TCP server startup and timeouts. |