dynalab-core/docs/API.md

DynaLab Core API

Core runs a DynaLab Core instance: it accepts connector data, retains live values, records data, prepares DLPak archives, plays recordings, and manages derive units. Connector authors should also read the protocol guide.

Important

Packet model change. The default packet and signal classes are lightweight @dataclass(slots=True) objects. They intentionally do not validate or coerce input at construction time. Pydantic models are retained only at a JSON boundary and are named with the Json prefix, such as JsonHeartbeat and JsonConnectorHello. Use the Json* classes with JSON-over-TCP and convert with to_pure(); use the unprefixed classes inside Core, connectors, and derive code. See Packet representations.

Warning

DynaLab Core is in alpha. Its API and protocol can change between releases.

Quick start

from dynalab_core import Core
from dynalab_core.config import CoreConfig

core = Core(CoreConfig(host="127.0.0.1", port=58763))
try:
    core.start()
    core.wait()
finally:
    core.stop()

Constructing Core starts its internal workers. start() starts the TCP server. Always call stop() during shutdown; a stopped core cannot be started again.

Configuration

CoreConfig is a Pydantic configuration model in dynalab_core.config.

Field	Type	Default	Meaning
host	str	"127.0.0.1"	TCP bind address.
port	int	58763	TCP bind port; validated from 0 through 65535.
thread_debug	bool	False	Enables periodic per-thread CPU debug logging.
playback_mode	"reprocess" | "read"	"reprocess"	Whether playback recalculates bound derived signals or reads their recorded values. Offline reprocess() requires "read".

CoreConfig.bind_str() -> str returns "host:port".

Lifecycle

Core(config: CoreConfig)

Creates a core, its input, playback, and derive workers, plus the JSON/TCP server object. It does not bind the network port until start().

start() -> None

Starts the configured JSON/TCP server. It may be called only once, while Core is initialised; otherwise it raises CoreError(code="invalid_state"). A start timeout or bind failure raises JsonServerError with code "start_timeout" or "startup_failed".

wait(timeout: float | None = None) -> None

Blocks until stop() is called or the optional timeout elapses.

stop() -> bool

Stops the server, connector registry, derive registry, input worker, and playback worker. Returns True only when all required components stop within their time limits; it returns False if shutdown is incomplete.

Modes and buffers

CoreMode is "realtime" | "playback" | "offline"; new instances start in "realtime". set_mode() clears cached live values and cannot run during an active recording (CoreError(code="recording_active")).

Mode	Source	Purpose
"realtime"	Connected connectors	Live values and recording.
"playback"	Loaded DLPak	Time-based replay.
"offline"	Recorded or loaded DLPak	Prepare a processing buffer without replaying it.

Endpoints remain connected in non-realtime modes, but only realtime connector values are routed into Core.

set_mode(mode: CoreMode) -> None / get_mode() -> CoreMode

Sets or returns the current mode. CoreMode is a type contract; callers must pass one of the three values above. Changing mode does not automatically pause playback, so pause it before leaving playback mode.

load_playback_engine(input: Path | None = None) -> None

Available only in "playback" mode (CoreError(code="invalid_mode")). Pass a .dlpak path, or omit input to load the current processing buffer. With no processing buffer, omitting it raises CoreError(code="no_playback_data"). Loading does not start playback; use play().

load_processing_buffer(input: Path | None = None) -> None

Available only in "offline" mode (CoreError(code="invalid_mode")). Pass a .dlpak path to load it, or omit input to turn the completed recording into a processing buffer and create its manifest. Omitting input before a recording is available raises CoreError(code="no_playback_data").

reprocess() -> None

Recalculates derived signals in the current processing buffer using the bound derive units. Call it in "offline" mode after load_processing_buffer(), with CoreConfig(playback_mode="read"). Units with source inputs run first; units that depend on their derived outputs run in dependency order. Each unit runs in sample order using per-signal binary searches instead of full-buffer scans. For other inputs it selects the nearest recorded value, preferring the earlier sample when distances tie. This retains the previous offline alignment policy, including selection of future samples. The operation removes previously recorded derived samples and descriptors, then merges the new outputs into the processing buffer. Calling it again replaces those outputs. It does not write an archive or start playback; use write() to save the result, or switch to "playback" mode, call load_playback_engine(), and then play().

The mode, configuration, and buffer checks raise CoreError with codes "invalid_mode", "invalid_config", and "empty_buffer", respectively. Unresolved derive dependencies raise DeriveError(code="unresolved_dependencies"); a unit that produces no values raises DeriveError(code="no_values"). Errors from a derive function propagate to the caller. Reprocessing changes the in-memory processing buffer before all units finish, so a failed call may leave its old derived data removed. Reload an unchanged archive before retrying if that data is needed.

Signals and live values

SignalDescriptor is the lightweight internal signal type in dynalab_core.protocols.packets.handshake. Its fields are id, name, type ("number" or "binary"), min_value, max_value, unit, timeout_ms (default 2000), and origin ("source" or "derived", default "source"). Use JsonSignalDescriptor only when serializing a descriptor.

get_signal_descriptor(signal_id: UUID) -> SignalDescriptor | None

Returns a copy of the matching descriptor or None. Realtime descriptors come from connectors and bound derive units. Playback descriptors come from the loaded archive; derived descriptors are recalculated only when playback_mode="reprocess".

get_all_signal_descriptors() -> list[SignalDescriptor]

Returns descriptors available in the current mode. In "read" playback mode, recorded derived descriptors are included; in "reprocess" mode, bound derive units supply them instead.

get_live_value(signal_id: UUID) -> float | None

Returns the most recent non-stale value for a signal, or None when unknown, not yet received, or past the signal's timeout_ms.

get_all_live_values() -> dict[UUID, float]

Returns a snapshot of current non-stale values. Signals without a live value are omitted. Ingress timestamps use time.monotonic_ns(); see the protocol guide for details.

Recording and DLPak archives

start_recording() -> None / stop_recording() -> None

Recording is available only in "realtime" mode. Starting clears the prior in-memory buffer; stopping normalizes and sorts samples and creates an offload buffer with a manifest. Errors use "incorrect_mode", "recording_active", or "not_recording" as appropriate.

get_recording_state() -> bool

Returns whether a recording is active.

write(dir: str | Path, filename: str, *, source: BufferSource = "processing") -> None

Writes the selected buffer as a .dlpak archive; the extension is appended when absent. BufferSource is "recording" | "processing" | "playback", defined in dynalab_core.types. The default remains "processing" for compatibility. Use source="recording" to save a completed recording directly, without switching modes or loading the processing buffer. Stop recording first; saving an active recording raises CoreError(code="recording_active"). source="playback" saves the loaded archive, not the current playback position or newly recalculated live values. Saving does not change mode or load data.

An unavailable buffer raises CoreError(code="no_recording"). Archive failures raise DLPakError(code="write_failed"). Writes use a temporary file in the destination directory and replace the destination only after the archive has been completed successfully. The destination directory must already exist.

from pathlib import Path

core.start_recording()
# Receive connector data, then finish and save it.
core.stop_recording()
core.write("recordings", "run-001", source="recording")

core.set_mode("offline")
core.load_processing_buffer(Path("recordings/run-001.dlpak"))

DLPak format and standalone access

DLPak in dynalab_core.dlpak can also be used without Core:

from pathlib import Path
from dynalab_core.dlpak import DLPak

archive = DLPak().read(Path("recordings/run-001.dlpak"))
series = archive.get_data_range(start_ns=0, end_ns=1_000_000_000)
archive.write("recordings", "run-001-copy")

Archives are ZIP files containing manifest.json and data.csv. The manifest contains format_version (currently 1), a UTC recording timestamp, signal descriptors, and comments. Older manifests without format_version are read as version 1; unsupported versions are rejected. Signal IDs in the manifest must be unique, and every signal present in the CSV must have a descriptor. Descriptors without samples are allowed. Invalid archives raise DLPakError(code="read_failed"), with the underlying cause retained.

The CSV header is timestamp_ns followed by signal UUID columns. Empty cells represent absent samples. Rows are ordered by timestamp. Repeated timestamps preserve multiple values for the same signal; samples are not collapsed.

Buffer queries for plotting

get_data_range(signal_ids: list[UUID] | None = None, start_ns: int | None = None, end_ns: int | None = None, *, source: BufferSource | None = None) -> dict[UUID, list[tuple[int, float]]]

Returns an independent snapshot grouped by signal ID. Each list contains (timestamp_ns, value) pairs in timestamp order, including duplicate timestamps. Both bounds are inclusive. Omit a bound for an open-ended range, or omit both for all samples. Omit signal_ids to select every signal; an unknown requested ID maps to an empty list. An inverted range raises ValueError. Data is returned at its original resolution without resampling or downsampling.

By default, the selected buffer follows the current mode:

Mode	Queried buffer
"realtime"	Active recording, or the latest completed recording when not recording.
"offline"	Loaded processing buffer.
"playback"	Loaded playback archive, including its recorded derived samples.

Pass source="recording", "processing", or "playback" to query a specific buffer without changing modes. An unavailable buffer raises CoreError(code="no_recording"); an available empty buffer returns empty data. No history is collected outside recordings. Playback queries access the loaded archive, not newly recalculated live derived values.

Completed recordings use nanoseconds relative to their first sample. Active recordings still contain ingress time.monotonic_ns() timestamps. Imported archives retain their stored timestamps. Bounds and returned timestamps always use the selected buffer's stored time base; stopping a recording normalizes it.

bounds = core.get_time_range()
series = core.get_data_range(
    [rpm_signal_id, torque_signal_id],
    start_ns=1_000_000_000,
    end_ns=2_000_000_000,
)
# series[rpm_signal_id] is a list of (timestamp_ns, value) pairs.

get_time_range(*, source: BufferSource | None = None) -> tuple[int, int] | None

Returns the minimum and maximum stored timestamps for the same buffer selection as get_data_range(). An empty buffer returns None; a single sample returns its timestamp for both bounds.

DLPak and ValueBuffer expose the same get_data_range() and get_time_range() methods without the source argument. Buffer mutations maintain per-signal indexes; out-of-order signal samples are sorted before querying. Once sorted, a range query uses binary search plus copying the selected samples, rather than scanning unrelated signals.

Playback

play() -> None / pause() -> None

Start or resume, and pause, a loaded playback engine. Both require "playback" mode (CoreError(code="invalid_mode")). play() before loading data raises PlaybackError(code="no_data"). Playback returns to timestamp zero after it reaches the end.

seek(timestamp_ns: int) -> None

Sets the playback position relative to the recording's first sample. Requires "playback" mode; a negative timestamp raises PlaybackError(code="invalid_timestamp").

get_playback_state() -> bool

Returns True while the playback engine is emitting data.

Derived signals

bind_derive_unit(dlunit: DLUnit | None = None, file: Path | None = None) -> UUID

Binds an in-memory DLUnit or loads one from a .dlunit path, then returns its ID. Provide exactly one source. Missing or competing sources raise CoreError(code="bind_derive_failed"); invalid source or function shape raises DeriveError (for example "invalid_source" or "invalid_signature").

Derive source operates on internal ValueDescriptor and SignalDescriptor dataclasses, not their Json* equivalents. Source is executed with exec(); only load units from trusted sources.

unbind_derive_unit(unit_id: UUID) -> None

Stops and removes a bound derive unit. An unknown ID has no effect.

Exceptions

Expected library errors inherit from DynaLabError in dynalab_core.errors and expose a stable code attribute. Catch a specific subsystem class for recovery, or DynaLabError at an application boundary. Operating-system and validation causes are retained through error.__cause__ where applicable.

Exception	Area
CoreError	Lifecycle, modes, recording, buffer preparation, derive binding.
DLPakError	DLPak archive read/write and completeness.
DLUnitError	DLUnit archive read/write and completeness.
DeriveError	Derive source, signature, arguments, registration.
PlaybackError	Playback data, state, and timestamps.
ConnectorError	Connector queues and registration.
JsonServerError	JSON/TCP server startup and timeouts.