Version 0.2.1

Version 0.2.1 now has derive unit context for derivative calculus

docs/Processing.md

@@ -0,0 +1,152 @@
+# DLUnit processing and derivatives
+
+DLUnits can optionally request a `DeriveContext` to calculate time derivatives
+of their declared inputs. The same interface works during realtime acquisition,
+offline `Core.reprocess()`, and playback with `playback_mode="reprocess"`.
+Playback with `playback_mode="read"` reads previously saved outputs instead.
+
+## Example: speed to acceleration
+
+Use this as the DLUnit's `unit.py` source:
+
+```python
+from dynalab_core.processing import DeriveContext
+from dynalab_core.protocols.packets.data import ValueDescriptor
+from dynalab_core.protocols.packets.handshake import SignalDescriptor
+
+
+def process(
+    speed: ValueDescriptor,
+    ctx: DeriveContext,
+    output: SignalDescriptor,
+) -> ValueDescriptor | None:
+    acceleration = ctx.derivative(speed, order=1, max_gap_s=0.5)
+    if acceleration is None:
+        return None
+    return ValueDescriptor(output.id, acceleration, speed.timestamp)
+```
+
+Bind it using the existing API:
+
+```python
+from uuid import uuid4
+from dynalab_core.dlunit import DLUnit
+from dynalab_core.protocols.packets.handshake import SignalDescriptor
+
+acceleration_signal = SignalDescriptor(
+    id=uuid4(), name="Acceleration (m/s²)", type="number", origin="derived"
+)
+unit = DLUnit()
+unit.set_manifest(uuid4(), [speed_signal], acceleration_signal)
+unit.set_data(source_text)  # The unit.py source above.
+core.bind_derive_unit(dlunit=unit)
+# Optionally persist it with unit.write(directory, "acceleration.dlunit").
+```
+
+`speed_signal` must describe the existing speed input. If its values are in m/s,
+the output values are in m/s². Core does not infer physical units or rename
+signals automatically. Derivative outputs are ordinary derived signals: they
+can feed another DLUnit, be recorded, and be saved in a DLPak.
+
+## Function contract
+
+A function takes its `ValueDescriptor` inputs in manifest order, optionally one
+`DeriveContext`, then one final `SignalDescriptor` describing the output. Use
+the actual imported classes in annotations, without postponed/string
+annotations. The source must expose exactly one Python function; importing the
+context class is supported.
+
+Existing functions such as `process(a, b, output) -> ValueDescriptor` continue
+to work. To skip output, explicitly annotate `-> ValueDescriptor | None` and
+return `None`. This also works without a context. Returning `None` with the
+original `-> ValueDescriptor` annotation remains an invalid result.
+
+No placeholder is emitted when a function returns `None`: live values are not
+updated, recording gets no sample, and downstream units receive no event for
+that output. A previously published live output remains subject to its normal
+timeout; skipping an output does not explicitly clear it.
+
+An offline unit annotated to allow `None` may produce an empty series. Core logs
+`derive.offline_no_values` and skips units requiring that empty output, logging
+`derive.offline_upstream_empty`. Reprocessing still succeeds, and descriptors
+are retained for the empty outputs. Units that do not allow `None` retain the
+`DeriveError(code="no_values")` behavior if they execute but emit nothing.
+
+## `ctx.derivative(value, order=1, *, max_gap_s=None) -> float | None`
+
+- `value` is one of the function's supplied input descriptors.
+- `order` is an integer: 1, 2, or 3. Other values raise `ValueError`.
+- `max_gap_s` is an optional finite positive maximum interval between
+  consecutive samples in the derivative window. Invalid limits raise
+  `ValueError`. The default places no gap limit.
+- The result uses seconds: input units/s, input units/s², or input units/s³.
+- Warm-up requires 2, 3, or 4 distinct timestamps, respectively. Until then,
+  the result is `None`. A window containing a gap above `max_gap_s` also returns
+  `None`; estimates resume after enough samples arrive following that gap.
+- Repeated calls are read-only and return the same estimate. You can request
+  different orders for the same input in one invocation.
+
+The estimator differentiates the degree-`order` polynomial interpolating the
+most recent `order + 1` samples. For order 1 this is the usual backward secant:
+`(new_value - previous_value) / elapsed_seconds`. Orders 2 and 3 use the actual
+unequal sample intervals rather than repeatedly dividing by a nominal period.
+Timestamp differences are taken as integers before converting to seconds.
+
+These are numerical estimates without smoothing. They use only present and
+past samples, and higher orders can magnify measurement noise. Very uneven
+spacing and floating-point precision also limit accuracy. A non-finite
+computed estimate returns `None`.
+
+## History and alignment
+
+Core retains at most four samples per declared input, per context-enabled unit.
+The router advances history only when that input itself updates and queues a
+snapshot for each invocation. Later arrivals cannot alter an already queued
+context, and calls to `derivative()` do not advance history.
+
+Context-enabled units wait until all declared inputs have a sample at or before
+the triggering event's timestamp. Each input retains its own original timestamp.
+Only the most recent four accepted samples are available, so severely late
+events may no longer have a companion sample in history and are skipped.
+Equal or older timestamps for a signal are ignored by that unit's history and
+do not trigger its processing function. A newer non-finite value clears that
+signal's history and skips the invocation; finite samples restart warm-up.
+These rules do not remove or alter raw recording samples.
+
+When another input changes, a derivative of an unchanged input reuses that
+input's existing estimate. Its history does not gain another sample. If the
+output combines multiple inputs, choose its timestamp explicitly, for example
+`max(a.timestamp, b.timestamp)`. Companion values are retained in the bounded
+history; they do not expire through the realtime live-value timeout. Apply any
+additional age policy in the function using the supplied timestamps.
+
+Offline processing uses the same bounded history and causal alignment, with a
+fresh history on every pass. Functions without a context retain the original
+offline nearest-neighbor alignment, including possible future samples and
+timestamps assigned from the triggering event.
+
+Online and offline estimates match for the same chronological accepted input
+sequence. Offline processing sorts stored samples; online out-of-order arrivals
+can therefore differ. Queue overload can also change the observed sequence:
+router drops omit samples from history, while a unit queue drop omits that
+invocation but its sample remains in later history snapshots. Asynchronous
+multi-input derived chains can observe a different event ordering online.
+Offline processing does not drop samples due to queue capacity.
+
+Starting or stopping recording does not reset ongoing online history. If a
+recording begins midstream, its first online derivative outputs may depend on
+samples preceding the recording. Offline recalculation warms up from the
+recorded samples and can omit those early estimates.
+
+## Lifecycle
+
+Online histories are invalidated when Core changes mode, a source connector
+registers/disconnects, processing or playback data is replaced, or a playback
+control changes its generation (play/resume, pause, seek, or restart). Connector
+changes conservatively invalidate all units' histories. Queued requests and
+results from an older revision are discarded, including results completing
+after the reset. New data warms up the estimator again.
+
+Offline histories are local to the processing pass and never share online
+state. DLUnit archives save source and manifest, not history. DLPak and connector
+packet formats are unchanged. No numerical-library dependency is required.