Version 0.2.1

Version 0.2.1 now has derive unit context for derivative calculus

docs/API.md

@@ -127,10 +127,13 @@ Recalculates derived signals in the current processing buffer using the bound
 derive units. Call it in `"offline"` mode after `load_processing_buffer()`, with
 `CoreConfig(playback_mode="read")`. Units with source inputs run first; units
 that depend on their derived outputs run in dependency order. Each unit runs
-in sample order using per-signal binary searches instead of full-buffer scans.
-For other inputs it selects the nearest recorded value, preferring the earlier
-sample when distances tie. This retains the previous offline alignment policy,
-including selection of future samples. The operation
+in sample order. Legacy alignment uses per-signal binary searches; contexts
+advance bounded histories.
+For functions without a processing context, other inputs select the nearest
+recorded value, preferring the earlier sample when distances tie; this can
+select future samples. Functions accepting `DeriveContext` use causal history
+and preserve each input's timestamp instead. See [processing and derivatives](Processing.md).
+The operation
 works on a deep copy, removes its previously recorded derived samples and
 descriptors, and merges the new outputs. It replaces the processing buffer
 only after all units finish successfully. Calling it again replaces those
@@ -141,8 +144,10 @@ then `play()`.
 The mode, configuration, and buffer checks raise `CoreError` with codes
 `"invalid_mode"`, `"invalid_config"`, and `"empty_buffer"`, respectively.
 Missing inputs raise `DeriveError(code="missing_inputs")`, cycles raise
-`DeriveError(code="dependency_cycle")`, and a unit that produces no values raises
-`DeriveError(code="no_values")`. Errors from a derive function propagate to the
+`DeriveError(code="dependency_cycle")`. A unit that produces no values raises
+`DeriveError(code="no_values")` unless its return annotation allows `None`.
+An optional-output unit may produce an empty series; units requiring that output
+are skipped with a diagnostic. Errors from a derive function propagate to the
 synchronous caller or are stored in the background operation status. Failure
 leaves the original processing buffer, including its prior derived results
 and metadata, unchanged.
@@ -366,6 +371,14 @@ Derive source operates on internal `ValueDescriptor` and `SignalDescriptor`
 dataclasses, not their `Json*` equivalents. Source is executed with `exec()`;
 only load units from trusted sources.
 
+Functions may optionally accept a `DeriveContext` between their input values and
+the final output descriptor. `ctx.derivative(value, order=1, max_gap_s=None)`
+estimates time derivatives of orders 1–3 using bounded history in realtime,
+offline processing, and recalculated playback. Annotate the return type as
+`ValueDescriptor | None` to skip outputs during warm-up. Existing functions keep
+their original signatures and behavior. See [DLUnit processing and derivatives](Processing.md)
+for examples, numerical behavior, input alignment, and reset rules.
+
 ### `validate_derive_units() -> None`
 
 Checks the configured graph against signals available in the current mode.
@@ -375,6 +388,8 @@ raise `DeriveError(code="missing_inputs")`; dependency cycles raise
 `"duplicate_output"`. Outputs must have `origin="derived"`, otherwise validation
 raises `"invalid_origin"`. A derive function must return a `ValueDescriptor`
 whose `signal_id` is its configured output, or it raises `"invalid_result"`.
+Functions annotated `-> ValueDescriptor | None` may also return `None` to emit
+no sample for that invocation.
 
 Structural validation runs before a unit is registered, so cycles and duplicate
 outputs cannot begin realtime execution. Units can be bound before their sources

docs/Processing.md

@@ -0,0 +1,152 @@
+# DLUnit processing and derivatives
+
+DLUnits can optionally request a `DeriveContext` to calculate time derivatives
+of their declared inputs. The same interface works during realtime acquisition,
+offline `Core.reprocess()`, and playback with `playback_mode="reprocess"`.
+Playback with `playback_mode="read"` reads previously saved outputs instead.
+
+## Example: speed to acceleration
+
+Use this as the DLUnit's `unit.py` source:
+
+```python
+from dynalab_core.processing import DeriveContext
+from dynalab_core.protocols.packets.data import ValueDescriptor
+from dynalab_core.protocols.packets.handshake import SignalDescriptor
+
+
+def process(
+    speed: ValueDescriptor,
+    ctx: DeriveContext,
+    output: SignalDescriptor,
+) -> ValueDescriptor | None:
+    acceleration = ctx.derivative(speed, order=1, max_gap_s=0.5)
+    if acceleration is None:
+        return None
+    return ValueDescriptor(output.id, acceleration, speed.timestamp)
+```
+
+Bind it using the existing API:
+
+```python
+from uuid import uuid4
+from dynalab_core.dlunit import DLUnit
+from dynalab_core.protocols.packets.handshake import SignalDescriptor
+
+acceleration_signal = SignalDescriptor(
+    id=uuid4(), name="Acceleration (m/s²)", type="number", origin="derived"
+)
+unit = DLUnit()
+unit.set_manifest(uuid4(), [speed_signal], acceleration_signal)
+unit.set_data(source_text)  # The unit.py source above.
+core.bind_derive_unit(dlunit=unit)
+# Optionally persist it with unit.write(directory, "acceleration.dlunit").
+```
+
+`speed_signal` must describe the existing speed input. If its values are in m/s,
+the output values are in m/s². Core does not infer physical units or rename
+signals automatically. Derivative outputs are ordinary derived signals: they
+can feed another DLUnit, be recorded, and be saved in a DLPak.
+
+## Function contract
+
+A function takes its `ValueDescriptor` inputs in manifest order, optionally one
+`DeriveContext`, then one final `SignalDescriptor` describing the output. Use
+the actual imported classes in annotations, without postponed/string
+annotations. The source must expose exactly one Python function; importing the
+context class is supported.
+
+Existing functions such as `process(a, b, output) -> ValueDescriptor` continue
+to work. To skip output, explicitly annotate `-> ValueDescriptor | None` and
+return `None`. This also works without a context. Returning `None` with the
+original `-> ValueDescriptor` annotation remains an invalid result.
+
+No placeholder is emitted when a function returns `None`: live values are not
+updated, recording gets no sample, and downstream units receive no event for
+that output. A previously published live output remains subject to its normal
+timeout; skipping an output does not explicitly clear it.
+
+An offline unit annotated to allow `None` may produce an empty series. Core logs
+`derive.offline_no_values` and skips units requiring that empty output, logging
+`derive.offline_upstream_empty`. Reprocessing still succeeds, and descriptors
+are retained for the empty outputs. Units that do not allow `None` retain the
+`DeriveError(code="no_values")` behavior if they execute but emit nothing.
+
+## `ctx.derivative(value, order=1, *, max_gap_s=None) -> float | None`
+
+- `value` is one of the function's supplied input descriptors.
+- `order` is an integer: 1, 2, or 3. Other values raise `ValueError`.
+- `max_gap_s` is an optional finite positive maximum interval between
+  consecutive samples in the derivative window. Invalid limits raise
+  `ValueError`. The default places no gap limit.
+- The result uses seconds: input units/s, input units/s², or input units/s³.
+- Warm-up requires 2, 3, or 4 distinct timestamps, respectively. Until then,
+  the result is `None`. A window containing a gap above `max_gap_s` also returns
+  `None`; estimates resume after enough samples arrive following that gap.
+- Repeated calls are read-only and return the same estimate. You can request
+  different orders for the same input in one invocation.
+
+The estimator differentiates the degree-`order` polynomial interpolating the
+most recent `order + 1` samples. For order 1 this is the usual backward secant:
+`(new_value - previous_value) / elapsed_seconds`. Orders 2 and 3 use the actual
+unequal sample intervals rather than repeatedly dividing by a nominal period.
+Timestamp differences are taken as integers before converting to seconds.
+
+These are numerical estimates without smoothing. They use only present and
+past samples, and higher orders can magnify measurement noise. Very uneven
+spacing and floating-point precision also limit accuracy. A non-finite
+computed estimate returns `None`.
+
+## History and alignment
+
+Core retains at most four samples per declared input, per context-enabled unit.
+The router advances history only when that input itself updates and queues a
+snapshot for each invocation. Later arrivals cannot alter an already queued
+context, and calls to `derivative()` do not advance history.
+
+Context-enabled units wait until all declared inputs have a sample at or before
+the triggering event's timestamp. Each input retains its own original timestamp.
+Only the most recent four accepted samples are available, so severely late
+events may no longer have a companion sample in history and are skipped.
+Equal or older timestamps for a signal are ignored by that unit's history and
+do not trigger its processing function. A newer non-finite value clears that
+signal's history and skips the invocation; finite samples restart warm-up.
+These rules do not remove or alter raw recording samples.
+
+When another input changes, a derivative of an unchanged input reuses that
+input's existing estimate. Its history does not gain another sample. If the
+output combines multiple inputs, choose its timestamp explicitly, for example
+`max(a.timestamp, b.timestamp)`. Companion values are retained in the bounded
+history; they do not expire through the realtime live-value timeout. Apply any
+additional age policy in the function using the supplied timestamps.
+
+Offline processing uses the same bounded history and causal alignment, with a
+fresh history on every pass. Functions without a context retain the original
+offline nearest-neighbor alignment, including possible future samples and
+timestamps assigned from the triggering event.
+
+Online and offline estimates match for the same chronological accepted input
+sequence. Offline processing sorts stored samples; online out-of-order arrivals
+can therefore differ. Queue overload can also change the observed sequence:
+router drops omit samples from history, while a unit queue drop omits that
+invocation but its sample remains in later history snapshots. Asynchronous
+multi-input derived chains can observe a different event ordering online.
+Offline processing does not drop samples due to queue capacity.
+
+Starting or stopping recording does not reset ongoing online history. If a
+recording begins midstream, its first online derivative outputs may depend on
+samples preceding the recording. Offline recalculation warms up from the
+recorded samples and can omit those early estimates.
+
+## Lifecycle
+
+Online histories are invalidated when Core changes mode, a source connector
+registers/disconnects, processing or playback data is replaced, or a playback
+control changes its generation (play/resume, pause, seek, or restart). Connector
+changes conservatively invalidate all units' histories. Queued requests and
+results from an older revision are discarded, including results completing
+after the reset. New data warms up the estimator again.
+
+Offline histories are local to the processing pass and never share online
+state. DLUnit archives save source and manifest, not history. DLPak and connector
+packet formats are unchanged. No numerical-library dependency is required.

docs/Protocol.md

@@ -83,7 +83,7 @@ packet classes.
 {
   "type": "dynalab_hello",
   "instance_id": "2f3c7f62-0a45-4d66-9f32-7fbeb92f9d36",
-  "core_version": {"type": "alpha", "major": 0, "minor": 1, "patch": 0},
+  "core_version": {"type": "alpha", "major": 0, "minor": 2, "patch": 1},
   "protocol_version": {"type": "alpha", "major": 0, "minor": 0, "patch": 1},
   "heartbeat_interval_ms": 1000,
   "heartbeat_timeout_ms": 5000

docs/README.md

@@ -12,4 +12,6 @@ This documentation is ordered in the following sections
 
 [**Protocol**](Protocol.md) - Documents the data ingress protocol, this is useful for developing connectors to pipe data into the DynaLab Core
 
+[**Processing and derivatives**](Processing.md) - DLUnit context API, first through third time derivatives, warm-up, and online/offline behavior
+
 [**Internal**](Internal/README.md) - Documents the inner working of DynaLab Core including data flow, internal API's, custom file types and much more