Derived units now use a DLUnit class similar to a DLPak to be easily
stored and loaded, the public API has been updated to either take a
DLUnit object or a Path where it reads the DLUnit from a *.dlunit file
The .dlunit file is a ZIP compressed archive containing two files, a
manifest.json which stores the units uuid, its input signals and its
output signal, and unit.py which is the python code for the unit, the
same rules apply (one python function with a specifically structured
signature) as when a string was passed in the old API
Errors have been added to have a DLUnitError similar to the DLPakError
as the DLUnit class is heavily inspired from the DLPak architecutre
Logging may still be primitive, this will be covered in the next AI
logging sweep to homogenize and clean up the logging aspect of things
All internal and public errors have been revised (GPT 5.6 Sol) to return
more detail without horrendously long error classes, this has reduced
the number of error types but each now carries a reason and a code for
differentiation if needed
PlaybackEngine has been implemented and is functional, core mode changes
what signal descriptors are returned to avoid duplicates, the playback
engine currently discards the derived signals that would be recorded in
the DLPak and derived signals are reprocessed live
Future will require the behavior listed above to be selectable so one of
two options are possible:
A - Derived signals are reprocessed live by the relevant derive units
(current behavior
B - Derived signals are played back directly from the PlaybackEngine,
this requires turning off the routing to the live derive units to avoid
duplicate values
This will also require the option to perform offline processing with
derive units to be able to create derived signals after the fact, this
will likely incur a new 'offline' mode in the core to gate things
correctly, this will likely be useful for heavier processing that cannot
be done live, filters that require a lookahead, or more precise derived
signals using interpolated signals for higher acurracy which also brings
the ability to offline process on a fixed timebase instead of following
either input signal to the derived signal
Derive units are now available in the core and has their relevant input
values routed and outputs are looped back into the core router
Fixed the dlpak manifest which had no signal descriptors
This contains the initial DeriveUnit implementation, including signature
validation, this initial implentation only supports ValueDescriptors in
the signature alongside the SignalDescriptor and returns a single
ValueDescriptor
The worker thread still needs to be defined and started but the offline
processing has already been validate to work as a first prototype
Implemented DLPak design with zip storage, DLPak consists of two files,
a manifest.json which provides details on the signals, time of recording
and currently blank comment field and data.csv which stores all the
signals as csv with the nanosecond timestamp and indexed against the
signal id
Further isolated json specifics to their own module under protocols/json
Moved global packet classes to their own protocols/packets module
Implemented ConnectorEndpoint and ConnectorRegistry as transport
agnostic connector endpoints for universal connector handling, exposed a
ConnectorRegistry API that will need to be made available to the
JsonServer for it to be able to register and unregister endpoints as it
opens and closes connections
Implemented a top down shutdown architecture where each object is
responsible for its children, this allows for a logical and heirachical
flow, each object has its own stop event, which when its stop method is
called, it sets and then awaits the stopped event to be set by any
worker threads etc, with a mandatory timeout to avoid hanging on
shutdown
Isolated the JsonServer to server.py inside protocols/json
Added specific errors for server startup timeout and server startup
failed
Refactored errors for Core to CoreError generic and refactored
StateError to CoreStateMismatchError derived from CoreError
Completed the initial implementation of the JsonServer class, handling
the worker thread that runs the async task, as well as the connection
handler callback
The current handler awaits readline to cleanly exit when the connection
is closed, no data ingress is happening yet
Next steps include gating the start procedure to wait for everything to
start proprerly in order to avoid having start exit early and async
tasks crashing later
h3cx