8 Commits
25 changed files with 9968 additions and 498 deletions

No files matched your search

+36
View File
@@ -0,0 +1,36 @@
# Changelog
## 0.2.2
- Use `UniLoader_Global` and `UniLoader_Local` nicknames for global and project libraries, preventing project entries from shadowing global managed libraries.
- Keep existing asset filenames and portable paths unchanged.
- Add `doctor --global` and `doctor --project FILE` to preview nickname repairs; `--fix` backs up and migrates registrations, managed symbol bindings, and project schematic/PCB references.
- Preserve unrelated registrations, reject occupied target nicknames before writes, and retain exact legacy nickname mappings for later project migrations.
- Stop imports using legacy managed nicknames with instructions to run doctor.
Validation: 62 automated tests, Clippy, formatting, and release build passed. Live KiCad integration requires manual verification.
## 0.2.1
- Fix high idle watcher CPU usage caused by directory scans generating filesystem open events and triggering further scans.
- Ignore read/open access events before queuing watcher work, while retaining write-close events and filesystem overflow recovery.
- Batch queued filesystem events into one scan while preserving periodic reconciliation and partial-download retries.
- Add a Linux idle CPU regression check alongside the download, rename, duplicate and restart integration checks.
Validation: 59 automated tests, Clippy, formatting, and release build passed. Local Linux idle profiling confirmed the watcher no longer continuously rescans its download directory.
## 0.2.0
- Import components into global or portable project libraries, creating and registering libraries when needed.
- Reuse confidently matching footprints and compare uncertain matches in native Rust dialogs.
- Discover possible open KiCad projects and choose a destination for watched downloads.
- Copy 3D models into managed libraries, repair model paths, and resolve ambiguous model choices.
- Keep pending imports and history, retry failures, and back up modified library files.
- Bundle egui/eframe dialogs and footprint previews in the executable, with rfd file selection.
- Document setup and a persistent user systemd service.
Validation: 58 automated tests, Clippy, formatting, and release build passed. Live desktop dialogs, portal file pickers, and KiCad integration still require manual verification.
## 0.1.0
- Initial component importer release.
Generated
+2698 -44
View File
File diff suppressed because it is too large. Load diff
+11 -1
View File
@@ -1,7 +1,8 @@
[package] [package]
name = "uniloader" name = "uniloader"
version = "0.1.0" version = "0.2.2"
edition = "2024" edition = "2024"
rust-version = "1.95"
[dependencies] [dependencies]
chrono = "0.4.44" chrono = "0.4.44"
@@ -10,3 +11,12 @@ serde = { version = "1.0.228", features = ["derive"] }
thiserror = "2.0.18" thiserror = "2.0.18"
toml = "1.0.6" toml = "1.0.6"
zip = "8.2.0" zip = "8.2.0"
anyhow = "1"
clap = { version = "4", features = ["derive"] }
fs2 = "0.4"
serde_json = "1"
sha2 = "0.10"
tempfile = "3"
walkdir = "2"
eframe = { version = "0.36.2", default-features = false, features = ["default_fonts", "glow", "wayland", "x11"] }
rfd = { version = "0.17.2", default-features = false, features = ["xdg-portal", "wayland"] }
+79 -29
View File
@@ -1,42 +1,92 @@
# UniLoader # UniLoader
A simple daemon that automatically imports any libraries into KiCad. It currently supports symbols and footprints from [SamacSys](https://componentsearchengine.com/), [UltraLibrarian](https://app.ultralibrarian.com/search), [SnapMagic](https://www.snapeda.com/home/) UniLoader imports KiCad components from downloaded ZIPs or extracted folders,
including [SamacSys](https://componentsearchengine.com/),
[UltraLibrarian](https://app.ultralibrarian.com/search) and
[SnapMagic](https://www.snapeda.com/home/). Run it manually or watch your download
folder throughout your graphical login session.
## Installation It supports symbols, footprints and referenced STEP/VRML models, with:
An automated install script is planned but currently the repo needs to be cloned, built and installed manually - Global (`UniLoader_Global`) and project-local (`UniLoader_Local`) library nicknames, created and registered automatically.
- A destination dialog offering detected KiCad project suggestions, Global and Browse.
- Conservative exact footprint reuse and comparison dialogs for uncertain matches.
- Broken model-path repair, explicit ambiguous model choices and missing-model warnings.
- Preserved source archives, pending import retries, backups and import history.
## Quick start
Linux requires Rust 1.95 or newer and the usual
desktop graphics libraries. The binary includes egui/eframe dialogs; rfd file
browsing uses the desktop's XDG portal. `notify-send` is optional. See the
[setup guide](docs/SETUP.md) for packages, configuration, recovery and the shipped
[systemd user service](systemd/uniloader.service).
```sh
cargo build --release --locked
install -Dm755 target/release/uniloader ~/.local/bin/uniloader
~/.local/bin/uniloader import /path/to/component.zip --project /path/to/Board/Board.kicad_pro --dry-run
~/.local/bin/uniloader import /path/to/component.zip --project /path/to/Board/Board.kicad_pro
~/.local/bin/uniloader watch
``` ```
git clone https://git.h3cx.dev/h3cx/UniLoader.git A default configuration is generated on first launch at
`${XDG_CONFIG_HOME:-~/.config}/uniloader/config.toml`. The existing minimal config
cd UniLoader remains supported:
cargo build -r
```
You will then find the binary under `target/release` you can run it manually or create a systemd service to automatically run it at boot
## Usage
To get started you will need to set up your config, to do so create a config file
```
mkdir ~/.config/uniloader
touch ~/.config/uniloader/config.toml
```
Then use your favorite text editor to open this config and paste in the minimal config
```
```toml
[libraries] [libraries]
path = "~/KiCad/libraries/" path = "~/KiCad/libraries"
# For global imports, choose your actual installed KiCad version if detection
# finds none or more than one:
# kicad_config = "~/.config/kicad/9.0"
``` ```
Now that you have set this up, the application should be ready to go, just head over to your favorite library website and download any component.
Upon downloading the first component from a given online library you will be required to add the library to KiCad. In KiCad open your symbol editor and head to `File->Add Library` then naviagate to the path you set in your config and add the library. Repeat this from the footprint editor for the footprints libraries. See [the complete configuration example](config/config_full.toml). Project
imports create libraries beside the `.kicad_pro` file using `${KIPRJMOD}` paths;
KiCad may need a library refresh or reopen. Missing models are reported and mean
that a component is not yet fully portable.
```sh
uniloader import /path/to/component.zip --global
uniloader import /path/to/extracted-folder --project /path/to/Board/Board.kicad_pro
uniloader pending
uniloader retry IMPORT_ID
uniloader discover
```
No-argument startup also runs the watcher. `--noninteractive` leaves unresolved
choices pending; `--force` on manual imports deliberately revisits a completed
source. Check an imported component in KiCad before using it in a board.
## Upgrade existing libraries to v0.2.2
Close KiCad and stop the watcher, then preview and apply the nickname migration:
```sh
uniloader doctor --global
uniloader doctor --global --fix
uniloader doctor --project /path/to/Board/Board.kicad_pro
uniloader doctor --project /path/to/Board/Board.kicad_pro --fix
```
Repeat the project commands for each existing project. Project repair also checks
and repairs the configured global libraries. Existing library filenames stay the
same. Changed files are backed up; see the [setup guide](docs/SETUP.md#nickname-migration-v022)
for migration scope and recovery. Reopen KiCad afterwards.
## Scope
This is practical Linux development tooling. Project discovery currently uses
process arguments and working directories as suggestions; check the selected
path. KiCad IPC discovery, tray controls, comparison overlays and automatic undo
are follow-ups. Legacy `.lib` conversion and CAD-format conversion are outside
the current release. The preview is approximate; inspect unusual footprints in
KiCad. The [specification](SPEC.md) records the intended behavior and delivery
priorities.
## Warranty ## Warranty
Use this program at your own risk, no warranty is provided for anything that it could do to your libraries, KiCad or operating system. If you encounter any issues feel free to open an issue. Use this program at your own risk, no warranty is provided for anything that it
could do to your libraries, KiCad or operating system. If you encounter any issues
feel free to open an issue. Licensed under [GPL-3.0-or-later](LICENSE).
+399
View File
@@ -0,0 +1,399 @@
# UniLoader: practical development specification
Status: initial usable release implemented (2026-10-11). This document retains
both the first-release requirements and subsequent improvements below.
Implemented: manual/watched imports, conservative footprint matching and desktop
comparison, process-based project suggestions, global/portable local libraries,
automatic library creation/registration, reference rewriting, model path repair,
pending/retry/history, backups, configuration and the systemd setup guide.
Deferred as planned: authoritative IPC discovery, comparison overlays, tray
controls and guarded undo. Remembered destinations currently last for the running
session; fixed project/global preferences are configured in TOML. Desktop dialogs
use Rust egui/eframe compiled into the executable; rfd uses the XDG desktop portal
for file browsing. Automated fixture tests cover file behavior; live KiCad and
desktop acceptance still need verification in a graphical session.
## Goal and implementation approach
Keep the existing download-and-import workflow, prevent duplicate footprints,
and support portable project libraries alongside global libraries. Ask the user
where a component should go, using open KiCad projects as suggestions.
This is personal development tooling. Favor a small, useful implementation over
a polished application: Linux first, simple dialogs, ordinary files for state,
and a few focused tests. No database, plugin framework, web backend, custom GUI
design system, or full transaction engine is required. Small Rust modules and
egui/eframe dialogs are sufficient. Keep the UI in the same installed binary;
launch an internal dialog process when needed to give its window event loop a
main thread while the watcher continues discovering downloads. Use rfd for file
selection through the desktop portal, with X11 and Wayland support and OpenGL
rendering. No scripting runtime or companion scripts are required.
Preserving originals, avoiding silent overwrites, and keeping footprint links
correct are still required.
## Existing behavior and prerequisites
The current Rust daemon watches `~/Downloads`, recognizes SamacSys,
UltraLibrarian and SnapMagic archives, and writes provider-specific libraries
under one configured directory. Preserve those providers and existing config.
Fix these problems before building new decisions on top of the importer:
- Expand `~` in configured paths and support XDG config/data/cache locations.
- Replace the hand-written symbol scanner with a quote/escape/Unicode-aware
S-expression reader. Preserve unknown fields and original format versions;
do not relabel newer content as an older KiCad format.
- Wait for downloads to finish: handle creation and rename/move events, ignore
temporary downloads, debounce duplicates, and retry temporarily invalid ZIPs
for a bounded period. File-size stability alone is insufficient.
- Preserve source ZIPs by default. Extract into unique temporary directories;
clean these after success. Failed imports must remain retryable.
- Include the source path and actual cause in errors rather than silently
dropping failures. One bad archive must not stop the watcher.
- Stage output, back up changed libraries/tables, and replace individual files
atomically. On partial failure report what changed and retain recovery files.
Full crash-atomic multi-file transactions are outside the initial scope.
- Prevent simultaneous writers, including manual imports while the watcher runs.
## Import workflow
1. Detect a completed supported download and extract its component assets.
2. Resolve the destination, prompting unless configuration or an explicit CLI
option already determines it.
3. Find existing footprint candidates in the destination's effective libraries.
4. Resolve exact matches automatically and ask about uncertain matches.
5. Prepare symbols, footprints, models, reference changes and library-table edits.
6. Validate the plan, write it, then report success or an actionable failure.
Use one common pipeline after provider extraction. A modest split into config,
sources, KiCad parsing, matching, destinations, dialogs and watcher modules is
enough. Provider handlers should discover assets, not independently install them.
Keep a small persisted queue with pending/failed/completed states and source
hashes. Closing a dialog means “later”, not consent. A pending comparison should
not block discovery or independent imports. Serialize writes and show one dialog
at a time. Remember completed imports by source content and destination so repeat
filesystem events are harmless; allow deliberate reimport.
## Destination prompt and open-project discovery
Default watcher policy: ask “Import into project or global libraries?” once per
archive. Show component names, the project name and full directory path.
- One detected project: preselect local import to it; retain Global as an option.
- Several detected projects: show a selector, deduplicating schematic/PCB views
belonging to the same project. Do not silently guess which project is current.
- None detected: offer Global and Browse for a `.kicad_pro` file.
- Offer Import, Later and Skip, plus “remember for this session/project”.
- Remembering a project preference suppresses prompts only when that project is
unambiguously the destination. Forget stale selections when a project closes;
an explicitly configured fixed target remains usable when KiCad is closed.
- Destination preferences do not suppress uncertain-footprint comparisons.
Discovery should use KiCad IPC where available. Enumerate accessible running
instances, query open documents and resolve their project directories. Probe
capabilities instead of assuming all KiCad releases expose the same API.
IPC may need enabling in KiCad; an independent watcher must discover/connect to
instances rather than assume plugin-provided environment variables are present.
KiCad 9/10 IPC is PCB-editor focused; schematic IPC support is documented for
KiCad 11. A project-manager-only session therefore needs a fallback. For a quick
Linux implementation, inspect KiCad process command lines/working directories
for existing project files and label these as suggestions. This is best effort:
startup arguments can be stale, window titles are not reliable full paths, and
unsaved documents may have no usable project. Always retain Browse. Process-based
discovery may ship before IPC if that gets a usable version out sooner.
Freeze the selected destination for each import. A later window/project change
must not redirect an already reviewed import. Missing IPC, an unavailable GUI or
ambiguous detection must leave a pending choice, never silently choose Global.
## Global and project-local libraries
Keep the existing `[libraries].path` as the global storage root. Existing
provider-specific libraries can remain; consolidating or migrating them is not
required. Reuse footprints across providers regardless of storage grouping.
Default project layout:
```text
project/
project.kicad_pro
sym-lib-table
fp-lib-table
libraries/
UniLoader.kicad_sym
UniLoader.pretty/
models/
```
- Add/merge `UniLoader_Global` or `UniLoader_Local` entries into the appropriate symbol and footprint tables.
Preserve unrelated entries and never silently repoint an occupied nickname.
- Project URIs use `${KIPRJMOD}/libraries/...`; global URIs use stable absolute
paths. Let users select the KiCad configuration/version directory if ambiguous.
- Resolve project/global precedence and KiCad path variables when searching
registered libraries. Allow extra search paths; report unreadable libraries and
unresolved variables. Do not claim a complete search when some roots failed.
- Rewrite each imported symbol's `Footprint` property to its final registered
`LibraryNickname:FootprintName`. Match associations explicitly; never attach
every symbol to the first footprint in a multi-component archive.
- Default local mode is portable: reuse an existing project footprint in place;
copy a selected global/external footprint into the project before referencing it.
- Copy referenced external 3D models when available and rewrite their paths to
project-relative paths. Preserve model offsets, rotations and scale. List
missing models or other unresolved dependencies in the import result.
- Do not copy every global library. Only bring in assets needed by the import.
- Do not change existing schematic/PCB instances as a side effect of an import.
If symbol or footprint importing is disabled, honor the existing options. Do not
create a dangling footprint link: resolve it to an available footprint, retain an
already valid link, or leave the component pending with an explanation.
### Automatic library creation
An empty destination must work without manual library setup. Create missing
parent directories, a valid `.kicad_sym` library, the `.pretty` footprint
directory and model storage as needed for the enabled asset types. Create missing
`sym-lib-table` and `fp-lib-table` files in the selected scope and register the
new libraries with matching nicknames. Generate symbols using a supported format
version appropriate to the imported content, not a hardcoded downgrade.
If libraries already exist, merge into them using the conflict policies above.
If a path exists but is not the expected library/directory type, report it instead
of replacing it. Choose a fresh nickname or ask if the intended nickname is
already registered to another library. Global setup must use the selected KiCad
version's configuration directory; project setup must stay inside the selected
project. Missing libraries should never require a trip through KiCad's “Add
Library” dialogs. A subsequent editor refresh/reopen may still be needed.
### 3D model import and broken-path repair
Support STEP (`.step`/`.stp`) and VRML (`.wrl`) assets bundled with supported
provider archives or manual folder imports. Search the extracted bundle
recursively: models need not sit beside their footprints. Import only referenced
models by default. Handle multiple model references per footprint.
For each reference, first try its valid resolved location, then resolve against
the source bundle and configured model search roots. Handle provider-specific
absolute paths, Windows separators/drive prefixes, renamed extraction roots,
and KiCad path variables. A unique matching relative suffix or filename is an
acceptable quick repair strategy. If several files fit, compare content to
collapse identical copies; otherwise ask which model to use rather than choosing
arbitrarily. Do not crawl the whole home directory or fetch models online.
Copy resolved external models into the selected destination's model storage.
Use `${KIPRJMOD}/libraries/models/...` for portable projects and stable absolute
paths for global libraries. Rewrite the footprint's model references, preserving
offset, rotation, scale and other supported settings. Resolve model filename
collisions by content: identical files can share storage; different files need
distinct names. Include models when copying an existing external footprint into
a portable project, not just when importing a fresh archive.
Preserve valid embedded models if present; do not require embedding support for
the initial release. If a model cannot be found, still allow the symbol/footprint
import with a visible warning and mark its model dependency unresolved. Keep the
original reference and record the failure so it can be repaired later. Do not
claim the result is fully portable while external dependencies remain unresolved.
The result should list copied, repaired, reused and missing models.
Add an `options.models` switch, defaulting to enabled, plus configurable model
search roots. Disabling model import must be reported as a portability limitation
when it leaves external references. An initial implementation that repairs common
provider paths and reports difficult cases is sufficient; complete CAD-format
conversion and STEP/VRML geometry analysis are outside scope.
## Footprint matching and duplicate prevention
Search configured/registered libraries and other assets in the same import plan.
An initial in-memory index rebuilt at startup is sufficient; persistent indexing
and sophisticated incremental updates can wait. Recheck selected content before
writing if it may have changed while the dialog was open.
Default decisions:
| Classification | Action |
| --- | --- |
| No plausible candidate after a successful search | Import automatically |
| Conservatively normalized exact match | Reuse automatically |
| Same name, different content | Ask; never overwrite automatically |
| Similar geometry or incomplete comparison | Show candidate comparison |
| Search failure that prevents a reliable decision | Leave pending with details |
“Exact” means parsed content equality after a small, documented normalization
allowlist, such as whitespace, the root footprint name and generated identifiers.
Retain pad numbering, geometry, layers, drills, mask/paste settings, attributes,
graphics and model semantics. Handle model paths in their library context; equal
relative strings from different libraries do not prove equal models. Unknown
constructs must be preserved and must not be discarded to manufacture equality.
An initial stricter matcher that misses some duplicates is acceptable.
Candidate similarity can start with simple rules: pad count/number set, pad types,
positions, sizes, shapes, layers, drill sizes and overall dimensions. Names and
package hints help rank candidates but do not establish equivalence. Equal pads
with different courtyard, mask or model content remain a comparison case.
Rotation-invariant matching and advanced geometry analysis can wait.
When several exact matches exist, prefer a destination-local copy, then a stable
configured library order. In portable mode, copy an external match locally.
The comparison dialog must show:
- Incoming footprint and selectable candidates with library/name/path.
- Side-by-side previews at the same scale, with pad numbers and dimensions.
- A short difference list, particularly numbering, pitch, drills and pad sizes.
- Use selected existing footprint, Import separately, and Later/Skip actions.
A simple SVG preview in a helper window is acceptable. Unsupported drawing
elements should be indicated; users can open the files in KiCad for inspection.
Overlay rendering is desirable after the basic comparison works. Do not require
a complete KiCad rendering engine for the first release.
Separate imports need a deterministic collision-free name and corresponding
symbol-reference updates. Remember user mappings using incoming and selected
content fingerprints; invalidate them if either footprint changes.
Expose matching enable/disable, search roots, exact-match policy, similarity
tolerances, ambiguous-match policy and remembered mappings in configuration.
Disabling matching must still prevent silent filename overwrites. Headless
policies can keep separate copies or skip; default ambiguity handling is pending.
## CLI, configuration and small usability additions
Keep no-argument watcher startup working. Add these proposed commands:
```text
uniloader watch
uniloader import component.zip --global
uniloader import component.zip --project /path/to/project.kicad_pro
uniloader import extracted-component/ --project /path/to/project.kicad_pro
uniloader import component.zip --project /path/to/project.kicad_pro --dry-run
uniloader pending
uniloader retry <import-id>
```
`--dry-run` reports proposed destinations, reuse/copy decisions, unresolved
choices and table/reference changes without writing libraries or remembering
decisions. Add CLI configuration override and a noninteractive mode that cannot
hang waiting for dialogs.
Configuration should cover watch directories, global root, destination policy
(`ask`, `global`, `project`), fixed project, portable local mode, enabled asset
types, matching policies, GUI availability and source retention. Preserve the
existing minimal TOML configuration with sensible defaults. Update the complete
example alongside implementation rather than documenting speculative keys.
Treat symbol-name collisions explicitly: skip identical symbols; ask or create
a separate correctly renamed symbol for different content. Preserve internal
unit names/inheritance when renaming. No automatic destructive replacement.
Maintain a small import history recording source, destination, decisions and
changed files/backups. A tray destination selector and “undo last import” are
follow-ups. Undo must refuse to overwrite subsequent edits; manual backup restore
instructions are sufficient initially. No full library-management UI is needed.
## Setup guide and systemd user service
Implementation must include `docs/SETUP.md`, linked from the README, and a shipped
example `systemd/uniloader.service`. The guide must document:
1. Linux build/runtime dependencies, supported KiCad versions, graphics/portal needs,
and any IPC configuration. State limitations of process-based discovery.
2. `cargo build --release --locked` and installation of the executable to
`~/.local/bin/uniloader`; no root installation required.
3. Minimal and full configuration, watched directories, library search scope,
global/local examples, model search/path repair, automatic library creation,
and how remembered decisions are reset.
4. A foreground smoke test before enabling automatic startup.
5. User service installation, startup, logs, restart, disable and removal.
6. Desktop session environment troubleshooting for X11/Wayland and how to
resume pending choices if dialogs could not be displayed.
7. Updating the binary/config, restoring backups, and known KiCad library refresh
limitations. Explain when a library/editor reload is needed.
Use a systemd **user** service, not a system service or root daemon. The intended
meaning of “always running” is throughout the user's graphical login session.
Start from the graphical session so dialogs have the right display environment.
Initial unit template, to verify with the final CLI:
```ini
[Unit]
Description=UniLoader KiCad component importer
PartOf=graphical-session.target
After=graphical-session.target
[Service]
Type=simple
ExecStart=%h/.local/bin/uniloader watch
Restart=on-failure
RestartSec=3
[Install]
WantedBy=graphical-session.target
```
The guide must provide copy/install commands and explain:
```sh
systemctl --user daemon-reload
systemctl --user enable --now uniloader.service
systemctl --user status uniloader.service
journalctl --user -u uniloader.service -f
systemctl --user restart uniloader.service
systemctl --user disable --now uniloader.service
```
Some compositors do not manage `graphical-session.target` or propagate display
variables automatically. Document a session-start hook that imports the actual
session's `DISPLAY`, `WAYLAND_DISPLAY`, `XAUTHORITY` and relevant desktop variables
when set, then starts the service. Do not hardcode another session's display or
tell users that enabling the unit alone fixes all desktops.
Running before login/after logout is optional later work. Explain that lingering
does not provide a graphical session: unattended operation would require a
configured noninteractive destination and persistent pending decisions.
## Delivery order and acceptance
### First usable release
Fix parser/path/download issues; preserve archives; introduce the common import
plan; implement manual global/project imports, table registration and reference
rewriting. Add the destination popup with best-effort discovery and Browse.
Add strict exact matching and basic candidate comparison. Ship setup docs and
the user service. Basic model copying belongs here so portable mode is useful;
report unsupported/missing dependencies clearly.
### Subsequent improvements
Broader IPC discovery, refined similarity rules, comparison overlays, tray
controls, history UI/guarded undo, and more complete model handling. Avoid making
these prerequisites for a working first release.
Use a small fixture set, with at least one representative archive per provider.
Focused automated checks should cover quoted parentheses/Unicode, reference
rewriting, exact reuse, same-name conflicts, table preservation and portable
paths. Manual acceptance should demonstrate:
- Reimporting the same archive creates no duplicate footprint or symbol.
- An exact match from another provider is reused without a comparison prompt.
- A near match opens comparison; choosing either action produces valid links.
- Zero/one/multiple discovered projects produce the specified destination choices.
- Copying a project elsewhere and opening it without the original custom global
libraries resolves imported symbols, footprints and available copied models.
- Importing into an empty destination creates and registers valid libraries.
- A bundled model with an obsolete absolute/Windows path is found, copied and
linked correctly; verify it in KiCad's 3D viewer. Multiple different models
with the same basename are not silently confused.
- Missing models are reported without discarding the usable symbol/footprint.
- Partial downloads, invalid archives and closed dialogs remain recoverable.
- The user service restarts after failure and shows dialogs in a real desktop
session. Verify with KiCad; file-only tests do not establish live integration.
## References
- [KiCad IPC capabilities, connection and version limitations](https://dev-docs.kicad.org/en/apis-and-binding/ipc-api/for-addon-developers/)
- [KiCad footprint libraries and project-relative paths](https://docs.kicad.org/9.0/en/pcbnew/pcbnew.html#_path_variable_substitution)
- [KiCad S-expression format](https://dev-docs.kicad.org/en/file-formats/sexpr-intro/)
- [systemd desktop-session integration](https://github.com/systemd/systemd/blob/main/docs/DESKTOP_ENVIRONMENTS.md)
+59
View File
@@ -0,0 +1,59 @@
# All active values below match the defaults. Paths accept ~ and ~/ prefixes.
# Relative paths resolve from the process working directory: prefer absolute or
# home-relative paths when running the systemd user service.
[libraries]
# Existing provider libraries under this root are included in footprint search.
path = "~/KiCad/libraries"
# Select your actual installed KiCad configuration/version directory for global
# table registration when automatic detection finds none or multiple versions.
# kicad_config = "~/.config/kicad/9.0"
# Extra recursive footprint/model search roots. Only add existing relevant dirs.
search_paths = []
model_search_paths = []
# search_paths = ["~/KiCad/other-libraries"]
# model_search_paths = ["~/KiCad/models"]
[libraries.variables]
# Optional KiCad path variables, used before environment values. Match the
# variable names/version in your libraries and use actual absolute paths.
# KICAD9_3DMODEL_DIR = "/usr/share/kicad/3dmodels"
# MY_COMPONENT_MODELS = "/absolute/path/to/models"
[options]
symbols = true
footprints = true
models = true
# Keep downloaded ZIPs after successful import.
preserve_source = true
[watch]
directories = ["~/Downloads"]
# Require stable metadata, then validate ZIP contents. Invalid downloads retry
# for retry_seconds; keep retry_seconds >= settle_seconds.
settle_seconds = 2
retry_seconds = 60
[destination]
# ask: show Project/Global/Browse; global: fixed global storage;
# project: fixed project file, requiring project below.
policy = "ask"
# project = "~/projects/Board/Board.kicad_pro"
# Copy selected external footprints/models into project storage.
portable = true
[matching]
enabled = true
# reuse: automatically reuse conservative exact matches; ask: review matches.
exact = "reuse"
# ask: comparison dialog or pending; separate: import separately; skip: skip
# components needing comparison. None of these silently overwrites a footprint.
ambiguous = "ask"
# Geometric similarity tolerance in millimetres.
tolerance = 0.15
[ui]
# Native Rust dialogs require a graphical session and working OpenGL; Browse
# uses the XDG desktop portal. With false, choices stay pending unless an
# explicit destination/ambiguity policy resolves them.
enabled = true
+353
View File
@@ -0,0 +1,353 @@
# UniLoader setup (Linux)
UniLoader imports component ZIPs or folders into global KiCad libraries or a
portable project. It watches downloads during your graphical login session and
asks where to import each component. It creates missing libraries and registers
them in the selected scope automatically.
## Requirements and installation
Install Rust 1.95 or newer and your distribution's
C build tools and `pkg-config`. Desktop dialogs use Rust
[egui/eframe](https://docs.rs/eframe/latest/eframe/) with the OpenGL renderer and
both Wayland and X11 support. File browsing uses
[rfd's XDG desktop portal backend](https://docs.rs/rfd/latest/rfd/#linux--bsd-backends).
Install `xdg-desktop-portal` and the backend appropriate for your desktop
(for example, KDE or GTK). A working OpenGL driver, display session and session
D-Bus are required. `notify-send` is optional for desktop notifications.
The UI is compiled into `uniloader`; there are no companion scripts to install.
The watcher starts another instance of the same executable internally for each
dialog, keeping the window event loop on its own main thread.
On Debian/Ubuntu, a starting set of build packages is `build-essential pkg-config
libxcb-render0-dev libxcb-shape0-dev libxcb-xfixes0-dev libxkbcommon-dev
libwayland-dev`; other distributions provide corresponding development packages.
For optional notifications install `libnotify-bin` on Debian/Ubuntu or `libnotify`
on Arch Linux. The portal backend can bring its own toolkit dependencies; UniLoader
does not link a GTK UI. See the [upstream eframe build notes](https://github.com/emilk/eframe_template/#testing-locally)
if compilation reports a missing system library.
Build from the repository directory:
```sh
cargo build --release --locked
install -Dm755 target/release/uniloader ~/.local/bin/uniloader
~/.local/bin/uniloader --help
```
Add `~/.local/bin` to your shell's `PATH` to use the shorter `uniloader` command.
Do not run the importer as root.
The importer targets KiCad's modern `.kicad_sym`, `.kicad_mod` and `.kicad_pro`
formats. It preserves source format versions; the installed KiCad must support
those versions. Legacy `.lib` symbol conversion is not included. Verify your
actual component in KiCad before relying on a library for a board design.
This release discovers possible open projects by inspecting Linux KiCad process
arguments and working directories. These are **suggestions**, not an authoritative
active-window selection: startup arguments can be stale, editor navigation can
change documents, and some launchers hide the file argument. IPC discovery is a
follow-up; no IPC configuration is required for this release. Always check the
project path in the destination dialog, or use Browse / `--project`.
## Configuration
The default configuration is `${XDG_CONFIG_HOME:-~/.config}/uniloader/config.toml`.
A missing configuration is generated with defaults on first launch. You can
select another file with `--config /absolute/path/config.toml`. Existing
`[libraries].path` and `[options]` settings remain supported. `~` expands in paths;
relative configuration paths resolve from the process working directory, so use
absolute or home-relative paths for a service.
A minimal setup is:
```toml
[libraries]
path = "~/KiCad/libraries"
# Select your installed KiCad version for global library registration:
kicad_config = "~/.config/kicad/9.0"
```
Use your actual KiCad version directory. When a single installed configuration
can be identified, UniLoader can discover it. If none or several are found,
set `kicad_config` explicitly before importing globally. Project imports create
project-local tables and do not require a global table.
The complete example is in `config/config_full.toml`. Available settings are:
```toml
[libraries]
path = "~/KiCad/libraries"
kicad_config = "~/.config/kicad/9.0"
search_paths = []
model_search_paths = ["~/KiCad/models"]
# Optional KiCad variables not already supplied by your environment:
[libraries.variables]
# KICAD9_3DMODEL_DIR = "/usr/share/kicad/3dmodels"
[options]
symbols = true
footprints = true
models = true
preserve_source = true
[watch]
directories = ["~/Downloads"]
settle_seconds = 2
retry_seconds = 60
[destination]
policy = "ask" # ask, global, project
# project = "/absolute/path/Board/Board.kicad_pro"
portable = true
[matching]
enabled = true
exact = "reuse" # reuse, ask
ambiguous = "ask" # ask, separate, skip
# Pad-position comparison tolerance, in millimetres:
tolerance = 0.15
[ui]
enabled = true
```
`destination.policy = "project"` requires a fixed `project` path. A fixed target
can be used when KiCad is closed. The default `ask` offers detected projects,
Global and Browse. Remembering a choice applies to the running session; restart
UniLoader to clear it. Footprint reuse mappings are persisted by content; to clear
those mappings, stop UniLoader and remove only the content-mapping JSON files in its data
directory’s `decisions/` subdirectory, then restart. Explicit fixed configuration persists until edited.
Matching searches effective registered libraries and configured `search_paths`.
Do not add an enormous recursive root such as your entire home directory.
Missing/unreadable search locations are reported rather than silently treated
as proof that no duplicate exists. Exact footprint reuse is conservative;
similar footprints or conflicting names prompt for comparison. The preview
shows both footprints at the same scale, numbered pads, bounds and differences.
It approximates some shapes and omits text, drill holes and 3D geometry; the
window reports limitations. Inspect source files in KiCad when necessary.
`ambiguous = "separate"` or `"skip"` permits an explicit noninteractive policy;
the default keeps unanswered choices pending.
## Foreground smoke test
Start with an expendable test project and a known downloaded component:
```sh
uniloader discover
uniloader import /path/to/component.zip --project /path/to/Test/Test.kicad_pro --dry-run
uniloader import /path/to/component.zip --project /path/to/Test/Test.kicad_pro
uniloader import /path/to/component.zip --global --dry-run
uniloader watch
```
A manual import also accepts an extracted component folder. No-argument startup
runs the watcher. Dry run reports choices and intended library changes without
writing libraries; unresolved decisions may need an interactive import.
Use `--noninteractive` to avoid opening dialogs. A pending choice is not consent.
Project imports create `libraries/UniLoader.kicad_sym`,
`libraries/UniLoader.pretty/`, and model storage as needed, and merge registration
entries into `sym-lib-table` and `fp-lib-table` beside the `.kicad_pro` file.
Project registration uses the nickname `UniLoader_Local`; global registration uses
`UniLoader_Global`. Library filenames remain `UniLoader.kicad_sym` and
`UniLoader.pretty` in both scopes. Project paths use `${KIPRJMOD}`. Portable mode copies selected external footprints
and available referenced models into the project. Global imports use the
configured library root and register entries in the selected KiCad configuration.
Existing unrelated table entries are preserved. Existing schematic and PCB
instances are not automatically replaced by a library import.
Open the imported symbol, footprint and model in KiCad. Refresh/reopen the
library browser or editor if KiCad has cached the table/library. For an existing
placed component, use KiCad's normal update-from-library operation when you want
it updated. Copy the test project to another directory and verify its imported
assets again to check portability.
The watcher handles component ZIP downloads (including common SamacSys,
UltraLibrarian and SnapMagic layouts); use a manual folder import for extracted
assets. Source archives are preserved by default. Incomplete or invalid ZIPs
are retried for the configured period, then remain recoverable rather than
stopping subsequent downloads.
## Nickname migration (v0.2.2)
Before upgrading existing libraries, close KiCad and stop the watcher:
```sh
systemctl --user stop uniloader.service
uniloader doctor --global
uniloader doctor --global --fix
uniloader doctor --project /path/to/Board/Board.kicad_pro
uniloader doctor --project /path/to/Board/Board.kicad_pro --fix
systemctl --user start uniloader.service
```
Without `--fix`, doctor previews the files it would change. `--global` checks the
configured global tables and managed symbol library. `--project` checks those
plus the selected project's tables, managed symbol library, and `.kicad_sch` and
`.kicad_pcb` files recursively inside its directory (including subsheets). Run it
for every existing project; projects elsewhere are not automatically discovered
or modified. External subsheets must be migrated separately. Set
`libraries.kicad_config` to the correct version directory when needed.
Doctor identifies managed registrations by their resolved library paths, preserves
unrelated registrations and metadata, and refuses an occupied target nickname.
It updates symbol footprint properties, schematic library IDs and cached symbol
IDs, and placed PCB footprint IDs according to the original project-over-global
resolution. An old ambiguous ID keeps that original resolution; doctor cannot
recover a different intended library. Asset filenames and model paths stay the
same. Changed S-expression files are reserialized, so whitespace may change.
Repairs use the import writer lock, atomic file replacement, and the normal
backup/history mechanism. The command prints backup paths under
`${XDG_DATA_HOME:-~/.local/share}/uniloader/backups/`; `uniloader pending --all`
also lists changed files and backups. Exact old nicknames (including collision
suffixes) are retained under `uniloader/nickname-migrations/` in the same data
root so later project repairs can update global references. Keep this migration
history until all projects are repaired. Repeated repairs leave unchanged files
alone. All files are parsed and collisions checked before writing, but a disk
failure can still interrupt a multi-file repair; restore the listed backups if
needed. Reopen KiCad after repair to reload its cached tables.
Imports into a destination still using an old managed nickname stop with a doctor
instruction. Repair that destination before resuming imports.
## 3D models and broken paths
STEP (`.step`/`.stp`) and VRML (`.wrl`) models are copied when referenced by an
imported footprint. Resolution tries existing paths, bundle files and configured
model search roots. Windows/provider absolute paths and obsolete extraction
roots can be repaired through unique relative suffixes or filenames. Different
matching model files require a selection rather than an arbitrary choice.
Model transforms remain attached to the reference. Project references point to
`${KIPRJMOD}/libraries/models/...`; global references use absolute storage paths.
Missing models leave the usable component import available with warnings and
retain the original reference. Such an import is not fully portable. Adding the
missing model to a search root and deliberately reimporting with `--force` can
repair the dependency. Disabling `options.models` may also leave external paths.
UniLoader does not download models or convert CAD formats.
## Pending imports and recovery
```sh
uniloader pending
uniloader pending --all # include completed/skipped imports
uniloader retry IMPORT_ID
uniloader retry IMPORT_ID --dry-run --noninteractive
```
Closing a dialog or choosing Later leaves a pending import; it does not choose
Global, overwrite a file, or guess a model. Retry after fixing the configuration,
restoring the display environment or obtaining a missing asset. Other downloads
can continue through the queue. `--force` deliberately reimports a completed
source when you need to revisit it.
Import state/history lives beneath `${XDG_DATA_HOME:-~/.local/share}/uniloader`;
temporary extraction uses `${XDG_CACHE_HOME:-~/.cache}/uniloader`.
Changed files have recovery backups recorded by the import result/history.
Stop the watcher before manually restoring a backup, compare the current file
with the backup, and copy back only the intended version. A backup may predate
later manual edits; do not blindly restore every file. Restart KiCad or refresh
its libraries afterwards. Automatic undo is not included.
## Run continuously with a systemd user unit
First verify the foreground test above, then install the shipped service:
```sh
install -Dm644 systemd/uniloader.service ~/.config/systemd/user/uniloader.service
systemctl --user daemon-reload
systemctl --user enable --now uniloader.service
systemctl --user status uniloader.service
journalctl --user -u uniloader.service -f
```
The service uses `%h/.local/bin/uniloader watch` and the default configuration.
It starts with `graphical-session.target`, stops with that target, and restarts
after unexpected failure. “Always running” means while logged into your desktop.
To use a nondefault config, run `systemctl --user edit uniloader.service` and add:
```ini
[Service]
ExecStart=
ExecStart=%h/.local/bin/uniloader --config %h/.config/uniloader/work.toml watch
```
Then reload/restart:
```sh
systemctl --user daemon-reload
systemctl --user restart uniloader.service
```
### Display/session troubleshooting
If imports remain pending and logs say dialogs are unavailable, try a manual
interactive import from your desktop terminal and check the reported display or
OpenGL error. Check the service manager environment with
`systemctl --user show-environment`; it needs the real session's display variables
and session D-Bus. File browsing also needs a running desktop portal and a backend
that supports file selection; check `journalctl --user -u xdg-desktop-portal` if
Browse fails. Never invent a `DISPLAY=:0` or copy another user's Xauthority.
Some window managers/compositors do not activate `graphical-session.target` or
import display variables. Put this shell hook in your compositor's session-start
configuration, after the graphical session exists:
```sh
#!/bin/sh
for variable in DISPLAY WAYLAND_DISPLAY XAUTHORITY XDG_RUNTIME_DIR XDG_CURRENT_DESKTOP XDG_SESSION_TYPE DBUS_SESSION_BUS_ADDRESS; do
if printenv "$variable" >/dev/null 2>&1; then
systemctl --user import-environment "$variable"
fi
done
systemctl --user start uniloader.service
```
If the service was already running with a stale environment, restart it after
importing variables, then retry pending imports. For compositors without target
lifecycle integration, add `systemctl --user stop uniloader.service` to your
session-exit hook. Enabling a unit alone cannot supply a missing GUI session.
The initial environment comes from the session startup hook, not a hardcoded
service `Environment=` line.
Lingering (`loginctl enable-linger`) does not create a graphical session and is
not needed here. Operation before login or after logout requires a configured
noninteractive destination and explicit ambiguity policy; unanswered decisions
will remain pending.
### Update, stop and remove
After updating the source:
```sh
cargo build --release --locked
systemctl --user stop uniloader.service
install -Dm755 target/release/uniloader ~/.local/bin/uniloader
systemctl --user start uniloader.service
```
Keep your existing config; compare it with the updated complete example for new
settings. A service restart reloads configuration.
To disable automatic startup:
```sh
systemctl --user disable --now uniloader.service
```
To remove the service and executable:
```sh
rm ~/.config/systemd/user/uniloader.service
systemctl --user daemon-reload
rm ~/.local/bin/uniloader
```
Remove any compositor hooks or service overrides you added as well. Imported
libraries, configuration, source archives and history are retained. Only remove
those separately when you are sure they are no longer needed.
+193
View File
@@ -0,0 +1,193 @@
use anyhow::{Context, Result, bail};
use serde::{Deserialize, Serialize};
use std::{
collections::BTreeMap,
env, fs,
path::{Path, PathBuf},
};
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
#[derive(Default)]
pub struct Config {
pub libraries: Libraries,
pub options: Options,
pub watch: Watch,
pub destination: DestinationConfig,
pub matching: Matching,
pub ui: Ui,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct Libraries {
pub path: PathBuf,
pub kicad_config: Option<PathBuf>,
pub search_paths: Vec<PathBuf>,
pub model_search_paths: Vec<PathBuf>,
pub variables: BTreeMap<String, String>,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct Options {
pub symbols: bool,
pub footprints: bool,
pub models: bool,
pub preserve_source: bool,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct Watch {
pub directories: Vec<PathBuf>,
pub settle_seconds: u64,
pub retry_seconds: u64,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct DestinationConfig {
pub policy: String,
pub project: Option<PathBuf>,
pub portable: bool,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct Matching {
pub enabled: bool,
pub exact: String,
pub ambiguous: String,
pub tolerance: f64,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(default)]
pub struct Ui {
pub enabled: bool,
}
pub fn home() -> PathBuf {
env::var_os("HOME")
.map(PathBuf::from)
.unwrap_or_else(|| PathBuf::from("."))
}
pub fn xdg(name: &str, fallback: &str) -> PathBuf {
env::var_os(name)
.map(PathBuf::from)
.filter(|p| p.is_absolute())
.unwrap_or_else(|| home().join(fallback))
}
pub fn data_dir() -> PathBuf {
xdg("XDG_DATA_HOME", ".local/share").join("uniloader")
}
pub fn cache_dir() -> PathBuf {
xdg("XDG_CACHE_HOME", ".cache").join("uniloader")
}
pub fn expand(p: &Path) -> PathBuf {
let s = p.to_string_lossy();
if s == "~" {
home()
} else if let Some(rest) = s.strip_prefix("~/") {
home().join(rest)
} else if p.is_absolute() {
p.to_owned()
} else {
env::current_dir().unwrap_or_else(|_| home()).join(p)
}
}
impl Default for Libraries {
fn default() -> Self {
Self {
path: home().join("KiCad/libraries"),
kicad_config: None,
search_paths: vec![],
model_search_paths: vec![],
variables: BTreeMap::new(),
}
}
}
impl Default for Options {
fn default() -> Self {
Self {
symbols: true,
footprints: true,
models: true,
preserve_source: true,
}
}
}
impl Default for Watch {
fn default() -> Self {
Self {
directories: vec![home().join("Downloads")],
settle_seconds: 2,
retry_seconds: 60,
}
}
}
impl Default for DestinationConfig {
fn default() -> Self {
Self {
policy: "ask".into(),
project: None,
portable: true,
}
}
}
impl Default for Matching {
fn default() -> Self {
Self {
enabled: true,
exact: "reuse".into(),
ambiguous: "ask".into(),
tolerance: 0.15,
}
}
}
impl Default for Ui {
fn default() -> Self {
Self { enabled: true }
}
}
impl Config {
pub fn load(path: Option<&Path>) -> Result<Self> {
let path = path
.map(expand)
.unwrap_or_else(|| xdg("XDG_CONFIG_HOME", ".config").join("uniloader/config.toml"));
let mut cfg: Self = if path.exists() {
toml::from_str(&fs::read_to_string(&path)?)
.with_context(|| format!("invalid config {}", path.display()))?
} else {
if path.parent().is_some() {
fs::create_dir_all(path.parent().unwrap())?;
}
let cfg = Self::default();
fs::write(&path, toml::to_string_pretty(&cfg)?)?;
cfg
};
cfg.libraries.path = expand(&cfg.libraries.path);
cfg.libraries.kicad_config = cfg.libraries.kicad_config.as_deref().map(expand);
cfg.destination.project = cfg.destination.project.as_deref().map(expand);
for p in cfg
.watch
.directories
.iter_mut()
.chain(cfg.libraries.search_paths.iter_mut())
.chain(cfg.libraries.model_search_paths.iter_mut())
{
*p = expand(p);
}
if !["ask", "global", "project"].contains(&cfg.destination.policy.as_str()) {
bail!("destination.policy must be ask, global or project");
}
if !["reuse", "ask"].contains(&cfg.matching.exact.as_str()) {
bail!("matching.exact must be reuse or ask");
}
if !["ask", "separate", "skip"].contains(&cfg.matching.ambiguous.as_str()) {
bail!("matching.ambiguous must be ask, separate or skip");
}
if !cfg.matching.tolerance.is_finite() || cfg.matching.tolerance < 0.0 {
bail!("matching.tolerance must be finite and nonnegative");
}
if cfg.watch.retry_seconds < cfg.watch.settle_seconds {
bail!("watch.retry_seconds must be >= settle_seconds");
}
Ok(cfg)
}
}
+279
View File
@@ -0,0 +1,279 @@
//! Best-effort Linux project suggestions and optional desktop dialogs.
use anyhow::{Context, Result};
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use std::{
collections::BTreeSet,
fs,
io::{Read, Write},
path::{Path, PathBuf},
process::{Command, Stdio},
};
#[derive(Debug, Deserialize)]
pub struct DestinationChoice {
#[serde(default)]
pub skip: bool,
pub project: Option<PathBuf>,
pub remember: bool,
}
#[derive(Debug, Serialize)]
pub struct Candidate {
pub label: String,
pub path: PathBuf,
pub raw: String,
pub differences: Vec<String>,
}
#[derive(Debug)]
pub enum FootprintChoice {
Reuse(usize),
Separate,
Skip,
}
fn project_path(path: &Path) -> Option<PathBuf> {
let candidate = match path.extension().and_then(|s| s.to_str()) {
Some("kicad_pro") => path.to_path_buf(),
Some("kicad_pcb" | "kicad_sch" | "pro" | "sch") => path.with_extension("kicad_pro"),
_ => return None,
};
candidate
.is_file()
.then(|| fs::canonicalize(&candidate).unwrap_or(candidate))
}
/// Process arguments/cwd are suggestions, not an authoritative active-window API.
pub fn projects() -> Vec<PathBuf> {
let mut found = BTreeSet::new();
let Ok(processes) = fs::read_dir("/proc") else {
return vec![];
};
#[cfg(unix)]
use std::os::unix::fs::MetadataExt;
#[cfg(unix)]
let owner = fs::metadata("/proc/self").ok().map(|m| m.uid());
for process in processes.flatten() {
#[cfg(unix)]
if owner.is_some() && process.metadata().ok().map(|m| m.uid()) != owner {
continue;
}
if !process
.file_name()
.to_string_lossy()
.chars()
.all(|c| c.is_ascii_digit())
{
continue;
}
let proc_path = process.path();
let Ok(cmdline) = fs::read(proc_path.join("cmdline")) else {
continue;
};
let args: Vec<_> = cmdline
.split(|b| *b == 0)
.filter(|b| !b.is_empty())
.map(|b| String::from_utf8_lossy(b).into_owned())
.collect();
let Some(executable) = args.first() else {
continue;
};
let name = Path::new(executable)
.file_name()
.unwrap_or_default()
.to_string_lossy();
if !matches!(name.as_ref(), "kicad" | "pcbnew" | "eeschema") {
continue;
}
let cwd = fs::read_link(proc_path.join("cwd")).ok();
for arg in args.iter().skip(1).filter(|a| !a.starts_with('-')) {
let path = Path::new(arg);
let full = if path.is_absolute() {
path.to_path_buf()
} else if let Some(cwd) = &cwd {
cwd.join(path)
} else {
continue;
};
if let Some(project) = project_path(&full) {
found.insert(project);
}
}
// The editor may omit the document argument. A project in its cwd is a
// useful fallback, but it remains explicitly labelled a suggestion.
if let Some(cwd) = &cwd
&& let Ok(entries) = fs::read_dir(cwd)
{
for entry in entries.flatten() {
if entry.path().extension().and_then(|s| s.to_str()) == Some("kicad_pro")
&& let Some(project) = project_path(&entry.path())
{
found.insert(project);
}
}
}
}
found.into_iter().collect()
}
fn dialog(request: Value) -> Result<Option<Value>> {
if std::env::var_os("DISPLAY").is_none() && std::env::var_os("WAYLAND_DISPLAY").is_none() {
eprintln!("UniLoader: no graphical session; choice remains pending");
return Ok(None);
}
let spawn = Command::new(std::env::current_exe().context("locating UniLoader executable")?)
.arg("__dialog")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::inherit())
.spawn();
let mut child = match spawn {
Ok(child) => child,
Err(error) => {
eprintln!("UniLoader: could not start native dialog ({error}); choice remains pending");
return Ok(None);
}
};
let payload = serde_json::to_vec(&request)?;
let write_result = child
.stdin
.take()
.context("dialog stdin unavailable")
.and_then(|mut input| {
input
.write_all(&payload)
.context("writing native dialog request")
});
if let Err(error) = write_result {
eprintln!("UniLoader: native dialog unavailable ({error:#}); choice remains pending");
let _ = child.kill();
let _ = child.wait();
return Ok(None);
}
let output = match child.wait_with_output() {
Ok(output) => output,
Err(error) => {
eprintln!("UniLoader: native dialog unavailable ({error}); choice remains pending");
return Ok(None);
}
};
if !output.status.success() {
eprintln!("UniLoader: native dialog failed; choice remains pending");
return Ok(None);
}
let value: Value = match serde_json::from_slice(&output.stdout) {
Ok(value) => value,
Err(error) => {
eprintln!(
"UniLoader: invalid native dialog response ({error}); choice remains pending"
);
return Ok(None);
}
};
Ok((!value.is_null()).then_some(value))
}
/// Internal entry point: run the native window on the subprocess main thread.
/// It deliberately avoids loading configuration or acquiring importer locks.
pub fn run_internal_dialog() -> Result<()> {
const LIMIT: u64 = 32 * 1024 * 1024;
let mut bytes = Vec::new();
std::io::stdin().take(LIMIT + 1).read_to_end(&mut bytes)?;
anyhow::ensure!(
bytes.len() as u64 <= LIMIT,
"native dialog request exceeds 32 MiB"
);
let request: Value = serde_json::from_slice(&bytes).context("invalid native dialog request")?;
let response =
if std::env::var_os("DISPLAY").is_none() && std::env::var_os("WAYLAND_DISPLAY").is_none() {
eprintln!("UniLoader: no graphical session; choice remains pending");
None
} else {
match crate::gui::run(request) {
Ok(value) => value,
Err(error) => {
eprintln!(
"UniLoader: native dialog unavailable ({error:#}); choice remains pending"
);
None
}
}
};
let stdout = std::io::stdout();
let mut output = stdout.lock();
serde_json::to_writer(&mut output, &response)?;
writeln!(output)?;
Ok(())
}
pub fn destination(projects: &[PathBuf], component: &str) -> Result<Option<DestinationChoice>> {
dialog(json!({"mode": "destination", "projects": projects, "component": component}))?
.map(serde_json::from_value)
.transpose()
.context("invalid destination choice")
}
pub fn compare(incoming: &str, candidates: &[Candidate]) -> Result<Option<FootprintChoice>> {
if candidates.is_empty() {
return Ok(None);
}
let Some(value) =
dialog(json!({"mode": "compare", "incoming": incoming, "candidates": candidates}))?
else {
return Ok(None);
};
match value["action"].as_str() {
Some("reuse") => {
let index = value["index"]
.as_u64()
.context("missing footprint candidate index")? as usize;
anyhow::ensure!(
index < candidates.len(),
"footprint candidate index out of range"
);
Ok(Some(FootprintChoice::Reuse(index)))
}
Some("separate") => Ok(Some(FootprintChoice::Separate)),
Some("skip") => Ok(Some(FootprintChoice::Skip)),
_ => anyhow::bail!("invalid footprint choice"),
}
}
pub fn choose_file(title: &str, paths: &[PathBuf]) -> Result<Option<PathBuf>> {
let Some(value) = dialog(json!({"mode": "file", "title": title, "paths": paths}))? else {
return Ok(None);
};
let path: PathBuf =
serde_json::from_value(value["path"].clone()).context("invalid file choice")?;
anyhow::ensure!(paths.contains(&path), "selected file was not a candidate");
Ok(Some(path))
}
/// Optional desktop notification; logs remain the authoritative import report.
pub fn notify(message: &str) {
let _ = Command::new("notify-send")
.args(["--app-name=UniLoader", "UniLoader", message])
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null())
.status();
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn discovers_existing_project_from_editor_document() {
let dir = std::env::temp_dir().join(format!("uniloader-desktop-{}", std::process::id()));
fs::create_dir_all(&dir).unwrap();
let project = dir.join("board.kicad_pro");
fs::write(&project, "{}").unwrap();
assert_eq!(
project_path(&dir.join("board.kicad_pcb")),
Some(fs::canonicalize(&project).unwrap())
);
assert_eq!(project_path(&dir.join("missing.kicad_sch")), None);
fs::remove_dir_all(dir).unwrap();
}
}
+367
View File
@@ -0,0 +1,367 @@
use crate::sexpr::{Node, parse};
use anyhow::{Context, Result, bail};
use std::{
collections::BTreeMap,
env,
path::{Path, PathBuf},
};
#[derive(Debug, Clone)]
pub struct Destination {
pub project: Option<PathBuf>,
pub root: PathBuf,
pub table_dir: PathBuf,
pub symbol_path: PathBuf,
pub pretty_dir: PathBuf,
pub models_dir: PathBuf,
}
impl Destination {
pub fn new(global_root: &Path, project: Option<&Path>, kicad_config: &Path) -> Result<Self> {
let project = project
.map(|p| -> Result<PathBuf> {
if p.extension().and_then(|x| x.to_str()) != Some("kicad_pro") || !p.is_file() {
bail!(
"Project must be an existing .kicad_pro file: {}",
p.display()
);
}
Ok(p.canonicalize()?
.parent()
.context("Project has no parent directory")?
.to_path_buf())
})
.transpose()?;
let root = match &project {
Some(p) => p.join("libraries"),
None => absolute(global_root)?,
};
let table_dir = match &project {
Some(p) => p.clone(),
None => absolute(kicad_config)?,
};
let result = Self {
project,
symbol_path: root.join("UniLoader.kicad_sym"),
pretty_dir: root.join("UniLoader.pretty"),
models_dir: root.join("models"),
root,
table_dir,
};
for path in [
&result.root,
&result.table_dir,
&result.pretty_dir,
&result.models_dir,
] {
if path.exists() && !path.is_dir() {
bail!("Expected a directory: {}", path.display());
}
}
if result.symbol_path.exists() && !result.symbol_path.is_file() {
bail!(
"Expected a symbol library file: {}",
result.symbol_path.display()
);
}
Ok(result)
}
pub fn nickname(&self) -> &'static str {
if self.project.is_some() {
"UniLoader_Local"
} else {
"UniLoader_Global"
}
}
fn uri(&self, path: &Path) -> String {
if let Some(project) = &self.project
&& let Ok(relative) = path.strip_prefix(project)
{
return format!("${{KIPRJMOD}}/{}", relative.to_string_lossy());
}
path.to_string_lossy().into_owned()
}
pub fn model_uri(&self, path: &Path) -> String {
self.uri(path)
}
pub fn footprint_uri(&self) -> String {
self.uri(&self.pretty_dir)
}
pub fn symbol_uri(&self) -> String {
self.uri(&self.symbol_path)
}
}
fn absolute(path: &Path) -> Result<PathBuf> {
let text = path.to_string_lossy();
let expanded = if text == "~" {
PathBuf::from(env::var("HOME")?)
} else if let Some(suffix) = text.strip_prefix("~/") {
PathBuf::from(env::var("HOME")?).join(suffix)
} else {
path.to_path_buf()
};
Ok(if expanded.is_absolute() {
expanded
} else {
env::current_dir()?.join(expanded)
})
}
fn field_value<'a>(node: &'a Node, name: &str) -> Option<&'a str> {
match node {
Node::List(items) => items.iter().find_map(|item| match item {
Node::List(parts) if matches!(parts.first(), Some(Node::Atom(head)) if head == name) =>
parts.get(1).and_then(|p| match p { Node::Atom(s) | Node::Str(s) => Some(s.as_str()), _ => None }),
_ => None,
}),
_ => None,
}
}
fn field(name: &str, value: &str) -> Node {
Node::List(vec![Node::Atom(name.into()), Node::Str(value.into())])
}
/// Merge our entry without changing another library's nickname or URI.
pub fn register_table(
existing: Option<&str>,
kind: &str,
requested_nickname: &str,
uri: &str,
) -> Result<(String, String)> {
if kind != "sym_lib_table" && kind != "fp_lib_table" {
bail!("Unsupported library table: {kind}");
}
let mut table = match existing {
Some(text) => parse(text)?,
None => Node::List(vec![
Node::Atom(kind.into()),
Node::List(vec![Node::Atom("version".into()), Node::Atom("7".into())]),
]),
};
let Node::List(items) = &mut table else {
bail!("Library table must be a list");
};
if !matches!(items.first(), Some(Node::Atom(head)) if head == kind) {
bail!("Expected {kind} library table");
}
// Reuse our existing registration even if a previous collision changed its name.
if let Some(name) = items.iter().find_map(|entry| {
(field_value(entry, "uri") == Some(uri) && field_value(entry, "type") == Some("KiCad") && !matches!(entry, Node::List(parts) if parts.iter().any(|part| matches!(part, Node::List(fields) if matches!(fields.first(), Some(Node::Atom(head)) if head == "disabled")))))
.then(|| field_value(entry, "name").map(str::to_owned))
.flatten()
}) {
if requested_nickname.starts_with("UniLoader_") && !name.starts_with(requested_nickname) {
bail!("Legacy managed library nickname {name}; run `uniloader doctor --global --fix` or `uniloader doctor --project /path/to/Board.kicad_pro --fix` before importing");
}
return Ok((format!("{table}\n"), name));
}
let mut nickname = requested_nickname.to_owned();
let mut suffix = 2;
while items
.iter()
.any(|entry| field_value(entry, "name") == Some(nickname.as_str()))
{
nickname = format!("{requested_nickname}_{suffix}");
suffix += 1;
}
items.push(Node::List(vec![
Node::Atom("lib".into()),
field("name", &nickname),
field("type", "KiCad"),
field("uri", uri),
field("options", ""),
field("descr", "Imported by UniLoader"),
]));
Ok((format!("{table}\n"), nickname))
}
#[derive(Debug, Clone)]
pub struct LibraryEntry {
pub nickname: String,
pub path: PathBuf,
}
pub fn resolve_variables(
text: &str,
project: Option<&Path>,
variables: &BTreeMap<String, String>,
) -> Result<String> {
let mut resolved = text.to_owned();
for _ in 0..16 {
let Some(start) = resolved.find("${") else {
return Ok(resolved);
};
let end = resolved[start + 2..]
.find('}')
.context("Unterminated KiCad path variable")?
+ start
+ 2;
let name = &resolved[start + 2..end];
let value = if name == "KIPRJMOD" {
project
.context("KIPRJMOD requires a project destination")?
.to_string_lossy()
.into_owned()
} else {
variables
.get(name)
.cloned()
.or_else(|| env::var(name).ok())
.with_context(|| format!("Unresolved KiCad path variable: {name}"))?
};
resolved.replace_range(start..=end, &value);
}
bail!("Recursive or excessive KiCad path variables in {text}")
}
/// Read project overrides before validating paths. A broken global URI must
/// not prevent importing when the project supplies a valid replacement.
pub fn effective_footprint_libraries(
global: &Path,
local: Option<&Path>,
project: Option<&Path>,
variables: &BTreeMap<String, String>,
) -> Result<Vec<LibraryEntry>> {
let mut effective = BTreeMap::<String, (Node, PathBuf)>::new();
for table in std::iter::once(global).chain(local) {
if !table.exists() {
continue;
}
let node = parse(
&std::fs::read_to_string(table).with_context(|| format!("Read {}", table.display()))?,
)?;
let Node::List(items) = node else {
bail!("Invalid footprint table {}", table.display());
};
if !matches!(items.first(), Some(Node::Atom(head)) if head == "fp_lib_table") {
bail!("Invalid footprint table {}", table.display());
}
for entry in items.into_iter().skip(1) {
if !matches!(&entry, Node::List(parts) if matches!(parts.first(), Some(Node::Atom(head)) if head == "lib"))
{
continue;
}
let nickname = field_value(&entry, "name")
.with_context(|| format!("Library entry missing name in {}", table.display()))?
.to_owned();
effective.insert(nickname, (entry, table.to_path_buf()));
}
}
let mut entries = Vec::new();
for (nickname, (entry, table)) in effective {
// Disabled/unsupported local registrations still shadow global entries.
if matches!(&entry, Node::List(parts) if parts.iter().any(|part| matches!(part, Node::List(fields) if matches!(fields.first(), Some(Node::Atom(head)) if head == "disabled"))))
{
continue;
}
if field_value(&entry, "type") != Some("KiCad") {
continue;
}
let uri = field_value(&entry, "uri")
.with_context(|| format!("Library {nickname} missing URI in {}", table.display()))?;
let path = PathBuf::from(
resolve_variables(uri, project, variables)
.with_context(|| format!("Library {nickname} in {}", table.display()))?,
);
let path = if path.is_absolute() {
path
} else {
table.parent().unwrap_or(Path::new(".")).join(path)
};
if !path.is_dir() {
bail!(
"Registered footprint library {nickname} is missing or not a directory: {}",
path.display()
);
}
std::fs::read_dir(&path).with_context(|| {
format!(
"Unreadable footprint library {nickname}: {}",
path.display()
)
})?;
entries.push(LibraryEntry { nickname, path });
}
Ok(entries)
}
#[cfg(test)]
pub fn footprint_libraries(
table: &Path,
project: Option<&Path>,
variables: &BTreeMap<String, String>,
) -> Result<Vec<LibraryEntry>> {
effective_footprint_libraries(table, None, project, variables)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn preserves_tables_and_avoids_nickname_collision() {
let initial = r#"(fp_lib_table (version 7) (lib (name "UniLoader") (type "KiCad") (uri "/other.pretty") (options "keep")) (custom "keep too"))"#;
let (text, name) =
register_table(Some(initial), "fp_lib_table", "UniLoader", "/new.pretty").unwrap();
assert_eq!(name, "UniLoader_2");
assert!(text.contains("keep too"));
assert!(text.contains("/other.pretty"));
let (_, again) =
register_table(Some(&text), "fp_lib_table", "UniLoader", "/new.pretty").unwrap();
assert_eq!(again, name);
}
#[test]
fn local_library_shadows_broken_global_before_path_resolution() {
let root = std::env::temp_dir().join(format!(
"uniloader-tables-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
std::fs::create_dir_all(root.join("project/libraries/Shared.pretty")).unwrap();
let global = root.join("global-table");
let local = root.join("project/fp-lib-table");
std::fs::write(&global, r#"(fp_lib_table (lib (name "Shared") (type "KiCad") (uri "${UNILOADER_MISSING_TEST_VARIABLE}/Shared.pretty")))"#).unwrap();
std::fs::write(&local, r#"(fp_lib_table (lib (name "Shared") (type "KiCad") (uri "${KIPRJMOD}/libraries/Shared.pretty")))"#).unwrap();
let entries = effective_footprint_libraries(
&global,
Some(&local),
Some(&root.join("project")),
&BTreeMap::new(),
)
.unwrap();
assert_eq!(entries.len(), 1);
assert_eq!(entries[0].nickname, "Shared");
assert_eq!(
entries[0].path,
root.join("project/libraries/Shared.pretty")
);
// An effective unresolved entry must still be reported.
assert!(footprint_libraries(&global, None, &BTreeMap::new()).is_err());
// A project-disabled entry shadows, rather than reviving the global one.
std::fs::write(&local, r#"(fp_lib_table (lib (name "Shared") (type "KiCad") (uri "/not/present") (disabled)))"#).unwrap();
assert!(
effective_footprint_libraries(
&global,
Some(&local),
Some(&root.join("project")),
&BTreeMap::new()
)
.unwrap()
.is_empty()
);
std::fs::remove_dir_all(root).unwrap();
}
#[test]
fn variable_expansion_reports_missing_and_recursion() {
let vars = BTreeMap::from([("CUSTOM".into(), "${KIPRJMOD}/models".into())]);
assert_eq!(
resolve_variables("${CUSTOM}/x.step", Some(Path::new("/project")), &vars).unwrap(),
"/project/models/x.step"
);
assert!(resolve_variables("${UNILOADER_MISSING_TEST_VARIABLE}", None, &vars).is_err());
assert!(resolve_variables("${KIPRJMOD}", None, &vars).is_err());
}
}
+252
View File
@@ -0,0 +1,252 @@
//! Explicit, backed-up migration of managed library registrations and references.
use crate::{
config::Config,
destination::{self, Destination},
importer,
sexpr::{Node, parse},
storage::{Lock, Plan, Record},
};
use anyhow::{Context, Result, bail};
use std::{collections::BTreeMap, fs, path::Path};
type Renames = BTreeMap<String, String>;
fn table(dest: &Destination, kind: &str, vars: &Renames, plan: &mut Plan) -> Result<Renames> {
let (file, target) = if kind == "sym_lib_table" {
("sym-lib-table", &dest.symbol_path)
} else {
("fp-lib-table", &dest.pretty_dir)
};
let path = dest.table_dir.join(file);
if !path.exists() {
return Ok(Renames::new());
}
let original = fs::read_to_string(&path)?;
let mut node = parse(&original)?;
if node.head() != Some(kind) {
bail!("Invalid table {}", path.display());
}
// Keep exact old names so a later project migration also handles legacy
// collision suffixes without guessing which nickname used to be ours.
let history_path = crate::config::data_dir()
.join("nickname-migrations")
.join(format!(
"{}.json",
crate::storage::hash(path.to_string_lossy().as_bytes())
));
let mut history: Renames = if history_path.exists() {
serde_json::from_slice(&fs::read(&history_path)?)?
} else {
Renames::new()
};
let original_history = history.clone();
let mut changes = Renames::new();
let mut occupied = BTreeMap::new();
for entry in node.children().iter().filter(|n| n.head() == Some("lib")) {
let name = entry
.field("name")
.and_then(Node::value)
.context("Library entry missing name")?;
if occupied.insert(name.to_owned(), ()).is_some() {
bail!("Duplicate nickname {name} in {}", path.display());
}
}
for entry in node
.children_mut()
.iter_mut()
.filter(|n| n.head() == Some("lib"))
{
let name = entry
.field("name")
.and_then(Node::value)
.context("Library entry missing name")?
.to_owned();
// Only change entries that resolve to our managed file/directory.
let uri = entry
.field("uri")
.and_then(Node::value)
.context("Library entry missing URI")?;
let Ok(resolved) = destination::resolve_variables(uri, dest.project.as_deref(), vars)
else {
continue;
};
let resolved = std::path::PathBuf::from(resolved);
let resolved = if resolved.is_absolute() {
resolved
} else {
dest.table_dir.join(resolved)
};
let matches = resolved == *target
|| (target.exists() && resolved.canonicalize().ok() == target.canonicalize().ok());
if !matches || entry.field("type").and_then(Node::value) != Some("KiCad") {
continue;
}
let wanted = dest.nickname();
if name == wanted
|| name
.strip_prefix(wanted)
.is_some_and(|s| s.starts_with('_'))
{
changes.extend(
history
.iter()
.filter(|(old, new)| *new == &name && !occupied.contains_key(*old))
.map(|(old, new)| (old.clone(), new.clone())),
);
continue;
}
if occupied.contains_key(wanted) {
bail!(
"Nickname {wanted} is occupied in {}; resolve it before migration",
path.display()
);
}
occupied.insert(wanted.into(), ());
entry.field_mut("name").unwrap().children_mut()[1] = Node::string(wanted);
history.insert(name.clone(), wanted.into());
changes.insert(name, wanted.into());
}
if history != original_history {
plan.put(history_path, serde_json::to_vec_pretty(&history)?)?;
}
if node != parse(&original)? {
plan.put(path, format!("{node}\n").into_bytes())?;
}
Ok(changes)
}
fn rename(value: &mut Node, mapping: &Renames) {
if let Node::Atom(s) | Node::Str(s) = value
&& let Some((nick, item)) = s.split_once(':')
&& let Some(new) = mapping.get(nick)
{
*s = format!("{new}:{item}");
}
}
fn references(node: &mut Node, symbols: &Renames, footprints: &Renames, parent: Option<&str>) {
let head = node.head().map(str::to_owned);
if let Node::List(items) = node {
match head.as_deref() {
Some("lib_id") | Some("extends") => {
if let Some(v) = items.get_mut(1) {
rename(v, symbols);
}
}
Some("symbol") if parent == Some("lib_symbols") => {
if let Some(v) = items.get_mut(1) {
rename(v, symbols);
}
}
Some("footprint") => {
if let Some(v) = items.get_mut(1) {
rename(v, footprints);
}
}
Some("property") if items.get(1).and_then(Node::atom) == Some("Footprint") => {
if let Some(v) = items.get_mut(2) {
rename(v, footprints);
}
}
_ => {}
}
for child in items {
references(child, symbols, footprints, head.as_deref());
}
}
}
fn patch(path: &Path, symbols: &Renames, footprints: &Renames, plan: &mut Plan) -> Result<()> {
if !path.is_file() {
return Ok(());
}
let original = fs::read_to_string(path).with_context(|| format!("Read {}", path.display()))?;
let mut node = parse(&original).with_context(|| format!("Parse {}", path.display()))?;
let before = node.clone();
references(&mut node, symbols, footprints, None);
if node != before {
plan.put(path.to_owned(), format!("{node}\n").into_bytes())?;
}
Ok(())
}
// A local entry shadows the global mapping even if it belongs to another tool.
fn overlay(dest: &Destination, file: &str, global: &Renames, local: Renames) -> Result<Renames> {
let mut result = global.clone();
let path = dest.table_dir.join(file);
if path.exists() {
for entry in parse(&fs::read_to_string(path)?)?.children() {
if entry.head() == Some("lib")
&& let Some(name) = entry.field("name").and_then(Node::value)
{
result.remove(name);
}
}
}
result.extend(local);
Ok(result)
}
pub fn run(cfg: &Config, project: Option<&Path>, fix: bool) -> Result<()> {
let _lock = Lock::acquire()?;
let kicad = importer::kicad_config(cfg, project.is_none())?;
let global = Destination::new(&cfg.libraries.path, None, &kicad)?;
let mut plan = Plan::default();
let sym = table(
&global,
"sym_lib_table",
&cfg.libraries.variables,
&mut plan,
)?;
let fp = table(&global, "fp_lib_table", &cfg.libraries.variables, &mut plan)?;
patch(&global.symbol_path, &sym, &fp, &mut plan)?;
if let Some(project) = project {
let local = Destination::new(&cfg.libraries.path, Some(project), &kicad)?;
let local_sym = table(&local, "sym_lib_table", &cfg.libraries.variables, &mut plan)?;
let local_fp = table(&local, "fp_lib_table", &cfg.libraries.variables, &mut plan)?;
let sym = overlay(&local, "sym-lib-table", &sym, local_sym)?;
let fp = overlay(&local, "fp-lib-table", &fp, local_fp)?;
patch(&local.symbol_path, &sym, &fp, &mut plan)?;
for entry in walkdir::WalkDir::new(local.project.as_ref().unwrap()).follow_links(false) {
let entry = entry?;
if entry.file_type().is_file()
&& entry
.path()
.extension()
.is_some_and(|e| e == "kicad_sch" || e == "kicad_pcb")
{
patch(entry.path(), &sym, &fp, &mut plan)?;
}
}
}
for path in plan.writes.keys() {
println!(
"{}: {}",
if fix { "Repair" } else { "Would repair" },
path.display()
);
}
if plan.writes.is_empty() {
println!("No nickname repairs needed.");
} else if fix {
let mut record = Record::new(
global.root.clone(),
crate::storage::hash(b"doctor-nicknames"),
);
record.id = format!(
"doctor-{}",
chrono::Utc::now().timestamp_nanos_opt().unwrap_or_default()
);
record.status = "importing".into();
plan.commit(&mut record)?;
record.status = "completed".into();
record.message = "Library nickname migration completed".into();
record.save()?;
for backup in record.backups {
println!("Backup: {}", backup.display());
}
} else {
println!("Preview only; rerun with --fix to apply repairs.");
}
println!(
"Run doctor --project /path/to/Board.kicad_pro for each existing project. Close KiCad and stop the watcher before --fix; reopen KiCad afterwards."
);
Ok(())
}
+536
View File
@@ -0,0 +1,536 @@
//! Native desktop dialogs. Runs in the binary's isolated `__dialog` process.
use crate::preview::{Preview, fit_scale};
use anyhow::{Context, Result, ensure};
use eframe::egui;
use serde::Deserialize;
use serde_json::{Value, json};
use std::{
path::PathBuf,
sync::{Arc, Mutex, mpsc},
};
#[derive(Deserialize)]
#[serde(tag = "mode", rename_all = "lowercase")]
enum Request {
Destination {
component: String,
projects: Vec<PathBuf>,
},
Compare {
incoming: String,
candidates: Vec<Candidate>,
},
File {
title: String,
paths: Vec<PathBuf>,
},
}
#[derive(Deserialize)]
struct Candidate {
label: String,
path: PathBuf,
raw: String,
differences: Vec<String>,
}
#[derive(Clone, PartialEq)]
enum Destination {
Global,
Project(PathBuf),
}
enum Dialog {
Destination {
component: String,
projects: Vec<PathBuf>,
selected: Destination,
remember: bool,
browse: Option<mpsc::Receiver<Option<PathBuf>>>,
browse_message: Option<String>,
},
Compare {
incoming: Preview,
candidates: Vec<Candidate>,
previews: Vec<Preview>,
selected: usize,
},
File {
title: String,
paths: Vec<PathBuf>,
selected: Option<usize>,
},
}
impl Dialog {
fn from_request(request: Value) -> Result<Self> {
Ok(
match serde_json::from_value(request).context("invalid desktop dialog request")? {
Request::Destination {
component,
projects,
} => {
let selected = projects
.first()
.cloned()
.map(Destination::Project)
.unwrap_or(Destination::Global);
Self::Destination {
component,
projects,
selected,
remember: false,
browse: None,
browse_message: None,
}
}
Request::Compare {
incoming,
candidates,
} => {
ensure!(!candidates.is_empty(), "no footprint candidates to compare");
let previews = candidates
.iter()
.map(|c| Preview::from_raw(&c.raw))
.collect();
Self::Compare {
incoming: Preview::from_raw(&incoming),
candidates,
previews,
selected: 0,
}
}
Request::File { title, paths } => {
ensure!(!paths.is_empty(), "no model files to choose from");
Self::File {
title,
paths,
selected: None,
}
}
},
)
}
fn file_response(paths: &[PathBuf], selected: Option<usize>) -> Option<Value> {
selected
.and_then(|index| paths.get(index))
.map(|path| json!({"path": path}))
}
/// Returns a response only after an explicit action. The outer Option is
/// completion; an inner None means Later, leaving the import pending.
fn show(&mut self, ui: &mut egui::Ui) -> Option<Option<Value>> {
ui.heading("UniLoader");
ui.add_space(8.0);
let mut result = None;
match self {
Self::Destination {
component,
projects,
selected,
remember,
browse,
browse_message,
} => {
ui.label(format!("Import: {component}"));
ui.label("Choose the component's destination.");
ui.weak(
"Detected KiCad projects are suggestions; check the path before importing.",
);
ui.add_space(8.0);
egui::ScrollArea::vertical()
.max_height(230.0)
.id_salt("projects")
.show(ui, |ui| {
for project in projects.iter() {
ui.radio_value(
selected,
Destination::Project(project.clone()),
project.display().to_string(),
);
}
ui.radio_value(selected, Destination::Global, "Global libraries");
});
if let Some(receiver) = browse {
match receiver.try_recv() {
Ok(path) => {
if let Some(path) = path {
if path.extension().is_some_and(|ext| ext == "kicad_pro")
&& path.is_file()
{
if !projects.contains(&path) {
projects.push(path.clone());
}
*selected = Destination::Project(path);
*browse_message = None;
} else {
*browse_message =
Some("Select an existing .kicad_pro file.".into());
}
} else {
*browse_message = Some("No project selected. You can retry Browse or select a destination above.".into());
}
*browse = None;
}
Err(mpsc::TryRecvError::Disconnected) => {
*browse_message = Some("The desktop file picker is unavailable. Select a detected project or global libraries.".into());
*browse = None;
}
Err(mpsc::TryRecvError::Empty) => {}
}
}
if ui
.add_enabled(browse.is_none(), egui::Button::new("Browse for project…"))
.clicked()
{
let (sender, receiver) = mpsc::channel();
let ctx = ui.ctx().clone();
std::thread::spawn(move || {
let path = rfd::FileDialog::new()
.set_title("Select KiCad project")
.add_filter("KiCad project", &["kicad_pro"])
.pick_file();
let _ = sender.send(path);
ctx.request_repaint();
});
*browse = Some(receiver);
*browse_message = None;
}
if browse.is_some() {
ui.weak("Waiting for the desktop file picker…");
}
if let Some(message) = browse_message {
ui.label(message.as_str());
}
ui.add_space(8.0);
ui.checkbox(
remember,
"Remember for this session while this project remains open",
);
ui.separator();
ui.horizontal_wrapped(|ui| {
if ui
.add_enabled(browse.is_none(), egui::Button::new("Import"))
.clicked()
{
let project = match selected {
Destination::Global => None,
Destination::Project(path) => Some(path.clone()),
};
result = Some(Some(
json!({"project":project,"remember":remember,"skip":false}),
));
}
if ui.button("Skip component").clicked() {
result = Some(Some(json!({"project":null,"remember":false,"skip":true})));
}
});
}
Self::Compare {
incoming,
candidates,
previews,
selected,
} => {
ui.label("Compare footprint candidates");
egui::ComboBox::from_id_salt("candidate")
.width(ui.available_width().min(800.0))
.selected_text(&candidates[*selected].label)
.show_ui(ui, |ui| {
for (index, candidate) in candidates.iter().enumerate() {
ui.selectable_value(selected, index, &candidate.label);
}
});
ui.weak(candidates[*selected].path.display().to_string());
let right = &previews[*selected];
let pane_width = (ui.available_width() - ui.spacing().item_spacing.x) / 2.0;
let scale = fit_scale(incoming, right, pane_width, 280.0);
ui.columns(2, |columns| {
columns[0].label("Incoming footprint");
incoming.paint(&mut columns[0], scale);
columns[1].label("Existing candidate");
right.paint(&mut columns[1], scale);
});
ui.weak("Both previews use the same scale. Drill holes are simplified; text and 3D geometry are omitted. Unsupported shapes are listed below.");
egui::ScrollArea::vertical()
.max_height(170.0)
.id_salt("differences")
.show(ui, |ui| {
for difference in &candidates[*selected].differences {
ui.label(format!("• {difference}"));
}
for warning in incoming.warnings.iter().chain(right.warnings.iter()) {
ui.colored_label(egui::Color32::YELLOW, warning);
}
});
ui.separator();
ui.horizontal_wrapped(|ui| {
if ui.button("Use selected existing").clicked() {
result = Some(Some(json!({"action":"reuse","index":selected})));
}
if ui.button("Import separately").clicked() {
result = Some(Some(json!({"action":"separate"})));
}
if ui.button("Skip component").clicked() {
result = Some(Some(json!({"action":"skip"})));
}
});
}
Self::File {
title,
paths,
selected,
} => {
ui.label(title.as_str());
ui.weak("Multiple model files match. Select the file to use.");
egui::ScrollArea::vertical()
.max_height(350.0)
.id_salt("models")
.show(ui, |ui| {
for (index, path) in paths.iter().enumerate() {
ui.selectable_value(selected, Some(index), path.display().to_string());
}
});
ui.separator();
if ui
.add_enabled(selected.is_some(), egui::Button::new("Use selected file"))
.clicked()
{
result = Self::file_response(paths, *selected).map(Some);
}
}
}
ui.add_space(6.0);
if ui.button("Later").clicked() {
result = Some(None);
}
result
}
}
struct App {
dialog: Dialog,
response: Arc<Mutex<Option<Value>>>,
}
impl eframe::App for App {
fn ui(&mut self, ui: &mut egui::Ui, _frame: &mut eframe::Frame) {
egui::CentralPanel::default().show(ui, |ui| {
egui::ScrollArea::vertical().show(ui, |ui| {
if let Some(response) = self.dialog.show(ui) {
if let Ok(mut output) = self.response.lock() {
*output = response;
}
ui.ctx().send_viewport_cmd(egui::ViewportCommand::Close);
}
});
});
}
}
pub fn run(request: Value) -> Result<Option<Value>> {
let dialog = Dialog::from_request(request)?;
let compare = matches!(dialog, Dialog::Compare { .. });
let response = Arc::new(Mutex::new(None));
let app_response = response.clone();
let options = eframe::NativeOptions {
viewport: egui::ViewportBuilder::default()
.with_app_id("uniloader")
.with_inner_size(if compare {
[700.0, 700.0]
} else {
[700.0, 490.0]
})
.with_min_inner_size([660.0, 400.0]),
..Default::default()
};
eframe::run_native(
"UniLoader — component import",
options,
Box::new(move |_cc| {
Ok(Box::new(App {
dialog,
response: app_response,
}))
}),
)
.map_err(|error| anyhow::anyhow!("native desktop dialog failed: {error}"))?;
let result = response
.lock()
.map_err(|_| anyhow::anyhow!("dialog response lock failed"))?
.clone();
Ok(result)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn models_require_explicit_selection() {
let dialog = Dialog::from_request(
json!({"mode":"file","title":"Choose model","paths":["/a.step","/b.step"]}),
)
.unwrap();
let Dialog::File {
paths, selected, ..
} = dialog
else {
panic!("wrong dialog")
};
assert!(selected.is_none());
assert!(Dialog::file_response(&paths, selected).is_none());
assert_eq!(
Dialog::file_response(&paths, Some(1)),
Some(json!({"path":"/b.step"}))
);
assert!(Dialog::file_response(&paths, Some(2)).is_none());
}
#[test]
fn rejects_invalid_requests() {
for request in [
json!({"mode":"other"}),
json!({"mode":"file","title":"Choose","paths":[]}),
json!({"mode":"compare","incoming":"","candidates":[]}),
] {
assert!(Dialog::from_request(request).is_err());
}
}
fn frame(
dialog: &mut Dialog,
ctx: &egui::Context,
events: Vec<egui::Event>,
) -> (egui::FullOutput, Option<Option<Value>>) {
let mut answer = None;
let output = ctx.run_ui(
egui::RawInput {
screen_rect: Some(egui::Rect::from_min_size(
egui::Pos2::ZERO,
egui::vec2(900.0, 900.0),
)),
events,
..Default::default()
},
|ui| {
egui::CentralPanel::default().show(ui, |ui| {
answer = dialog.show(ui);
});
},
);
(output, answer)
}
fn text_center(shape: &egui::epaint::Shape, label: &str) -> Option<egui::Pos2> {
match shape {
egui::epaint::Shape::Text(text) if text.galley.text() == label => {
Some(text.pos + text.galley.rect.center().to_vec2())
}
egui::epaint::Shape::Vec(shapes) => {
shapes.iter().find_map(|shape| text_center(shape, label))
}
_ => None,
}
}
fn click(dialog: &mut Dialog, ctx: &egui::Context, label: &str) -> Option<Option<Value>> {
let (mut output, _) = frame(dialog, ctx, vec![]);
let pos = output
.shapes
.iter()
.find_map(|shape| text_center(&shape.shape, label))
.unwrap_or_else(|| panic!("missing UI label: {label}"));
output.textures_delta.clear();
let mut answer = None;
for pressed in [true, false] {
let (mut output, response) = frame(
dialog,
ctx,
vec![
egui::Event::PointerMoved(pos),
egui::Event::PointerButton {
pos,
button: egui::PointerButton::Primary,
pressed,
modifiers: Default::default(),
},
],
);
output.textures_delta.clear();
if response.is_some() {
answer = response;
}
}
answer
}
#[test]
fn actual_buttons_produce_expected_responses() {
let destination = json!({"mode":"destination","component":"Example","projects":[]});
let compare = json!({"mode":"compare","incoming":"(footprint a)","candidates":[{"label":"Existing","path":"/x.kicad_mod","raw":"(footprint b)","differences":[]}]});
for (request, label, expected) in [
(
destination.clone(),
"Import",
Some(json!({"project":null,"remember":false,"skip":false})),
),
(
destination.clone(),
"Skip component",
Some(json!({"project":null,"remember":false,"skip":true})),
),
(destination, "Later", None),
(
compare.clone(),
"Use selected existing",
Some(json!({"action":"reuse","index":0})),
),
(
compare,
"Import separately",
Some(json!({"action":"separate"})),
),
] {
let mut dialog = Dialog::from_request(request).unwrap();
assert_eq!(
click(&mut dialog, &egui::Context::default(), label),
Some(expected)
);
}
}
#[test]
fn model_confirmation_stays_disabled_until_selection() {
let mut dialog = Dialog::from_request(
json!({"mode":"file","title":"Choose model","paths":["/a.step","/b.step"]}),
)
.unwrap();
let ctx = egui::Context::default();
assert_eq!(click(&mut dialog, &ctx, "Use selected file"), None);
assert_eq!(click(&mut dialog, &ctx, "/b.step"), None);
assert_eq!(
click(&mut dialog, &ctx, "Use selected file"),
Some(Some(json!({"path":"/b.step"})))
);
}
#[test]
fn idle_dialogs_do_not_consent() {
let requests = [
json!({"mode":"destination","component":"Example Ω","projects":["/tmp/board.kicad_pro"]}),
json!({"mode":"file","title":"Choose model","paths":["/a.step","/b.step"]}),
json!({"mode":"compare","incoming":"(footprint a)","candidates":[{"label":"Existing","path":"/x.kicad_mod","raw":"(footprint b)","differences":["Pad locations differ"]}]}),
];
for request in requests {
let mut dialog = Dialog::from_request(request).unwrap();
let ctx = egui::Context::default();
let mut output = ctx.run_ui(Default::default(), |ui| {
egui::CentralPanel::default().show(ui, |ui| {
assert!(dialog.show(ui).is_none());
});
});
output.textures_delta.clear();
}
}
}
+837
View File
@@ -0,0 +1,837 @@
use crate::{
config::{self, Config},
desktop,
destination::{self, Destination},
matching, models,
sexpr::{self, Node},
sources,
storage::{self, Plan, Record},
};
use anyhow::{Context, Result, bail};
use serde::{Deserialize, Serialize};
use std::{
collections::{BTreeMap, BTreeSet},
fs,
path::{Path, PathBuf},
};
#[derive(Default, Clone)]
pub struct Request {
pub global: bool,
pub project: Option<PathBuf>,
pub dry_run: bool,
pub noninteractive: bool,
pub force: bool,
}
#[derive(Default)]
pub struct Session {
pub remembered: Option<(Option<PathBuf>, Vec<PathBuf>)>,
}
struct Foot {
path: PathBuf,
node: Node,
nickname: Option<String>,
local: bool,
fingerprint: Option<String>,
}
impl Foot {
fn name(&self) -> &str {
self.path
.file_stem()
.and_then(|s| s.to_str())
.unwrap_or_else(|| self.node.value().unwrap_or(""))
}
fn fingerprint(&self) -> String {
self.fingerprint
.clone()
.unwrap_or_else(|| matching::exact_fingerprint(&self.node, &self.path))
}
}
#[derive(Serialize, Deserialize)]
struct Decision {
path: PathBuf,
fingerprint: String,
}
fn text(path: &Path) -> Result<Option<String>> {
if path.exists() {
Ok(Some(fs::read_to_string(path)?))
} else {
Ok(None)
}
}
fn encoded(n: &Node) -> Vec<u8> {
format!("{n}\n").into_bytes()
}
fn safe_name(name: &str) -> Result<String> {
if name.is_empty() || name == "." || name == ".." || name.contains(['/', '\\', ':', '\0']) {
bail!("invalid KiCad asset name: {name:?}");
}
Ok(name.into())
}
pub fn kicad_config(cfg: &Config, required: bool) -> Result<PathBuf> {
if let Some(p) = &cfg.libraries.kicad_config {
return Ok(p.clone());
}
let root = config::xdg("XDG_CONFIG_HOME", ".config").join("kicad");
let mut dirs = vec![];
if root.exists() {
for e in fs::read_dir(&root)? {
let p = e?.path();
if p.is_dir()
&& p.file_name().is_some_and(|s| {
s.to_string_lossy()
.chars()
.all(|c| c.is_ascii_digit() || c == '.')
})
{
dirs.push(p);
}
}
}
if dirs.len() == 1 {
return Ok(dirs.remove(0));
}
if required {
bail!(
"pending: set libraries.kicad_config to the KiCad version directory (none or multiple detected)"
);
}
// No global table to read. This placeholder is never created in project mode.
Ok(root.join("unselected-version"))
}
fn target(
cfg: &Config,
req: &Request,
record: &Record,
session: &mut Session,
) -> Result<Option<PathBuf>> {
if req.global {
return Ok(None);
}
if let Some(p) = &req.project {
return Ok(Some(config::expand(p)));
}
if record.destination.is_some() {
return Ok(record.project.clone());
}
match cfg.destination.policy.as_str() {
"global" => Ok(None),
"project" => {
Ok(Some(cfg.destination.project.clone().context(
"destination.project must be set for project policy",
)?))
}
_ => {
let projects = desktop::projects();
if let Some((remembered, original)) = &session.remembered {
if original == &projects
&& (remembered.is_none()
|| (projects.len() == 1 && projects.first() == remembered.as_ref()))
{
return Ok(remembered.clone());
}
session.remembered = None;
}
if req.noninteractive || !cfg.ui.enabled {
bail!("pending: destination requires a desktop choice; use --global or --project");
}
let choice = desktop::destination(
&projects,
&record
.source
.file_name()
.unwrap_or_default()
.to_string_lossy(),
)?
.context("pending: destination choice postponed")?;
if choice.skip {
bail!("skipped: destination import skipped");
}
if choice.remember && !req.dry_run {
session.remembered = Some((choice.project.clone(), projects));
}
Ok(choice.project)
}
}
}
fn variables(cfg: &Config, kicad: &Path) -> BTreeMap<String, String> {
let mut vars = cfg.libraries.variables.clone();
if let Some(version) = kicad
.file_name()
.and_then(|s| s.to_str())
.and_then(|s| s.split('.').next())
{
for (suffix, path) in [
("FOOTPRINT_DIR", "/usr/share/kicad/footprints"),
("3DMODEL_DIR", "/usr/share/kicad/3dmodels"),
("SYMBOL_DIR", "/usr/share/kicad/symbols"),
] {
if Path::new(path).exists()
&& std::env::var_os(format!("KICAD{version}_{suffix}")).is_none()
{
vars.entry(format!("KICAD{version}_{suffix}"))
.or_insert_with(|| path.into());
}
}
}
vars
}
fn index(
cfg: &Config,
dest: &Destination,
kicad: &Path,
vars: &BTreeMap<String, String>,
owned_nickname: &str,
) -> Result<Vec<Foot>> {
let global_table = kicad.join("fp-lib-table");
let local_table = dest.table_dir.join("fp-lib-table");
let libs = destination::effective_footprint_libraries(
&global_table,
dest.project.as_ref().map(|_| local_table.as_path()),
dest.project.as_deref(),
vars,
)?;
let mut out = vec![];
let mut seen = BTreeSet::new();
for lib in libs {
let nick = lib.nickname;
let path = lib.path;
if !path.is_dir() {
bail!(
"pending: registered footprint library {} does not exist",
path.display()
);
}
for e in fs::read_dir(&path)? {
let p = e?.path();
if p.extension().is_some_and(|e| e == "kicad_mod") {
seen.insert(p.clone());
out.push(Foot {
node: sexpr::parse(&fs::read_to_string(&p)?)?,
local: dest.project.as_ref().is_some_and(|r| p.starts_with(r)),
path: p,
nickname: Some(nick.clone()),
fingerprint: None,
});
}
}
}
// Existing managed/provider libraries and explicit unregistered roots can be
// copied into the destination, but must never supply an invalid library ID.
for root in std::iter::once(&cfg.libraries.path)
.chain(cfg.libraries.search_paths.iter())
.chain(std::iter::once(&dest.pretty_dir))
{
if !root.exists() {
if cfg.libraries.search_paths.contains(root) {
bail!(
"pending: configured search root {} is missing",
root.display()
);
}
continue;
}
for e in walkdir::WalkDir::new(root)
.follow_links(false)
.sort_by_file_name()
{
let e =
e.with_context(|| format!("pending: reading search root {}", root.display()))?;
let p = e.path();
if !e.file_type().is_file()
|| !p.extension().is_some_and(|e| e == "kicad_mod")
|| !seen.insert(p.to_owned())
{
continue;
}
let own = p.parent() == Some(dest.pretty_dir.as_path());
out.push(Foot {
node: sexpr::parse(&fs::read_to_string(p)?)?,
path: p.to_owned(),
nickname: (own && cfg.options.footprints).then(|| owned_nickname.into()),
local: dest.project.as_ref().is_some_and(|r| p.starts_with(r)),
fingerprint: None,
});
}
}
out.sort_by(|a, b| b.local.cmp(&a.local).then(a.path.cmp(&b.path)));
Ok(out)
}
fn model_choices(
node: &Node,
path: &Path,
bundle: &Path,
cfg: &Config,
vars: &BTreeMap<String, String>,
dest: &Destination,
req: &Request,
) -> Result<BTreeMap<String, PathBuf>> {
let mut choices = BTreeMap::new();
if !cfg.options.models {
return Ok(choices);
}
for model in node.children().iter().filter(|n| n.head() == Some("model")) {
let Some(reference) = model.value() else {
continue;
};
if reference.starts_with("${KICAD_EMBEDDED}") || reference.starts_with("kicad-embed://") {
continue;
}
let candidates = models::model_candidates(
reference,
path,
bundle,
&cfg.libraries.model_search_paths,
vars,
dest.project.as_deref(),
)?;
if candidates.len() > 1 {
if req.noninteractive || !cfg.ui.enabled {
bail!(
"pending: ambiguous 3D model {reference}: {}",
candidates
.iter()
.map(|p| p.display().to_string())
.collect::<Vec<_>>()
.join(", ")
);
}
let chosen =
desktop::choose_file(&format!("Select model for {reference}"), &candidates)?
.context("pending: model choice postponed")?;
choices.insert(reference.into(), chosen);
} else if let Some(p) = candidates.first() {
choices.insert(reference.into(), p.clone());
}
}
Ok(choices)
}
fn fingerprint(node: &Node, path: &Path, choices: &BTreeMap<String, PathBuf>) -> String {
let mut n = node.clone();
for model in n
.children_mut()
.iter_mut()
.filter(|n| n.head() == Some("model"))
{
if let Some(p) = model.value().and_then(|s| choices.get(s)) {
model.children_mut()[1] = Node::string(p.to_string_lossy());
}
}
matching::exact_fingerprint(&n, path)
}
fn unique_footprint(mut node: Node, dest: &Destination, plan: &Plan) -> Result<(Node, PathBuf)> {
let name = safe_name(node.value().context("footprint missing name")?)?;
let hash = storage::hash(node.to_string().as_bytes());
for n in 0..1000 {
let candidate = if n == 0 {
name.clone()
} else if n == 1 {
format!("{name}_{}", &hash[..8])
} else {
format!("{name}_{}_{}", &hash[..8], n)
};
node.children_mut()[1] = Node::string(&candidate);
let path = dest.pretty_dir.join(format!("{candidate}.kicad_mod"));
let existing = if let Some(v) = plan.writes.get(&path) {
Some(v.clone())
} else if path.exists() {
Some(fs::read(&path)?)
} else {
None
};
if existing
.as_ref()
.is_none_or(|v| sexpr::parse(&String::from_utf8_lossy(v)).ok().as_ref() == Some(&node))
{
return Ok((node, path));
}
}
bail!("cannot allocate a footprint filename")
}
fn binding_add(map: &mut BTreeMap<String, BTreeSet<String>>, alias: String, value: String) {
map.entry(alias).or_default().insert(value);
}
pub fn run(cfg: &Config, req: &Request, record: &mut Record, session: &mut Session) -> Result<()> {
let _lock = storage::Lock::acquire()?;
let bundle = sources::load(&record.source)
.with_context(|| format!("reading {}", record.source.display()))?;
let project = target(cfg, req, record, session)?;
let kicad = kicad_config(cfg, project.is_none())?;
let dest = Destination::new(&cfg.libraries.path, project.as_deref(), &kicad)?;
let target_id = if let Some(p) = &dest.project {
format!("project:{}", p.display())
} else {
format!("global:{}:{}", dest.root.display(), kicad.display())
};
if record
.destination
.as_ref()
.is_some_and(|saved| saved != &target_id)
{
bail!(
"pending: destination configuration changed since this import was reviewed; start a new manual import with the intended destination"
);
}
if !req.force
&& let Some(completed) = storage::records()?.into_iter().find(|r| {
r.status == "completed"
&& r.hash == record.hash
&& r.destination.as_ref() == Some(&target_id)
})
{
println!("Already imported into {target_id}; use --force to recheck");
if !req.dry_run && record.id != completed.id {
let path = config::data_dir()
.join("imports")
.join(format!("{}.json", record.id));
if let Ok(bytes) = fs::read(&path)
&& let Ok(old) = serde_json::from_slice::<Record>(&bytes)
&& old.status != "completed"
{
let _ = fs::remove_file(path);
}
}
return Ok(());
}
let previous_id = record.id.clone();
record.id = format!(
"{}-{}",
&record.hash[..16],
&storage::hash(target_id.as_bytes())[..8]
);
record.destination = Some(target_id);
record.project = project.clone();
if !req.dry_run {
record.save()?;
if previous_id != record.id {
let p = config::data_dir()
.join("imports")
.join(format!("{previous_id}.json"));
if let Ok(bytes) = fs::read(&p)
&& let Ok(old) = serde_json::from_slice::<Record>(&bytes)
&& old.status != "completed"
{
let _ = fs::remove_file(p);
}
}
}
let vars = variables(cfg, &kicad);
let mut plan = Plan::default();
plan.messages.push(format!(
"Source: {} ({})",
record.source.display(),
bundle.provider
));
plan.messages
.push(format!("Destination: {}", dest.root.display()));
let (fp_table, fp_nickname) = destination::register_table(
text(&dest.table_dir.join("fp-lib-table"))?.as_deref(),
"fp_lib_table",
dest.nickname(),
&dest.footprint_uri(),
)?;
let (sym_table, sym_nickname) = destination::register_table(
text(&dest.table_dir.join("sym-lib-table"))?.as_deref(),
"sym_lib_table",
dest.nickname(),
&dest.symbol_uri(),
)?;
let mut library_index = index(cfg, &dest, &kicad, &vars, &fp_nickname)
.context("pending: footprint search could not be completed")?;
let mut bindings: BTreeMap<String, BTreeSet<String>> = BTreeMap::new();
let mut decisions = vec![];
let decision_dir = config::data_dir().join("decisions");
let mut assets: Vec<(PathBuf, Node, Vec<String>, PathBuf)> = bundle
.footprints
.iter()
.map(|(p, n)| (p.clone(), n.clone(), vec![], bundle.root.clone()))
.collect();
// Symbols sometimes reference an already installed footprint without bundling it.
// Bring that dependency through the same portable-copy/model pipeline.
if cfg.options.symbols {
let bundled_aliases: BTreeSet<String> = bundle
.footprints
.iter()
.flat_map(|(p, n)| {
let parent = p
.parent()
.and_then(Path::file_stem)
.and_then(|s| s.to_str())
.unwrap_or("");
[
n.value().unwrap_or(""),
p.file_stem().and_then(|s| s.to_str()).unwrap_or(""),
]
.into_iter()
.map(move |name| format!("{parent}:{name}"))
})
.collect();
let mut added = BTreeSet::new();
for (_, lib) in &bundle.symbols {
for symbol in lib.children().iter().filter(|n| n.head() == Some("symbol")) {
for prop in symbol
.children()
.iter()
.filter(|n| n.head() == Some("property") && n.value() == Some("Footprint"))
{
let Some(reference) = prop.children().get(2).and_then(Node::atom) else {
continue;
};
let Some((nick, name)) = reference.split_once(':') else {
continue;
};
if bundled_aliases.contains(reference) || !added.insert(reference.to_owned()) {
continue;
}
if let Some(f) = library_index
.iter()
.find(|f| f.nickname.as_deref() == Some(nick) && f.name() == name)
{
assets.push((
f.path.clone(),
f.node.clone(),
vec![reference.to_owned()],
f.path.parent().unwrap_or(Path::new(".")).to_owned(),
));
}
}
}
}
}
for (source, node, extra_aliases, source_root) in &assets {
let choices = model_choices(node, source, source_root, cfg, &vars, &dest, req)?;
let incoming_hash = fingerprint(node, source, &choices);
let exact = if cfg.matching.enabled {
library_index
.iter()
.position(|f| f.fingerprint() == incoming_hash)
} else {
None
};
let potential: Vec<usize> = if cfg.matching.enabled {
library_index
.iter()
.enumerate()
.filter(|(_, f)| matching::similar(node, &f.node, cfg.matching.tolerance))
.map(|(i, _)| i)
.collect()
} else {
vec![]
};
let remembered = fs::read(decision_dir.join(format!("{incoming_hash}.json")))
.ok()
.and_then(|v| serde_json::from_slice::<Decision>(&v).ok())
.and_then(|d| {
library_index
.iter()
.position(|f| f.path == d.path && f.fingerprint() == d.fingerprint)
});
let reuse = if exact.is_some() && cfg.matching.exact == "reuse" {
exact
} else if remembered.is_some() {
remembered
} else if !potential.is_empty() || exact.is_some() {
let mut candidates = potential;
if let Some(i) = exact
&& !candidates.contains(&i)
{
candidates.insert(0, i);
}
match cfg.matching.ambiguous.as_str() {
"separate" => None,
"skip" => bail!(
"skipped: footprint {} has uncertain candidates",
node.value().unwrap_or("?")
),
_ => {
if req.noninteractive || !cfg.ui.enabled {
bail!(
"pending: footprint {} requires comparison with {} candidates",
node.value().unwrap_or("?"),
candidates.len()
);
}
if req.dry_run {
bail!(
"pending: dry run requires footprint comparison for {}",
node.value().unwrap_or("?")
);
}
let items = candidates
.iter()
.map(|i| {
let f = &library_index[*i];
desktop::Candidate {
label: format!(
"{}:{}",
f.nickname.as_deref().unwrap_or("unregistered"),
f.name()
),
path: f.path.clone(),
raw: f.node.to_string(),
differences: matching::differences(node, &f.node),
}
})
.collect::<Vec<_>>();
match desktop::compare(&node.to_string(), &items)?
.context("pending: footprint comparison postponed")?
{
desktop::FootprintChoice::Reuse(n) => {
let i = *candidates.get(n).context("invalid candidate selection")?;
decisions.push((
incoming_hash.clone(),
Decision {
path: library_index[i].path.clone(),
fingerprint: library_index[i].fingerprint(),
},
));
Some(i)
}
desktop::FootprintChoice::Separate => None,
desktop::FootprintChoice::Skip => {
bail!("skipped: component skipped in footprint comparison")
}
}
}
}
} else {
None
};
let selected = reuse.map(|i| &library_index[i]);
if let Some(f) = selected
&& f.path.exists()
&& !plan.writes.contains_key(&f.path)
{
let fresh = sexpr::parse(&fs::read_to_string(&f.path)?)?;
if matching::exact_fingerprint(&fresh, &f.path) != f.fingerprint() {
bail!(
"pending: selected footprint changed during comparison: {}",
f.path.display()
);
}
}
let in_place = selected.is_some_and(|f| {
f.nickname.is_some() && (dest.project.is_none() || !cfg.destination.portable || f.local)
});
let final_id = if in_place {
let f = selected.unwrap();
if dest.project.is_some() && cfg.destination.portable {
let mut repaired = f.node.clone();
let root = f.path.parent().unwrap_or(Path::new("."));
let chosen_choices =
model_choices(&repaired, &f.path, root, cfg, &vars, &dest, req)?;
let model_plan = models::repair_models_with_choices(
&mut repaired,
&f.path,
root,
&dest,
&cfg.libraries.model_search_paths,
&vars,
cfg.options.models,
&chosen_choices,
)?;
if repaired != f.node && !cfg.options.footprints {
bail!(
"pending: footprint import disabled but portable model repair requires updating {}",
f.path.display()
);
}
for (p, bytes) in model_plan.copies {
plan.put(p, bytes)?;
}
plan.messages.extend(model_plan.messages);
if repaired != f.node {
plan.put(f.path.clone(), encoded(&repaired))?;
}
}
plan.messages
.push(format!("Reuse footprint {}", f.path.display()));
format!("{}:{}", f.nickname.as_ref().unwrap(), f.name())
} else {
if !cfg.options.footprints {
bail!(
"pending: footprint import disabled and {} has no usable registered match",
node.value().unwrap_or("?")
);
}
let (mut chosen, chosen_path, chosen_root) = if let Some(f) = selected {
(
f.node.clone(),
f.path.clone(),
f.path.parent().unwrap_or(Path::new(".")).to_owned(),
)
} else {
(node.clone(), source.clone(), source_root.clone())
};
let chosen_choices = if selected.is_some() {
model_choices(&chosen, &chosen_path, &chosen_root, cfg, &vars, &dest, req)?
} else {
choices
};
let chosen_hash = fingerprint(&chosen, &chosen_path, &chosen_choices);
let model_plan = models::repair_models_with_choices(
&mut chosen,
&chosen_path,
&chosen_root,
&dest,
&cfg.libraries.model_search_paths,
&vars,
cfg.options.models,
&chosen_choices,
)?;
for (p, bytes) in model_plan.copies {
plan.put(p, bytes)?;
}
plan.messages.extend(model_plan.messages);
let (chosen, path) = unique_footprint(chosen, &dest, &plan)?;
let name = chosen.value().unwrap().to_owned();
plan.messages
.push(format!("Install footprint {}", path.display()));
plan.put(path.clone(), encoded(&chosen))?;
library_index.insert(
0,
Foot {
path,
node: chosen,
nickname: Some(fp_nickname.clone()),
local: dest.project.is_some(),
fingerprint: Some(chosen_hash),
},
);
format!("{fp_nickname}:{name}")
};
for alias in extra_aliases {
binding_add(&mut bindings, alias.clone(), final_id.clone());
}
if !extra_aliases.is_empty() {
continue;
}
let original = node.value().context("footprint missing name")?.to_owned();
binding_add(&mut bindings, original.clone(), final_id.clone());
if let Some(stem) = source.file_stem().and_then(|s| s.to_str()) {
binding_add(&mut bindings, stem.into(), final_id.clone());
}
if let Some(parent) = source
.parent()
.and_then(Path::file_stem)
.and_then(|s| s.to_str())
{
binding_add(&mut bindings, format!("{parent}:{original}"), final_id);
}
}
if cfg.options.footprints && !assets.is_empty() {
plan.directories.push(dest.pretty_dir.clone());
plan.put(dest.table_dir.join("fp-lib-table"), fp_table.into_bytes())?;
}
if cfg.options.symbols && !bundle.symbols.is_empty() {
let current = if let Some(raw) = text(&dest.symbol_path)? {
sexpr::parse(&raw)?
} else {
Node::list(vec![
Node::symbol("kicad_symbol_lib"),
Node::list(vec![Node::symbol("version"), Node::symbol("20211014")]),
Node::list(vec![Node::symbol("generator"), Node::string("uniloader")]),
])
};
if current.head() != Some("kicad_symbol_lib") {
bail!("target is not a symbol library");
}
let mut incoming = vec![];
for (_, lib) in &bundle.symbols {
let mut rewritten = lib.clone();
for symbol in rewritten
.children_mut()
.iter_mut()
.filter(|n| n.head() == Some("symbol"))
{
for prop in symbol
.children_mut()
.iter_mut()
.filter(|n| n.head() == Some("property") && n.value() == Some("Footprint"))
{
let original = prop
.children()
.get(2)
.and_then(Node::atom)
.unwrap_or("")
.to_owned();
if original.is_empty() {
continue;
}
let resolved = bindings
.get(&original)
.or_else(|| bindings.get(original.rsplit(':').next().unwrap_or(&original)));
let id = if let Some(set) = resolved {
if set.len() != 1 {
bail!("pending: ambiguous symbol footprint association {original}");
}
set.first().unwrap().clone()
} else {
let Some((libname, fpname)) = original.split_once(':') else {
bail!("pending: no footprint found for {original}");
};
let found = library_index
.iter()
.find(|f| f.nickname.as_deref() == Some(libname) && f.name() == fpname);
let f = found.with_context(|| {
format!("pending: missing registered footprint {original}")
})?;
if dest.project.is_some() && cfg.destination.portable && !f.local {
bail!(
"pending: symbol references external footprint {original} not bundled; import its footprint into the project first"
);
}
original
};
prop.children_mut()[2] = Node::string(id);
}
}
incoming.push(rewritten);
}
let (merged, messages) = crate::symbols::merge(current, incoming)?;
plan.messages.extend(messages);
plan.put(dest.symbol_path.clone(), encoded(&merged))?;
plan.put(dest.table_dir.join("sym-lib-table"), sym_table.into_bytes())?;
plan.messages.push(format!(
"Symbol library {sym_nickname}: {}",
dest.symbol_path.display()
));
}
for message in &plan.messages {
println!("{message}");
}
for p in plan.writes.keys() {
println!(
"{} {}",
if req.dry_run { "Would write" } else { "Write" },
p.display()
);
}
if req.dry_run {
println!("Dry run: no libraries or decisions written");
return Ok(());
}
record.status = "importing".into();
record.message = plan.messages.join("\n");
record.save()?;
plan.commit(record)?;
for (key, decision) in decisions {
fs::create_dir_all(&decision_dir)?;
storage::atomic(
&decision_dir.join(format!("{key}.json")),
&serde_json::to_vec_pretty(&decision)?,
)?;
}
record.status = "completed".into();
record.message = plan.messages.join("\n");
record.save()?;
if !cfg.options.preserve_source && record.source.is_file() {
fs::remove_file(&record.source).context("import succeeded but source cleanup failed")?;
}
println!("Imported successfully. KiCad may need a library refresh/reopen.");
Ok(())
}
+418 -424
View File
@@ -1,458 +1,452 @@
// Copyright (C) 2026 Hector van der Aa <hector@h3cx.dev> // Copyright (C) 2026 Hector van der Aa <hector@h3cx.dev>
// SPDX-License-Identifier: GPL-3.0-or-later // SPDX-License-Identifier: GPL-3.0-or-later
mod config;
mod desktop;
mod destination;
mod doctor;
mod gui;
mod importer;
mod matching;
mod models;
mod preview;
mod sexpr;
mod sources;
mod storage;
mod symbols;
use anyhow::{Context, Result, bail};
use clap::{Parser, Subcommand};
use config::Config;
use importer::{Request, Session};
use notify::{
Event, EventKind, RecursiveMode, Watcher,
event::{AccessKind, AccessMode},
};
use std::{ use std::{
env, collections::{BTreeMap, BTreeSet},
fs::{self, File}, fs,
io::Read,
path::{Path, PathBuf}, path::{Path, PathBuf},
sync::mpsc, sync::mpsc,
time::{Duration, Instant, SystemTime},
}; };
use storage::Record;
use notify::{Event, EventKind, RecursiveMode, Watcher, event::CreateKind}; #[derive(Parser)]
use serde::{Deserialize, Serialize}; #[command(
use thiserror::Error; version,
use zip::ZipArchive; about = "Import downloaded KiCad components into global or portable project libraries"
)]
#[derive(Debug, Deserialize, Serialize)] struct Cli {
struct Config { #[arg(long, global = true)]
libraries: LibConfig, config: Option<PathBuf>,
#[arg(long, global = true)]
noninteractive: bool,
#[command(subcommand)]
command: Option<Commands>,
}
#[derive(Subcommand)]
enum Commands {
// Isolate the native event loop on its own main thread while the watcher
// continues receiving downloads. This is the same executable, not a helper.
#[command(name = "__dialog", hide = true)]
Dialog,
/// Watch configured download directories (also the default command).
Watch,
/// Import a ZIP archive or extracted component directory.
Import {
source: PathBuf,
#[arg(long, conflicts_with = "project")]
global: bool,
#[arg(long)]
project: Option<PathBuf>,
#[arg(long)]
dry_run: bool,
#[arg(long)]
force: bool,
},
/// List pending and failed imports; use --all for full history.
Pending {
#[arg(long)]
all: bool,
},
/// Retry an import listed by pending.
Retry {
id: String,
#[arg(long)]
dry_run: bool,
},
/// Check managed nicknames; --fix backs up and migrates libraries/references.
Doctor {
#[arg(long, required_unless_present = "project", conflicts_with = "project")]
global: bool,
#[arg(long)]
project: Option<PathBuf>,
#[arg(long)]
fix: bool,
},
/// Show project suggestions from running KiCad processes.
Discover,
} }
impl Config { fn execute(cfg: &Config, req: &Request, mut record: Record, session: &mut Session) -> Result<()> {
fn load_or_create() -> Result<Self, ConfigError> { match importer::run(cfg, req, &mut record, session) {
let config_path = Ok(()) => Ok(()),
PathBuf::from(env::var("HOME").unwrap()).join(".config/uniloader/config.toml"); Err(error) => {
let message = format!("{error:#}");
if config_path.exists() { if message.contains("another UniLoader import is active") {
let contents = fs::read_to_string(&config_path).map_err(|_| ConfigError::ConfigErr)?; return Err(error);
let config: Config = toml::from_str(&contents).map_err(|_| ConfigError::ConfigErr)?; }
Ok(config) record.status = if message.contains("pending:") {
"pending"
} else if message.contains("skipped:") {
"skipped"
} else {
"failed"
}
.into();
record.message = message.clone();
if !req.dry_run {
record.save()?;
}
bail!(
"{}: {message} (import {}; retry with `uniloader retry {}`)",
record.source.display(),
record.id,
record.id
)
}
}
}
fn new_record(source: PathBuf, req: &Request) -> Result<Record> {
let source = fs::canonicalize(source)?;
let hash = storage::source_hash(&source)?;
let mut record = Record::new(source, hash);
if req.global || req.project.is_some() {
let target = if req.global {
"global".to_owned()
} else { } else {
fs::create_dir_all(config_path.parent().unwrap()) config::expand(req.project.as_ref().unwrap())
.map_err(|_| ConfigError::ConfigErr)?; .to_string_lossy()
let config = Config { .into_owned()
libraries: LibConfig { };
path: PathBuf::from("~/KiCad/libraries/"), record.id = format!("{}-{}", record.id, &storage::hash(target.as_bytes())[..8]);
}, }
}; Ok(record)
let contents = toml::to_string_pretty(&config).map_err(|_| ConfigError::ConfigErr)?; }
fs::write(&config_path, contents).map_err(|_| ConfigError::ConfigErr)?; struct Waiting {
Ok(config) size: u64,
modified: Option<SystemTime>,
changed: Instant,
first: Instant,
}
fn zip_file(p: &Path) -> bool {
p.is_file() && p.extension().is_some_and(|e| e.eq_ignore_ascii_case("zip"))
}
fn complete_zip(p: &Path) -> Result<bool> {
let mut zip = zip::ZipArchive::new(fs::File::open(p)?)?;
let mut supported = false;
for i in 0..zip.len() {
let f = zip.by_index(i)?;
let name = f.name().to_ascii_lowercase();
if name.ends_with(".kicad_sym") || name.ends_with(".kicad_mod") {
supported = true;
} }
} }
Ok(supported)
} }
fn event_requires_scan(event: &Event) -> bool {
#[derive(Debug, Deserialize, Serialize, Clone)] // Reading a watched directory emits OPEN events on Linux, including from
struct LibConfig { // our own scans. Only a writer closing a file can make access relevant.
path: PathBuf, event.need_rescan()
|| !matches!(event.kind, EventKind::Access(_))
|| matches!(
event.kind,
EventKind::Access(AccessKind::Close(AccessMode::Write))
)
} }
fn watch(cfg: Config, noninteractive: bool) -> Result<()> {
enum SourceType { let _watch_lock = storage::Lock::watcher()?;
SymacSys, let (events_tx, events_rx) = mpsc::channel();
UltraLibrarian, let mut watcher = notify::recommended_watcher(move |event: notify::Result<Event>| {
SnapMagic, // Filter before queuing so reads cannot keep the receive loop awake.
} if event.as_ref().map_or(true, event_requires_scan) {
let _ = events_tx.send(event);
#[derive(Error, Debug)] }
enum ConfigError { })?;
#[error("Config error")] for dir in &cfg.watch.directories {
ConfigErr, if !dir.is_dir() {
} bail!(
"watch directory {} does not exist; create it or configure watch.directories",
#[derive(Error, Debug)] dir.display()
enum SourceError { );
#[error("Unknown Source")] }
SourceUnknown, watcher.watch(dir, RecursiveMode::NonRecursive)?;
#[error("Filename Error")] println!("Watching {}", dir.display());
FilenameError, }
} let (jobs_tx, jobs_rx) = mpsc::channel::<Record>();
let (done_tx, done_rx) = mpsc::channel::<String>();
#[derive(Error, Debug)] let worker_cfg = cfg.clone();
enum ImportError { std::thread::spawn(move || {
#[error("Unknown Path")] let mut session = Session::default();
InvalidPath, let req = Request {
#[error("Corrputed Zip")] noninteractive,
InvalidZip, ..Request::default()
} };
for record in jobs_rx {
struct SymbolLibrary { let id = record.id.clone();
version: String, if let Err(e) = execute(&worker_cfg, &req, record, &mut session) {
symbols: Vec<Symbol>, eprintln!("{e:#}");
} desktop::notify("Import needs attention. Run uniloader pending for details.");
} else {
struct Symbol { desktop::notify("Component import finished. See the service log for details.");
name: String, }
raw: String, let _ = done_tx.send(id);
} }
});
impl SymbolLibrary { let mut waiting: BTreeMap<PathBuf, Waiting> = BTreeMap::new();
fn new() -> Self { let mut submitted: BTreeSet<String> = BTreeSet::new();
SymbolLibrary { let mut seen: BTreeMap<PathBuf, (u64, Option<SystemTime>)> = BTreeMap::new();
version: "20211014".to_string(), // A restart resumes pending choices once, without endlessly reopening closed dialogs.
symbols: Vec::new(), for r in storage::records()? {
if ["pending", "importing"].contains(&r.status.as_str()) && r.source.exists() {
submitted.insert(r.id.clone());
jobs_tx.send(r)?;
} }
} }
fn write(&self) -> String { let scan = |waiting: &mut BTreeMap<PathBuf, Waiting>,
let mut out = format!( seen: &BTreeMap<PathBuf, (u64, Option<SystemTime>)>| {
"(kicad_symbol_lib\n\t(version {})\n\t(generator \"uniloader\")\n", for dir in &cfg.watch.directories {
self.version, if let Ok(entries) = fs::read_dir(dir) {
); for entry in entries.flatten() {
let p = entry.path();
for sym in &self.symbols { if !zip_file(&p) {
out.push_str(&sym.raw); continue;
out.push('\n'); }
} if let Ok(m) = entry.metadata() {
let signature = (m.len(), m.modified().ok());
out.push(')'); if seen.get(&p) == Some(&signature) {
out continue;
} }
waiting
fn from_str(input: &str) -> Self { .entry(p)
let input = input.replace("\r\n", "\n"); .and_modify(|w| {
let mut version = String::new(); if (w.size, w.modified) != signature {
let mut symbols = Vec::new(); w.size = signature.0;
w.modified = signature.1;
if let Some(v_start) = input.find("(version ") { w.changed = Instant::now();
let rest = &input[v_start + 9..]; }
if let Some(v_end) = rest.find(')') { })
version = rest[..v_end].trim().to_string(); .or_insert(Waiting {
size: signature.0,
modified: signature.1,
changed: Instant::now(),
first: Instant::now(),
});
}
}
} }
} }
};
scan(&mut waiting, &seen);
let mut last_scan = Instant::now();
loop {
match events_rx.recv_timeout(Duration::from_millis(500)) {
Ok(event) => {
let mut rescan = false;
// Bound draining so sustained downloads cannot starve retries.
for event in std::iter::once(event).chain(events_rx.try_iter().take(1023)) {
match event {
Ok(_) => rescan = true,
Err(e) => eprintln!("watch error: {e}"),
}
}
if rescan {
scan(&mut waiting, &seen);
}
}
Err(mpsc::RecvTimeoutError::Timeout) => {}
Err(mpsc::RecvTimeoutError::Disconnected) => bail!("filesystem watcher disconnected"),
}
if last_scan.elapsed() >= Duration::from_secs(5) {
scan(&mut waiting, &seen);
last_scan = Instant::now();
}
for id in done_rx.try_iter() {
submitted.remove(&id);
}
let ready = waiting
.iter()
.filter(|(_, w)| w.changed.elapsed() >= Duration::from_secs(cfg.watch.settle_seconds))
.map(|(p, _)| p.clone())
.collect::<Vec<_>>();
for p in ready {
if !p.exists() {
waiting.remove(&p);
continue;
}
match complete_zip(&p) {
Ok(supported) => {
let w = waiting.remove(&p).unwrap();
seen.insert(p.clone(), (w.size, w.modified));
if !supported {
continue;
}
let req = Request::default();
match new_record(p.clone(), &req) {
Ok(r) => {
let history = storage::records()?;
// Completed files stay ignored until --force/manual import; pending
// files are resumed once at startup, not repeatedly every scan.
if history
.iter()
.any(|old| old.hash == r.hash && old.source == r.source)
|| !submitted.insert(r.id.clone())
{
continue;
}
r.save()?;
jobs_tx.send(r)?;
}
Err(e) => eprintln!("{}: {e:#}", p.display()),
}
}
Err(e) => {
if waiting[&p].first.elapsed() >= Duration::from_secs(cfg.watch.retry_seconds) {
let w = waiting.remove(&p).unwrap();
seen.insert(p.clone(), (w.size, w.modified));
if let Ok(mut r) = new_record(p.clone(), &Request::default()) {
r.status = "failed".into();
r.message = format!("download never became a readable ZIP: {e:#}");
r.save()?;
}
eprintln!(
"{}: ZIP still unreadable after retry window: {e:#}",
p.display()
);
}
}
}
}
}
}
let mut i = 0; fn main() -> Result<()> {
let chars: Vec<char> = input.chars().collect(); let cli = Cli::parse();
if matches!(cli.command, Some(Commands::Dialog)) {
while i < chars.len() { return desktop::run_internal_dialog();
if input[i..].starts_with("(symbol ") { }
let start = i; let cfg = Config::load(cli.config.as_deref())?;
let mut depth = 0; match cli.command.unwrap_or(Commands::Watch) {
Commands::Dialog => unreachable!("handled before configuration loading"),
while i < chars.len() { Commands::Watch => watch(cfg, cli.noninteractive),
if chars[i] == '(' { Commands::Import {
depth += 1; source,
} else if chars[i] == ')' { global,
depth -= 1; project,
dry_run,
if depth == 0 { force,
let end = i + 1; } => {
let raw = input[start..end].to_string(); let req = Request {
global,
let name = raw.split('"').nth(1).unwrap_or("unknown").to_string(); project,
dry_run,
symbols.push(Symbol { name, raw }); noninteractive: cli.noninteractive,
force,
break; };
let record = new_record(source, &req)?;
execute(&cfg, &req, record, &mut Session::default())
}
Commands::Pending { all } => {
for r in storage::records()? {
if all || ["pending", "failed", "importing"].contains(&r.status.as_str()) {
println!(
"{} [{}] {}\n {}\n {}",
r.id,
r.status,
r.source.display(),
r.destination.as_deref().unwrap_or("destination not chosen"),
r.message
);
if all {
for p in &r.changed {
println!(" Changed: {}", p.display());
}
for p in &r.backups {
println!(" Backup: {}", p.display());
} }
} }
i += 1;
} }
} }
i += 1; Ok(())
} }
Commands::Retry { id, dry_run } => {
Self { version, symbols } let record = storage::records()?
} .into_iter()
} .find(|r| r.id == id)
.context("unknown import ID; run uniloader pending")?;
fn main() -> notify::Result<()> { let hash = storage::source_hash(&record.source)?;
let config = Config::load_or_create().unwrap(); if hash != record.hash {
println!("{:?}", config.libraries.path); bail!(
let (tx, rx) = mpsc::channel::<notify::Result<Event>>(); "source changed since queued; use `uniloader import` to review it as a new component"
let mut watcher = notify::recommended_watcher(tx)?; );
let downloads = PathBuf::from(env::var("HOME").unwrap()).join("Downloads"); }
watcher.watch(&downloads, RecursiveMode::NonRecursive)?; execute(
&cfg,
for res in rx { &Request {
match res { dry_run,
Ok(event) => { noninteractive: cli.noninteractive,
if event.kind != EventKind::Create(CreateKind::File) { force: true,
continue; ..Request::default()
} },
if !validate_zip(&event.paths[0]) { record,
continue; &mut Session::default(),
} )
}
let Ok(source_type) = find_source_type(&event.paths[0]) else { Commands::Doctor {
continue; global: _,
}; project,
fix,
match source_type { } => doctor::run(&cfg, project.as_deref(), fix),
SourceType::SymacSys => { Commands::Discover => {
println!("SymacSys detected, Installing..."); let p = desktop::projects();
let Ok(_) = import_symacsys(&event.paths[0], &config) else { if p.is_empty() {
continue; println!("No project suggestions found. Browse or use --project.");
}; } else {
} for path in p {
SourceType::UltraLibrarian => { println!("{} (process suggestion)", path.display());
println!("UltraLibrarian detected, Installing...");
let Ok(_) = import_ultralibrarian(&event.paths[0], &config) else {
continue;
};
}
SourceType::SnapMagic => {
println!("SnapMagic detected, Installing...");
let Ok(_) = import_snapmagic(&event.paths[0], &config) else {
continue;
};
}
} }
} }
Err(e) => println!("watch error: {:?}", e), Ok(())
} }
} }
Ok(())
} }
fn validate_zip(path: &Path) -> bool { #[cfg(test)]
path.is_file() mod watch_tests {
&& path use super::*;
.extension() use notify::event::{CreateKind, Flag, ModifyKind, RemoveKind};
.and_then(|e| e.to_str())
.map(|e| e.eq_ignore_ascii_case("zip"))
.unwrap_or(false)
}
fn find_source_type(path: &Path) -> Result<SourceType, SourceError> { #[test]
let name = path fn scans_changes_and_write_closes_but_ignores_read_access() {
.file_name() for kind in [
.ok_or(SourceError::FilenameError)? EventKind::Any,
.to_str() EventKind::Create(CreateKind::File),
.ok_or(SourceError::FilenameError)?; EventKind::Modify(ModifyKind::Any),
EventKind::Remove(RemoveKind::File),
if name.starts_with("LIB_") { EventKind::Access(AccessKind::Close(AccessMode::Write)),
return Ok(SourceType::SymacSys); ] {
} else if name.starts_with("ul_") { assert!(event_requires_scan(&Event::new(kind)), "{kind:?}");
return Ok(SourceType::UltraLibrarian); }
} for kind in [
AccessKind::Any,
let file = File::open(path).map_err(|_| SourceError::SourceUnknown)?; AccessKind::Read,
let mut archive = ZipArchive::new(file).map_err(|_| SourceError::SourceUnknown)?; AccessKind::Open(AccessMode::Read),
for i in 0..archive.len() { AccessKind::Open(AccessMode::Write),
let Ok(mut file) = archive.by_index(i) else { AccessKind::Close(AccessMode::Read),
continue; ] {
}; let event = Event::new(EventKind::Access(kind));
let name = file.name(); assert!(!event_requires_scan(&event), "{kind:?}");
assert!(event_requires_scan(&event.set_flag(Flag::Rescan)));
if name.to_lowercase().contains("how-to-import") {
let mut contents = String::new();
file.read_to_string(&mut contents)
.map_err(|_| SourceError::SourceUnknown)?;
if contents.to_lowercase().contains("snapeda") {
return Ok(SourceType::SnapMagic);
}
} }
} }
Err(SourceError::SourceUnknown)
}
fn extract_zip(path: &Path, config: &Config) -> Result<PathBuf, ImportError> {
let file = File::open(path).map_err(|_| ImportError::InvalidPath)?;
let mut archive = ZipArchive::new(file).map_err(|_| ImportError::InvalidZip)?;
let Some(file_name) = path.file_stem() else {
return Err(ImportError::InvalidPath);
};
let tmp_path = config.libraries.path.join("tmp").join(file_name);
fs::create_dir_all(&tmp_path).map_err(|_| ImportError::InvalidPath)?;
archive
.extract(&tmp_path)
.map_err(|_| ImportError::InvalidPath)?;
fs::remove_file(path).map_err(|_| ImportError::InvalidPath)?;
Ok(tmp_path)
}
fn import_symacsys(path: &Path, config: &Config) -> Result<(), ImportError> {
let tmp_path = extract_zip(path, config)?;
let data_dir = fs::read_dir(&tmp_path)
.map_err(|_| ImportError::InvalidPath)?
.filter_map(|e| e.ok())
.find(|entry| entry.file_type().map(|t| t.is_dir()).unwrap_or(false))
.map(|entry| entry.path())
.ok_or(ImportError::InvalidPath)?;
let kicad_dir = data_dir.join("KiCad");
let entries = fs::read_dir(&kicad_dir)
.map(|rd| rd.filter_map(|e| e.ok()).collect::<Vec<_>>())
.unwrap_or_default();
let sym_files: Vec<PathBuf> = entries
.iter()
.filter(|entry| {
entry
.path()
.extension()
.map(|e| e == "kicad_sym")
.unwrap_or(false)
})
.map(|entry| entry.path())
.collect();
let mod_files: Vec<PathBuf> = entries
.iter()
.filter(|entry| {
entry
.path()
.extension()
.map(|e| e == "kicad_mod")
.unwrap_or(false)
})
.map(|entry| entry.path())
.collect();
let root = config.libraries.path.join("SymacSys");
let sym = root.join("SymacSys_Components.kicad_sym");
let pretty = root.join("SymacSys_Footprints.pretty");
install_library(sym_files, mod_files, &root, &sym, &pretty)?;
fs::remove_dir_all(tmp_path).map_err(|_| ImportError::InvalidPath)?;
Ok(())
}
fn import_ultralibrarian(path: &Path, config: &Config) -> Result<(), ImportError> {
let tmp_path = extract_zip(path, config)?;
let data_dir = fs::read_dir(&tmp_path)
.map_err(|_| ImportError::InvalidPath)?
.filter_map(|e| e.ok())
.find(|entry| entry.file_type().map(|t| t.is_dir()).unwrap_or(false))
.map(|entry| entry.path())
.ok_or(ImportError::InvalidPath)?;
let pretty_dir = data_dir.join("footprints.pretty");
let root_entries = fs::read_dir(&data_dir)
.map(|rd| rd.filter_map(|e| e.ok()).collect::<Vec<_>>())
.unwrap_or_default();
let pretty_entries = fs::read_dir(&pretty_dir)
.map(|rd| rd.filter_map(|e| e.ok()).collect::<Vec<_>>())
.unwrap_or_default();
let sym_files: Vec<PathBuf> = root_entries
.iter()
.filter(|entry| {
entry
.path()
.extension()
.map(|e| e == "kicad_sym")
.unwrap_or(false)
})
.map(|entry| entry.path())
.collect();
let mod_files: Vec<PathBuf> = pretty_entries
.iter()
.filter(|entry| {
entry
.path()
.extension()
.map(|e| e == "kicad_mod")
.unwrap_or(false)
})
.map(|entry| entry.path())
.collect();
let root = config.libraries.path.join("UltraLibrarian");
let sym = root.join("UltraLibrarian_Components.kicad_sym");
let pretty = root.join("UltraLibrarian_Footprints.pretty");
install_library(sym_files, mod_files, &root, &sym, &pretty)?;
fs::remove_dir_all(tmp_path).map_err(|_| ImportError::InvalidPath)?;
Ok(())
}
fn import_snapmagic(path: &Path, config: &Config) -> Result<(), ImportError> {
let data_dir = extract_zip(path, config)?;
let entries = fs::read_dir(&data_dir)
.map(|rd| rd.filter_map(|e| e.ok()).collect::<Vec<_>>())
.unwrap_or_default();
let sym_files: Vec<PathBuf> = entries
.iter()
.filter(|entry| {
entry
.path()
.extension()
.map(|e| e == "kicad_sym")
.unwrap_or(false)
})
.map(|entry| entry.path())
.collect();
let mod_files: Vec<PathBuf> = entries
.iter()
.filter(|entry| {
entry
.path()
.extension()
.map(|e| e == "kicad_mod")
.unwrap_or(false)
})
.map(|entry| entry.path())
.collect();
let root = config.libraries.path.join("SnapMagic");
let sym = root.join("SnapMagic_components.kicad_sym");
let pretty = root.join("SnapMagic_Footprints.pretty");
install_library(sym_files, mod_files, &root, &sym, &pretty)?;
fs::remove_dir_all(data_dir).map_err(|_| ImportError::InvalidPath)?;
Ok(())
}
fn install_library(
sym_files: Vec<PathBuf>,
mod_files: Vec<PathBuf>,
lib_root: &Path,
sym_file: &Path,
pretty_dir: &Path,
) -> Result<(), ImportError> {
fs::create_dir_all(lib_root).map_err(|_| ImportError::InvalidPath)?;
fs::create_dir_all(pretty_dir).map_err(|_| ImportError::InvalidPath)?;
// Load or create symbol library
let mut current_lib = if sym_file.exists() {
let contents = fs::read_to_string(sym_file).map_err(|_| ImportError::InvalidPath)?;
SymbolLibrary::from_str(&contents)
} else {
SymbolLibrary::new()
};
// Install symbols
for sym_path in sym_files {
let sym_str = fs::read_to_string(&sym_path).map_err(|_| ImportError::InvalidPath)?;
let new_lib = SymbolLibrary::from_str(&sym_str);
for symbol in new_lib.symbols {
if !current_lib.symbols.iter().any(|s| s.name == symbol.name) {
current_lib.symbols.push(symbol);
}
}
}
if !current_lib.symbols.is_empty() {
let output = current_lib.write();
fs::write(sym_file, output).map_err(|_| ImportError::InvalidPath)?;
}
// Install footprints
for mod_path in mod_files {
let filename = mod_path.file_name().ok_or(ImportError::InvalidPath)?;
let target = pretty_dir.join(filename);
fs::copy(mod_path, target).map_err(|_| ImportError::InvalidPath)?;
}
Ok(())
} }
+346
View File
@@ -0,0 +1,346 @@
//! Conservative exact matching and deliberately simple candidate detection.
//!
//! Exact normalization ignores only the root footprint name and generated
//! uuid/tstamp records in recognized footprint elements. All other fields
//! (including unknown subtrees) remain.
use crate::sexpr::Node;
use sha2::{Digest, Sha256};
use std::{
collections::BTreeMap,
fs,
path::{Path, PathBuf},
};
fn digest(bytes: &[u8]) -> String {
format!("{:x}", Sha256::digest(bytes))
}
fn contextual_model(path: &str, source: &Path) -> String {
let normalized = path.replace('\\', "/");
let mut expanded = normalized.clone();
// Expand known environment variables, leaving unsupported ones explicit.
while let Some(start) = expanded.find("${") {
let Some(end) = expanded[start + 2..].find('}') else {
break;
};
let end = start + 2 + end;
let key = &expanded[start + 2..end];
let value = if key == "KIPRJMOD" {
source
.parent()
.into_iter()
.flat_map(Path::ancestors)
.find(|p| {
fs::read_dir(p).ok().is_some_and(|entries| {
entries
.filter_map(Result::ok)
.any(|e| e.path().extension().is_some_and(|x| x == "kicad_pro"))
})
})
.map(|p| p.to_string_lossy().into_owned())
.or_else(|| std::env::var(key).ok())
} else {
std::env::var(key).ok()
};
let Some(value) = value else {
break;
};
expanded.replace_range(start..=end, &value);
// Avoid pathological self-referential variables.
if value.contains("${") {
break;
}
}
let path = PathBuf::from(&expanded);
let resolved = if path.is_absolute() {
path
} else {
source.parent().unwrap_or(Path::new(".")).join(path)
};
if let Ok(bytes) = fs::read(&resolved) {
return format!("model-content:{}", digest(&bytes));
}
// Relative model strings from different libraries are not proof of identity.
// Include lexical origin even when canonicalization cannot resolve the model.
let origin = fs::canonicalize(source).unwrap_or_else(|_| {
if source.is_absolute() {
source.to_path_buf()
} else {
std::env::current_dir().unwrap_or_default().join(source)
}
});
format!(
"unresolved-model:{}:{}",
origin.to_string_lossy(),
normalized
)
}
pub fn exact_fingerprint(node: &Node, source_path: &Path) -> String {
fn normalized(n: &Node, source: &Path, root: bool) -> Node {
match n {
Node::List(v) => {
// Do not normalize an unknown subtree: its identifiers may have
// semantics we do not understand.
if !matches!(
n.head(),
Some(
"footprint"
| "module"
| "pad"
| "fp_line"
| "fp_arc"
| "fp_rect"
| "fp_circle"
| "fp_poly"
| "fp_curve"
| "fp_text"
| "fp_text_box"
| "property"
| "model"
)
) {
return n.clone();
}
let mut children: Vec<Node> = v
.iter()
.filter(|c| !matches!(c.head(), Some("uuid" | "tstamp")))
.map(|c| normalized(c, source, false))
.collect();
if root && matches!(n.head(), Some("footprint" | "module")) && children.len() > 1 {
children[1] = Node::string("<footprint-name>");
}
if n.head() == Some("model")
&& let Some(path) = n.value()
{
children[1] = Node::string(contextual_model(path, source));
}
Node::List(children)
}
_ => n.clone(),
}
}
digest(normalized(node, source_path, true).to_string().as_bytes())
}
fn pads(n: &Node) -> BTreeMap<String, Vec<&Node>> {
let mut result: BTreeMap<String, Vec<&Node>> = BTreeMap::new();
for pad in n.children().iter().filter(|n| n.head() == Some("pad")) {
result
.entry(pad.value().unwrap_or("").to_owned())
.or_default()
.push(pad);
}
result
}
fn coordinates(n: &Node, field: &str) -> Option<(f64, f64)> {
let v = n.field(field)?.children();
let x = v.get(1)?.atom()?.parse::<f64>().ok()?;
let y = v.get(2)?.atom()?.parse::<f64>().ok()?;
(x.is_finite() && y.is_finite()).then_some((x, y))
}
pub fn similar(a: &Node, b: &Node, tolerance: f64) -> bool {
if let (Some(an), Some(bn)) = (a.value(), b.value())
&& !an.is_empty()
&& an == bn
{
return true;
}
let ap = pads(a);
let bp = pads(b);
if ap.is_empty() || ap.len() != bp.len() {
return false;
}
let tolerance = if tolerance.is_finite() {
tolerance.max(0.0)
} else {
0.0
};
for (number, aa) in ap {
let Some(bb) = bp.get(&number) else {
return false;
};
if aa.len() != bb.len() {
return false;
}
// Multiple pads with the same number are common. Match one-to-one,
// without depending on file ordering.
let mut used = vec![false; bb.len()];
for pad in aa {
let Some((ax, ay)) = coordinates(pad, "at") else {
return false;
};
let Some(index) = bb.iter().enumerate().position(|(i, other)| {
!used[i]
&& coordinates(other, "at").is_some_and(|(bx, by)| {
(ax - bx).abs() <= tolerance && (ay - by).abs() <= tolerance
})
}) else {
return false;
};
used[index] = true;
}
}
true
}
pub fn differences(a: &Node, b: &Node) -> Vec<String> {
let mut result = Vec::new();
let ap = pads(a);
let bp = pads(b);
let ac: usize = ap.values().map(Vec::len).sum();
let bc: usize = bp.values().map(Vec::len).sum();
if ac != bc {
result.push(format!("Pad count: incoming {ac}, candidate {bc}"));
}
let ak: Vec<_> = ap.keys().collect();
let bk: Vec<_> = bp.keys().collect();
if ak != bk {
result.push(format!("Pad numbers: incoming {ak:?}, candidate {bk:?}"));
}
for (number, aa) in &ap {
if let Some(bb) = bp.get(number) {
for (a, b) in aa.iter().zip(bb.iter()) {
for (index, label) in [(2, "type"), (3, "shape")] {
let av = a
.children()
.get(index)
.and_then(Node::atom)
.unwrap_or("missing");
let bv = b
.children()
.get(index)
.and_then(Node::atom)
.unwrap_or("missing");
if av != bv {
result.push(format!("Pad {number} {label}: {av} → {bv}"));
}
}
for field in [
"at",
"size",
"drill",
"layers",
"solder_mask_margin",
"solder_paste_margin",
"solder_paste_margin_ratio",
] {
let av = a.field(field).map(ToString::to_string);
let bv = b.field(field).map(ToString::to_string);
if av != bv {
result.push(format!(
"Pad {number} {field}: {} → {}",
av.as_deref().unwrap_or("default"),
bv.as_deref().unwrap_or("default")
));
}
}
}
}
}
for field in [
"attr",
"layer",
"solder_mask_margin",
"solder_paste_margin",
"solder_paste_ratio",
"clearance",
] {
if a.field(field) != b.field(field) {
result.push(format!("Footprint {field} differs"));
}
}
let am: Vec<_> = a
.children()
.iter()
.filter(|n| n.head() == Some("model"))
.collect();
let bm: Vec<_> = b
.children()
.iter()
.filter(|n| n.head() == Some("model"))
.collect();
if am != bm {
result.push("3D model references or transforms differ".into());
}
let graphics = |n: &Node| -> Vec<String> {
n.children()
.iter()
.filter(|c| {
c.head()
.is_some_and(|h| h.starts_with("fp_") || h == "property")
})
.map(ToString::to_string)
.collect()
};
if graphics(a) != graphics(b) {
result.push("Graphics, labels or courtyard differ".into());
}
if result.is_empty() && a != b {
result.push("Other fields or generated identifiers differ; inspect the files".into());
}
result
}
#[cfg(test)]
mod tests {
use super::*;
use crate::sexpr::parse;
#[test]
fn exact_ignores_only_name_whitespace_and_identifiers() {
let a = parse(
r#"(footprint "A" (uuid "a") (pad "1" smd rect (at 0 0) (size 1 1) (layers "F.Cu")))"#,
)
.unwrap();
let b = parse(
r#"(footprint "B" (uuid "b") (pad "1" smd rect (at 0 0) (size 1 1) (layers "F.Cu")))"#,
)
.unwrap();
assert_eq!(
exact_fingerprint(&a, Path::new("a")),
exact_fingerprint(&b, Path::new("b"))
);
let c = parse(r#"(footprint "A" (uuid "a") (pad "1" smd rect (at 0 0) (size 1 1) (layers "F.Cu")) (unknown_future 1))"#).unwrap();
assert_ne!(
exact_fingerprint(&a, Path::new("a")),
exact_fingerprint(&c, Path::new("a"))
);
let future_a = parse(r#"(footprint "A" (future (uuid "a")))"#).unwrap();
let future_b = parse(r#"(footprint "A" (future (uuid "b")))"#).unwrap();
assert_ne!(
exact_fingerprint(&future_a, Path::new("a")),
exact_fingerprint(&future_b, Path::new("a"))
);
}
#[test]
fn relative_models_are_contextual_and_content_based() {
let temp = tempfile::tempdir().unwrap();
let a = temp.path().join("a");
let b = temp.path().join("b");
fs::create_dir_all(&a).unwrap();
fs::create_dir_all(&b).unwrap();
let fp = parse(r#"(footprint "A" (model "part.step" (offset (xyz 0 0 0))))"#).unwrap();
let pa = a.join("a.kicad_mod");
let pb = b.join("b.kicad_mod");
assert_ne!(exact_fingerprint(&fp, &pa), exact_fingerprint(&fp, &pb));
fs::write(a.join("part.step"), "same model").unwrap();
fs::write(b.join("part.step"), "same model").unwrap();
assert_eq!(exact_fingerprint(&fp, &pa), exact_fingerprint(&fp, &pb));
fs::write(b.join("part.step"), "different model").unwrap();
assert_ne!(exact_fingerprint(&fp, &pa), exact_fingerprint(&fp, &pb));
}
#[test]
fn similar_is_only_a_candidate_and_keeps_pad_numbering() {
let a = parse(r#"(footprint "A" (pad "1" smd rect (at 0 0) (size 1 1)))"#).unwrap();
let b = parse(r#"(footprint "B" (pad "1" smd rect (at 0.01 0) (size 2 1)))"#).unwrap();
let c = parse(r#"(footprint "C" (pad "2" smd rect (at 0 0) (size 1 1)))"#).unwrap();
assert!(similar(&a, &b, 0.05));
assert!(!similar(&a, &c, 0.05));
assert!(differences(&a, &b).iter().any(|d| d.contains("size")));
assert_ne!(
exact_fingerprint(&a, Path::new("a")),
exact_fingerprint(&b, Path::new("b"))
);
}
}
+604
View File
@@ -0,0 +1,604 @@
use crate::{
destination::{Destination, resolve_variables},
sexpr::Node,
};
use anyhow::{Context, Result, bail};
use sha2::{Digest, Sha256};
use std::{
collections::{BTreeMap, BTreeSet},
path::{Path, PathBuf},
};
use walkdir::WalkDir;
#[derive(Debug, Default)]
pub struct ModelPlan {
pub copies: Vec<(PathBuf, Vec<u8>)>,
pub messages: Vec<String>,
}
fn supported(path: &Path) -> bool {
path.extension()
.and_then(|e| e.to_str())
.is_some_and(|e| matches!(e.to_ascii_lowercase().as_str(), "step" | "stp" | "wrl"))
}
fn digest(bytes: &[u8]) -> String {
format!("{:x}", Sha256::digest(bytes))
}
fn reference_parts(reference: &str) -> Vec<String> {
reference
.replace('\\', "/")
.split('/')
.filter(|s| !s.is_empty() && *s != ".")
.map(|s| s.to_ascii_lowercase())
.collect()
}
fn suffix_score(path: &Path, reference: &[String]) -> usize {
let parts = reference_parts(&path.to_string_lossy());
parts
.iter()
.rev()
.zip(reference.iter().rev())
.take_while(|(a, b)| a == b)
.count()
}
// KIPRJMOD belongs to the footprint's source project when copying a
// registered project library into another project. The target project is only
// the fallback for fresh bundle imports without discoverable source context.
fn source_project(source: &Path) -> Option<PathBuf> {
for directory in source.parent()?.ancestors() {
let Ok(entries) = std::fs::read_dir(directory) else {
continue;
};
if entries.filter_map(Result::ok).any(|entry| {
entry
.path()
.extension()
.is_some_and(|ext| ext == "kicad_pro")
&& entry.path().is_file()
}) {
return Some(directory.to_path_buf());
}
}
None
}
/// Resolve an original path first; otherwise rank recursive bundle/search-root
/// matches by trailing path components. Equal-content candidates collapse safely.
pub fn model_candidates(
reference: &str,
source_footprint: &Path,
source_root: &Path,
search_roots: &[PathBuf],
variables: &BTreeMap<String, String>,
project: Option<&Path>,
) -> Result<Vec<PathBuf>> {
let source_project = source_project(source_footprint);
let project = source_project.as_deref().or(project);
let normalized = reference.replace('\\', "/");
if let Ok(expanded) = resolve_variables(&normalized, project, variables) {
let path = PathBuf::from(&expanded);
let direct = if path.is_absolute() {
path
} else {
source_footprint.parent().unwrap_or(source_root).join(path)
};
if direct.is_file() && supported(&direct) {
return Ok(vec![direct]);
}
let bundled = source_root.join(expanded.trim_start_matches('/'));
if bundled.is_file() && supported(&bundled) {
return Ok(vec![bundled]);
}
}
let parts = reference_parts(&normalized);
let Some(filename) = parts.last() else {
return Ok(vec![]);
};
let mut roots = vec![source_root.to_path_buf()];
roots.extend_from_slice(search_roots);
let mut candidates = BTreeSet::new();
let mut best_score = 0;
for root in roots {
if !root.exists() {
continue;
}
if root.is_file() {
if supported(&root)
&& root
.file_name()
.is_some_and(|s| s.to_string_lossy().eq_ignore_ascii_case(filename))
{
let score = suffix_score(&root, &parts);
if score > best_score {
candidates.clear();
best_score = score;
}
if score == best_score {
candidates.insert(root.canonicalize()?);
}
}
continue;
}
for entry in WalkDir::new(&root).follow_links(false) {
let entry =
entry.with_context(|| format!("Search model files in {}", root.display()))?;
if !entry.file_type().is_file() || !supported(entry.path()) {
continue;
}
if !entry
.file_name()
.to_string_lossy()
.eq_ignore_ascii_case(filename)
{
continue;
}
let score = suffix_score(entry.path(), &parts);
if score > best_score {
candidates.clear();
best_score = score;
}
if score == best_score {
candidates.insert(entry.path().canonicalize()?);
}
}
}
let mut unique = BTreeMap::new();
for candidate in candidates {
let bytes = std::fs::read(&candidate)
.with_context(|| format!("Read model {}", candidate.display()))?;
unique.entry(digest(&bytes)).or_insert(candidate);
}
let mut paths: Vec<_> = unique.into_values().collect();
paths.sort();
Ok(paths)
}
#[cfg(test)]
pub fn repair_models(
footprint: &mut Node,
source_footprint: &Path,
source_root: &Path,
destination: &Destination,
search_roots: &[PathBuf],
variables: &BTreeMap<String, String>,
enabled: bool,
) -> Result<ModelPlan> {
repair_models_with_choices(
footprint,
source_footprint,
source_root,
destination,
search_roots,
variables,
enabled,
&BTreeMap::new(),
)
}
#[allow(clippy::too_many_arguments)]
pub fn repair_models_with_choices(
footprint: &mut Node,
source_footprint: &Path,
source_root: &Path,
destination: &Destination,
search_roots: &[PathBuf],
variables: &BTreeMap<String, String>,
enabled: bool,
choices: &BTreeMap<String, PathBuf>,
) -> Result<ModelPlan> {
let mut plan = ModelPlan::default();
repair_node(
footprint,
source_footprint,
source_root,
destination,
search_roots,
variables,
enabled,
choices,
&mut plan,
)?;
Ok(plan)
}
#[allow(clippy::too_many_arguments)]
fn repair_node(
node: &mut Node,
source_footprint: &Path,
source_root: &Path,
destination: &Destination,
search_roots: &[PathBuf],
variables: &BTreeMap<String, String>,
enabled: bool,
choices: &BTreeMap<String, PathBuf>,
plan: &mut ModelPlan,
) -> Result<()> {
let Node::List(items) = node else {
return Ok(());
};
if matches!(items.first(), Some(Node::Atom(head)) if head == "model") {
let reference = match items.get(1) {
Some(Node::Str(s) | Node::Atom(s)) => s.clone(),
_ => bail!(
"Model reference missing a filename in {}",
source_footprint.display()
),
};
if reference.starts_with("embedded://")
|| reference.starts_with("kicad-embed://")
|| reference.starts_with("${KICAD_EMBEDDED}")
{
plan.messages
.push(format!("Preserved embedded model: {reference}"));
return Ok(());
}
if !enabled {
plan.messages.push(format!("Model import disabled; external dependency remains unresolved for portability: {reference}"));
return Ok(());
}
let candidates = model_candidates(
&reference,
source_footprint,
source_root,
search_roots,
variables,
destination.project.as_deref(),
)?;
let source = if let Some(choice) = choices.get(&reference) {
let canonical = choice
.canonicalize()
.with_context(|| format!("Chosen model is unavailable: {}", choice.display()))?;
if !candidates
.iter()
.any(|candidate| candidate.canonicalize().ok().as_ref() == Some(&canonical))
{
bail!(
"pending: chosen model is no longer a matching candidate: {}",
choice.display()
);
}
canonical
} else {
match candidates.as_slice() {
[] => {
plan.messages.push(format!("Missing model; dependency unresolved, original reference preserved: {reference}"));
return Ok(());
}
[one] => one.clone(),
many => bail!(
"pending: ambiguous model {reference}; choose one of: {}",
many.iter()
.map(|p| p.display().to_string())
.collect::<Vec<_>>()
.join(" | ")
),
}
};
let bytes =
std::fs::read(&source).with_context(|| format!("Read model {}", source.display()))?;
// A footprint already in this destination must not acquire another hash
// suffix each time its models are reviewed or the component is reimported.
if source.parent().and_then(|p| p.canonicalize().ok())
== destination.models_dir.canonicalize().ok()
&& destination.models_dir.is_dir()
{
items[1] = Node::Str(destination.model_uri(&source));
plan.messages
.push(format!("Reused model: {}", source.display()));
return Ok(());
}
let hash = digest(&bytes);
let stem = source.file_stem().unwrap_or_default().to_string_lossy();
let safe_stem: String = stem
.chars()
.map(|c| {
if c.is_alphanumeric() || c == '-' || c == '_' {
c
} else {
'_'
}
})
.collect();
let extension = source
.extension()
.unwrap_or_default()
.to_string_lossy()
.to_ascii_lowercase();
let mut target =
destination
.models_dir
.join(format!("{safe_stem}_{}.{}", &hash[..12], extension));
if target.exists()
&& std::fs::read(&target)
.with_context(|| format!("Read existing model {}", target.display()))?
!= bytes
{
target = destination
.models_dir
.join(format!("{safe_stem}_{hash}.{extension}"));
if target.exists() && std::fs::read(&target)? != bytes {
bail!("Model storage collision at {}", target.display());
}
}
let uri = destination.model_uri(&target);
let reused = target.is_file();
if !reused && !plan.copies.iter().any(|(path, _)| path == &target) {
plan.copies.push((target.clone(), bytes));
}
items[1] = Node::Str(uri.clone());
plan.messages.push(format!(
"{} model: {} -> {}{}",
if reused { "Reused" } else { "Copied" },
source.display(),
uri,
if reference != source.to_string_lossy() {
" (reference repaired)"
} else {
""
}
));
return Ok(());
}
for child in items {
repair_node(
child,
source_footprint,
source_root,
destination,
search_roots,
variables,
enabled,
choices,
plan,
)?;
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::sexpr::parse;
fn scratch() -> PathBuf {
let path = std::env::temp_dir().join(format!(
"uniloader-models-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
std::fs::create_dir_all(&path).unwrap();
path
}
#[test]
fn repairs_windows_paths_preserving_transforms() {
let root = scratch();
std::fs::create_dir_all(root.join("bundle/3d")).unwrap();
std::fs::write(root.join("bundle/3d/part.STEP"), b"model").unwrap();
std::fs::write(root.join("board.kicad_pro"), b"{}").unwrap();
let dest = Destination::new(
&root.join("global"),
Some(&root.join("board.kicad_pro")),
&root.join("config"),
)
.unwrap();
let mut fp = parse(r#"(footprint "x" (model "C:\\old\\3d\\part.STEP" (offset (xyz 1 2 3)) (scale (xyz 1 1 1)) (rotate (xyz 0 0 90))))"#).unwrap();
let plan = repair_models(
&mut fp,
&root.join("bundle/x.kicad_mod"),
&root.join("bundle"),
&dest,
&[],
&BTreeMap::new(),
true,
)
.unwrap();
assert_eq!(plan.copies.len(), 1);
let rendered = fp.to_string();
assert!(rendered.contains("${KIPRJMOD}/libraries/models/part_"));
assert!(rendered.contains("(xyz 0 0 90)"));
assert!(rendered.contains("(xyz 1 2 3)"));
std::fs::remove_dir_all(root).unwrap();
}
#[test]
fn collapses_identical_candidates_but_requires_choice_for_different_content() {
let root = scratch();
for dir in ["a", "b"] {
std::fs::create_dir(root.join(dir)).unwrap();
std::fs::write(root.join(dir).join("part.step"), b"same").unwrap();
}
let candidates = model_candidates(
"C:/obsolete/part.step",
&root.join("x.kicad_mod"),
&root,
&[],
&BTreeMap::new(),
None,
)
.unwrap();
assert_eq!(candidates.len(), 1);
std::fs::write(root.join("b/part.step"), b"different").unwrap();
assert_eq!(
model_candidates(
"C:/obsolete/part.step",
&root.join("x.kicad_mod"),
&root,
&[],
&BTreeMap::new(),
None
)
.unwrap()
.len(),
2
);
let dest = Destination::new(&root.join("global"), None, &root.join("config")).unwrap();
let mut fp = parse(r#"(footprint "x" (model "C:/obsolete/part.step"))"#).unwrap();
assert!(
repair_models(
&mut fp,
&root.join("x.kicad_mod"),
&root,
&dest,
&[],
&BTreeMap::new(),
true
)
.unwrap_err()
.to_string()
.starts_with("pending:")
);
let choices = BTreeMap::from([("C:/obsolete/part.step".into(), root.join("b/part.step"))]);
let plan = repair_models_with_choices(
&mut fp,
&root.join("x.kicad_mod"),
&root,
&dest,
&[],
&BTreeMap::new(),
true,
&choices,
)
.unwrap();
assert_eq!(plan.copies[0].1, b"different");
std::fs::remove_dir_all(root).unwrap();
}
#[test]
fn external_project_model_uses_source_context_and_becomes_portable() {
let root = scratch();
let source = root.join("source");
let target = root.join("target");
for project in [&source, &target] {
std::fs::create_dir_all(project.join("libraries/models")).unwrap();
std::fs::create_dir_all(project.join("libraries/Parts.pretty")).unwrap();
std::fs::write(project.join("board.kicad_pro"), b"{}").unwrap();
}
std::fs::write(source.join("libraries/models/chip.step"), b"SOURCE MODEL").unwrap();
// Same URI in another project is not the same dependency.
std::fs::write(
target.join("libraries/models/chip.step"),
b"WRONG TARGET MODEL",
)
.unwrap();
let dest = Destination::new(
&root.join("global"),
Some(&target.join("board.kicad_pro")),
&root.join("config"),
)
.unwrap();
let mut fp = parse(r#"(footprint "chip" (model "${KIPRJMOD}/libraries/models/chip.step" (offset (xyz 1 2 3))))"#).unwrap();
let source_fp = source.join("libraries/Parts.pretty/chip.kicad_mod");
let plan = repair_models(
&mut fp,
&source_fp,
source_fp.parent().unwrap(),
&dest,
&[],
&BTreeMap::new(),
true,
)
.unwrap();
assert_eq!(plan.copies.len(), 1);
assert_eq!(plan.copies[0].1, b"SOURCE MODEL");
assert!(plan.copies[0].0.starts_with(&target));
assert!(
fp.to_string()
.contains("${KIPRJMOD}/libraries/models/chip_")
);
assert!(fp.to_string().contains("(xyz 1 2 3)"));
std::fs::remove_dir_all(root).unwrap();
}
#[test]
fn conflicting_model_storage_keeps_existing_bytes() {
let root = scratch();
std::fs::write(root.join("chip.step"), b"NEW MODEL").unwrap();
let dest = Destination::new(&root.join("global"), None, &root.join("config")).unwrap();
std::fs::create_dir_all(&dest.models_dir).unwrap();
let hash = digest(b"NEW MODEL");
let collision = dest.models_dir.join(format!("chip_{}.step", &hash[..12]));
std::fs::write(&collision, b"EXISTING USER MODEL").unwrap();
let mut fp = parse(r#"(footprint "chip" (model "chip.step"))"#).unwrap();
let plan = repair_models(
&mut fp,
&root.join("chip.kicad_mod"),
&root,
&dest,
&[],
&BTreeMap::new(),
true,
)
.unwrap();
assert_ne!(plan.copies[0].0, collision);
assert_eq!(std::fs::read(&collision).unwrap(), b"EXISTING USER MODEL");
assert_eq!(plan.copies[0].1, b"NEW MODEL");
// A second collision refuses to plan an overwrite.
std::fs::write(&plan.copies[0].0, b"ANOTHER EXISTING MODEL").unwrap();
let mut original = parse(r#"(footprint "chip" (model "chip.step"))"#).unwrap();
assert!(
repair_models(
&mut original,
&root.join("chip.kicad_mod"),
&root,
&dest,
&[],
&BTreeMap::new(),
true
)
.is_err()
);
std::fs::remove_dir_all(root).unwrap();
}
#[test]
fn embedded_references_are_preserved_without_warnings() {
let root = scratch();
let dest = Destination::new(&root.join("global"), None, &root.join("config")).unwrap();
for reference in [
"embedded://chip.step",
"kicad-embed://chip.step",
"${KICAD_EMBEDDED}/chip.step",
] {
let mut fp = Node::list(vec![
Node::symbol("footprint"),
Node::string("chip"),
Node::list(vec![Node::symbol("model"), Node::string(reference)]),
]);
let before = fp.to_string();
let plan = repair_models(
&mut fp,
&root.join("chip.kicad_mod"),
&root,
&dest,
&[],
&BTreeMap::new(),
true,
)
.unwrap();
assert_eq!(fp.to_string(), before);
assert!(plan.copies.is_empty());
assert!(plan.messages[0].contains("Preserved embedded"));
}
std::fs::remove_dir_all(root).unwrap();
}
#[test]
fn missing_models_remain_visible() {
let root = scratch();
let dest = Destination::new(&root.join("global"), None, &root.join("config")).unwrap();
let mut fp = parse(r#"(footprint "x" (model "missing.wrl"))"#).unwrap();
let before = fp.to_string();
let plan = repair_models(
&mut fp,
&root.join("x.kicad_mod"),
&root,
&dest,
&[],
&BTreeMap::new(),
true,
)
.unwrap();
assert_eq!(fp.to_string(), before);
assert!(plan.copies.is_empty());
assert!(plan.messages[0].contains("Missing model"));
std::fs::remove_dir_all(root).unwrap();
}
}
+418
View File
@@ -0,0 +1,418 @@
//! A lightweight footprint preview. This is a comparison aid, not a KiCad renderer.
use crate::sexpr::{self, Node};
use eframe::egui::{self, Color32, Pos2, Stroke, Vec2};
#[derive(Default)]
pub struct Preview {
paths: Vec<Path>,
labels: Vec<([f32; 2], String)>,
bounds: Option<[f32; 4]>,
pub warnings: Vec<String>,
}
struct Path {
points: Vec<[f32; 2]>,
closed: bool,
fill: bool,
color: Color32,
}
const PAD: Color32 = Color32::from_rgb(208, 164, 78);
const OUTLINE: Color32 = Color32::from_rgb(120, 194, 224);
const HOLE: Color32 = Color32::from_rgb(30, 33, 38);
fn number(n: &Node, index: usize) -> Option<f32> {
n.children()
.get(index)?
.atom()?
.parse::<f32>()
.ok()
.filter(|v| v.is_finite() && v.abs() <= 100_000.0)
}
fn xy(n: &Node) -> Option<[f32; 2]> {
Some([number(n, 1)?, number(n, 2)?])
}
fn point(n: &Node, field: &str) -> Option<[f32; 2]> {
xy(n.field(field)?)
}
fn rotate(p: [f32; 2], center: [f32; 2], angle: f32) -> [f32; 2] {
let (s, c) = angle.to_radians().sin_cos();
// KiCad pad rotation is counter-clockwise in its downward-positive Y coordinates.
[
center[0] + p[0] * c + p[1] * s,
center[1] - p[0] * s + p[1] * c,
]
}
fn ellipse(center: [f32; 2], size: [f32; 2], angle: f32) -> Vec<[f32; 2]> {
(0..64)
.map(|i| {
let a = i as f32 * std::f32::consts::TAU / 64.0;
rotate(
[a.cos() * size[0] / 2.0, a.sin() * size[1] / 2.0],
center,
angle,
)
})
.collect()
}
fn rounded(center: [f32; 2], size: [f32; 2], radius: f32, angle: f32) -> Vec<[f32; 2]> {
let mut points = Vec::new();
for (cx, cy, start) in [
(1.0, 1.0, 0.0),
(-1.0, 1.0, 90.0),
(-1.0, -1.0, 180.0),
(1.0, -1.0, 270.0),
] {
for i in 0..=12 {
let a = (start + i as f32 * 7.5).to_radians();
points.push(rotate(
[
cx * (size[0] / 2.0 - radius) + radius * a.cos(),
cy * (size[1] / 2.0 - radius) + radius * a.sin(),
],
center,
angle,
));
}
}
points
}
impl Preview {
pub fn from_raw(raw: &str) -> Self {
let mut preview = Self::default();
let root = match sexpr::parse(raw) {
Ok(n) => n,
Err(e) => {
preview
.warnings
.push(format!("Cannot parse footprint: {e}"));
return preview;
}
};
if !matches!(root.head(), Some("footprint" | "module")) {
preview
.warnings
.push("Expected a footprint or module".into());
return preview;
}
for n in root.children() {
let valid = match n.head() {
Some("pad") => preview.pad(n),
Some("fp_line") => preview.line(n),
Some("fp_rect") => preview.rect(n),
Some("fp_circle") => preview.circle(n),
Some("fp_poly") => preview.poly(n),
Some("fp_arc") => preview.arc(n),
Some("fp_curve") => {
preview
.warnings
.push("Bezier curve omitted from preview".into());
true
}
_ => true,
};
if !valid {
preview.warnings.push(format!(
"Invalid {} omitted from preview",
n.head().unwrap_or("geometry")
));
}
}
if preview.bounds.is_none() {
preview
.warnings
.push("No drawable footprint geometry".into());
}
preview
}
fn add(&mut self, points: Vec<[f32; 2]>, closed: bool, fill: bool, color: Color32) {
if points.is_empty()
|| points
.iter()
.flatten()
.any(|v| !v.is_finite() || v.abs() > 200_000.0)
{
return;
}
for p in &points {
let b = self.bounds.get_or_insert([p[0], p[1], p[0], p[1]]);
b[0] = b[0].min(p[0]);
b[1] = b[1].min(p[1]);
b[2] = b[2].max(p[0]);
b[3] = b[3].max(p[1]);
}
self.paths.push(Path {
points,
closed,
fill,
color,
});
}
fn pad(&mut self, n: &Node) -> bool {
let Some(at) = n.field("at") else {
return false;
};
let Some(center) = xy(at) else { return false };
let angle = if at.children().len() > 3 {
let Some(a) = number(at, 3) else { return false };
a
} else {
0.0
};
let Some(size) = point(n, "size").filter(|s| s[0] > 0.0 && s[1] > 0.0) else {
return false;
};
let shape = n.children().get(3).and_then(Node::atom).unwrap_or("");
let points = match shape {
"circle" => ellipse(center, size, angle),
"rect" => rounded(center, size, 0.0, angle),
"oval" => rounded(center, size, size[0].min(size[1]) / 2.0, angle),
"roundrect" => {
let ratio = n
.field("roundrect_rratio")
.and_then(|r| number(r, 1))
.unwrap_or(0.25);
if !(0.0..=0.5).contains(&ratio) {
return false;
}
rounded(center, size, size[0].min(size[1]) * ratio, angle)
}
_ => {
self.warnings.push(format!(
"Unsupported pad shape '{shape}'; bounding rectangle shown"
));
rounded(center, size, 0.0, angle)
}
};
self.add(points, true, true, PAD);
if let Some(drill) = n.field("drill") {
let oval = drill.children().get(1).and_then(Node::atom) == Some("oval");
let d = number(drill, if oval { 2 } else { 1 });
if let Some(d) = d.filter(|d| *d > 0.0) {
let dy = if oval {
number(drill, 3).unwrap_or(d)
} else {
d
};
if dy > 0.0 {
let offset = drill.field("offset").and_then(xy).unwrap_or([0.0, 0.0]);
self.add(
rounded(
rotate(offset, center, angle),
[d, dy],
d.min(dy) / 2.0,
angle,
),
true,
true,
HOLE,
);
} else {
self.warnings.push("Invalid drill omitted".into());
}
} else {
self.warnings.push("Invalid drill omitted".into());
}
}
self.labels
.push((center, n.value().unwrap_or("").to_owned()));
true
}
fn line(&mut self, n: &Node) -> bool {
let (Some(a), Some(b)) = (point(n, "start"), point(n, "end")) else {
return false;
};
self.add(vec![a, b], false, false, OUTLINE);
true
}
fn rect(&mut self, n: &Node) -> bool {
let (Some(a), Some(b)) = (point(n, "start"), point(n, "end")) else {
return false;
};
self.add(vec![a, [b[0], a[1]], b, [a[0], b[1]]], true, false, OUTLINE);
true
}
fn circle(&mut self, n: &Node) -> bool {
let (Some(c), Some(e)) = (point(n, "center"), point(n, "end")) else {
return false;
};
let d = 2.0 * (e[0] - c[0]).hypot(e[1] - c[1]);
self.add(ellipse(c, [d, d], 0.0), true, false, OUTLINE);
true
}
fn poly(&mut self, n: &Node) -> bool {
let Some(pts) = n.field("pts") else {
return false;
};
let Some(points) = pts
.children()
.iter()
.skip(1)
.map(xy)
.collect::<Option<Vec<_>>>()
else {
return false;
};
if points.len() < 2 {
return false;
}
self.add(points, true, false, OUTLINE);
true
}
fn arc(&mut self, n: &Node) -> bool {
let (Some(a), Some(m), Some(b)) = (point(n, "start"), point(n, "mid"), point(n, "end"))
else {
self.warnings.push("Legacy arc omitted from preview".into());
return true;
};
let determinant =
2.0 * (a[0] * (m[1] - b[1]) + m[0] * (b[1] - a[1]) + b[0] * (a[1] - m[1]));
if determinant.abs() < 1e-6 {
return false;
}
let q = |p: [f32; 2]| p[0] * p[0] + p[1] * p[1];
let center = [
(q(a) * (m[1] - b[1]) + q(m) * (b[1] - a[1]) + q(b) * (a[1] - m[1])) / determinant,
(q(a) * (b[0] - m[0]) + q(m) * (a[0] - b[0]) + q(b) * (m[0] - a[0])) / determinant,
];
let angle = |p: [f32; 2]| (p[1] - center[1]).atan2(p[0] - center[0]);
let start = angle(a);
let end = (angle(b) - start).rem_euclid(std::f32::consts::TAU);
let middle = (angle(m) - start).rem_euclid(std::f32::consts::TAU);
let sweep = if middle <= end {
end
} else {
end - std::f32::consts::TAU
};
let radius = (a[0] - center[0]).hypot(a[1] - center[1]);
self.add(
(0..=64)
.map(|i| {
let t = start + sweep * i as f32 / 64.0;
[center[0] + radius * t.cos(), center[1] + radius * t.sin()]
})
.collect(),
false,
false,
OUTLINE,
);
true
}
pub fn span(&self) -> [f32; 2] {
let b = self.bounds.unwrap_or([-0.5, -0.5, 0.5, 0.5]);
[(b[2] - b[0]).max(0.001), (b[3] - b[1]).max(0.001)]
}
pub fn paint(&self, ui: &mut egui::Ui, scale: f32) -> egui::Response {
let (rect, response) = ui.allocate_exact_size(
Vec2::new(ui.available_width().max(1.0), 300.0),
egui::Sense::hover(),
);
let painter = ui.painter_at(rect);
painter.rect_filled(rect, 4.0, HOLE);
let b = self.bounds.unwrap_or([-0.5, -0.5, 0.5, 0.5]);
let scale = if scale.is_finite() && scale > 0.0 {
scale.min(100_000.0)
} else {
1.0
};
let transform = |p: [f32; 2]| {
Pos2::new(
rect.center().x + (p[0] - (b[0] + b[2]) / 2.0) * scale,
rect.center().y + (p[1] - (b[1] + b[3]) / 2.0) * scale,
)
};
for path in &self.paths {
let mut points: Vec<_> = path.points.iter().copied().map(transform).collect();
if path.fill {
painter.add(egui::Shape::convex_polygon(
points,
path.color,
Stroke::NONE,
));
} else {
if path.closed {
points.push(points[0]);
}
painter.add(egui::Shape::line(points, Stroke::new(1.5, path.color)));
}
}
for (p, label) in &self.labels {
painter.text(
transform(*p),
egui::Align2::CENTER_CENTER,
label,
egui::FontId::proportional(12.0),
Color32::WHITE,
);
}
let span = self.span();
painter.text(
rect.left_bottom() + Vec2::new(8.0, -8.0),
egui::Align2::LEFT_BOTTOM,
format!("{:.2} × {:.2} mm", span[0], span[1]),
egui::FontId::proportional(12.0),
Color32::WHITE,
);
response
}
}
pub fn fit_scale(left: &Preview, right: &Preview, width: f32, height: f32) -> f32 {
let a = left.span();
let b = right.span();
((width - 32.0).max(1.0) / a[0].max(b[0]))
.min((height - 40.0).max(1.0) / a[1].max(b[1]))
.clamp(0.0001, 100_000.0)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn rotated_asymmetric_pad_and_unicode() {
let p = Preview::from_raw(
r#"(footprint "µ (quoted)" (pad "Ω" smd rect (at 2 3 90) (size 4 2)))"#,
);
assert!(p.warnings.is_empty());
assert_eq!(p.labels[0].1, "Ω");
let span = p.span();
assert!((span[0] - 2.0).abs() < 0.001);
assert!((span[1] - 4.0).abs() < 0.001);
}
#[test]
fn malformed_and_nonfinite_geometry_is_safe() {
for input in [
"(footprint",
"(footprint x (pad 1 smd rect (at NaN 0) (size 2 2)))",
"(footprint x (pad 1 smd rect (at 0 0) (size -1 2)))",
"(footprint x (fp_line (start inf 0) (end 1 1)))",
] {
let p = Preview::from_raw(input);
assert!(!p.warnings.is_empty());
assert!(p.span().iter().all(|n| n.is_finite() && *n > 0.0));
}
}
#[test]
fn unknown_shapes_and_drill_offsets_are_visible() {
let p = Preview::from_raw(
"(footprint x (pad 1 thru_hole custom (at 0 0) (size 2 2) (drill 1 (offset 0.2 0))))",
);
assert!(p.warnings.iter().any(|w| w.contains("custom")));
assert_eq!(p.paths.len(), 2);
}
#[test]
fn outlines_and_arcs_have_expected_bounds() {
let p = Preview::from_raw(
"(footprint x (fp_rect (start -2 -3) (end 2 3)) (fp_circle (center 0 0) (end 1 0)) (fp_arc (start 1 0) (mid 0 1) (end -1 0)) (fp_poly (pts (xy 0 0) (xy 1 0) (xy 0 1))))",
);
assert!(p.warnings.is_empty());
assert_eq!(p.span(), [4.0, 6.0]);
assert_eq!(p.paths.len(), 4);
}
#[test]
fn renders_without_a_display() {
let p = Preview::from_raw("(footprint x (pad 1 smd oval (at 0 0 45) (size 2 4)))");
let ctx = egui::Context::default();
let mut output = ctx.run_ui(egui::RawInput::default(), |ui| {
p.paint(ui, fit_scale(&p, &p, 300.0, 300.0));
});
assert!(!output.shapes.is_empty());
output.textures_delta.clear();
}
}
+198
View File
@@ -0,0 +1,198 @@
//! Small, lossless-in-content KiCad S-expression reader.
use anyhow::{Result, bail};
use std::fmt;
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum Node {
Atom(String),
Str(String),
List(Vec<Node>),
}
impl Node {
pub fn head(&self) -> Option<&str> {
self.children().first().and_then(Node::atom)
}
pub fn atom(&self) -> Option<&str> {
match self {
Self::Atom(s) | Self::Str(s) => Some(s),
_ => None,
}
}
pub fn children(&self) -> &[Node] {
match self {
Self::List(v) => v,
_ => &[],
}
}
pub fn children_mut(&mut self) -> &mut Vec<Node> {
match self {
Self::List(v) => v,
_ => panic!("expected S-expression list"),
}
}
pub fn field(&self, name: &str) -> Option<&Node> {
self.children().iter().find(|n| n.head() == Some(name))
}
pub fn field_mut(&mut self, name: &str) -> Option<&mut Node> {
self.children_mut()
.iter_mut()
.find(|n| n.head() == Some(name))
}
pub fn value(&self) -> Option<&str> {
self.children().get(1).and_then(Node::atom)
}
pub fn string(s: impl Into<String>) -> Self {
Self::Str(s.into())
}
pub fn symbol(s: impl Into<String>) -> Self {
Self::Atom(s.into())
}
pub fn list(nodes: Vec<Node>) -> Self {
Self::List(nodes)
}
}
impl fmt::Display for Node {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Atom(s) => f.write_str(s),
Self::Str(s) => {
f.write_str("\"")?;
for c in s.chars() {
match c {
'"' => f.write_str("\\\"")?,
'\\' => f.write_str("\\\\")?,
'\n' => f.write_str("\\n")?,
'\r' => f.write_str("\\r")?,
'\t' => f.write_str("\\t")?,
_ => write!(f, "{c}")?,
}
}
f.write_str("\"")
}
Self::List(nodes) => {
f.write_str("(")?;
for (i, n) in nodes.iter().enumerate() {
if i > 0 {
f.write_str(" ")?;
}
write!(f, "{n}")?;
}
f.write_str(")")
}
}
}
}
pub fn parse(input: &str) -> Result<Node> {
struct Reader<'a> {
chars: std::iter::Peekable<std::str::Chars<'a>>,
depth: usize,
}
impl Reader<'_> {
fn spaces(&mut self) {
while self
.chars
.peek()
.is_some_and(|c| c.is_whitespace() || *c == '\u{feff}')
{
self.chars.next();
}
}
fn read(&mut self) -> Result<Node> {
self.spaces();
match self.chars.next() {
Some('(') => {
self.depth += 1;
if self.depth > 256 {
bail!("S-expression nesting exceeds 256 levels");
}
let mut v = Vec::new();
loop {
self.spaces();
match self.chars.peek() {
Some(')') => {
self.chars.next();
self.depth -= 1;
return Ok(Node::List(v));
}
None => bail!("unclosed S-expression list"),
_ => v.push(self.read()?),
}
}
}
Some(')') => bail!("unexpected closing parenthesis"),
Some('"') => {
let mut s = String::new();
loop {
match self.chars.next() {
Some('"') => return Ok(Node::Str(s)),
Some('\\') => match self.chars.next() {
Some('n') => s.push('\n'),
Some('r') => s.push('\r'),
Some('t') => s.push('\t'),
Some('"') => s.push('"'),
Some('\\') => s.push('\\'),
Some(c) => {
s.push('\\');
s.push(c);
}
None => bail!("unterminated string escape"),
},
Some(c) => s.push(c),
None => bail!("unterminated quoted string"),
}
}
}
Some(c) => {
let mut s = String::from(c);
while self
.chars
.peek()
.is_some_and(|c| !c.is_whitespace() && !matches!(c, '(' | ')' | '"'))
{
s.push(self.chars.next().unwrap());
}
Ok(Node::Atom(s))
}
None => bail!("empty S-expression input"),
}
}
}
let mut reader = Reader {
chars: input.chars().peekable(),
depth: 0,
};
let result = reader.read()?;
reader.spaces();
if reader.chars.peek().is_some() {
bail!("unexpected trailing content after S-expression");
}
Ok(result)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn unicode_quoted_parentheses_and_escapes_round_trip() {
let input = r#"(kicad_symbol_lib (version 20261101) (symbol "µ Ω (test)" (property "Description" "a \"quoted\" (thing) \\ end") (unknown_future (xyz "é"))))"#;
let ast = parse(input).unwrap();
assert_eq!(ast, parse(&ast.to_string()).unwrap());
assert_eq!(ast.field("symbol").unwrap().value(), Some("µ Ω (test)"));
assert_eq!(ast.field("version").unwrap().value(), Some("20261101"));
}
#[test]
fn rejects_incomplete_and_trailing_input() {
for bad in ["", "(a", "(a))", "(a) (b)", "(a \"unterminated)", ")"] {
assert!(parse(bad).is_err(), "{bad}");
}
}
#[test]
fn preserves_unknown_escape_and_fields() {
let ast = parse(r#"(future "C:\models\part.step" (unknown 42))"#).unwrap();
assert_eq!(ast, parse(&ast.to_string()).unwrap());
assert!(ast.field("unknown").is_some());
}
}
+120
View File
@@ -0,0 +1,120 @@
use crate::sexpr::{self, Node};
use anyhow::{Context, Result, bail};
use std::{
fs,
io::Read,
path::{Path, PathBuf},
};
use walkdir::WalkDir;
pub struct Bundle {
pub root: PathBuf,
pub symbols: Vec<(PathBuf, Node)>,
pub footprints: Vec<(PathBuf, Node)>,
pub provider: String,
_temp: Option<tempfile::TempDir>,
}
pub fn load(path: &Path) -> Result<Bundle> {
let mut temp = None;
let root = if path.is_dir() {
fs::canonicalize(path)?
} else {
let mut archive =
zip::ZipArchive::new(fs::File::open(path)?).context("not a complete readable ZIP")?;
fs::create_dir_all(crate::config::cache_dir())?;
let dir = tempfile::Builder::new()
.prefix("extract-")
.tempdir_in(crate::config::cache_dir())?;
let mut total = 0u64;
for i in 0..archive.len() {
let mut entry = archive.by_index(i)?;
let enclosed = entry.enclosed_name().context("unsafe archive path")?;
if entry.unix_mode().is_some_and(|m| m & 0o170000 == 0o120000) {
bail!("archive contains symbolic link");
}
total = total
.checked_add(entry.size())
.context("archive too large")?;
if total > 2 * 1024 * 1024 * 1024 {
bail!("archive exceeds 2 GiB expanded limit");
}
let out = dir.path().join(enclosed);
if entry.is_dir() {
fs::create_dir_all(out)?;
} else {
fs::create_dir_all(out.parent().unwrap())?;
let mut f = fs::File::create(out)?;
std::io::copy(&mut entry, &mut f)?;
}
}
let root = dir.path().to_owned();
temp = Some(dir);
root
};
let name = path
.file_name()
.unwrap_or_default()
.to_string_lossy()
.to_lowercase();
let mut provider = if name.starts_with("lib_") {
"SamacSys"
} else if name.starts_with("ul_") {
"UltraLibrarian"
} else {
"Generic"
}
.to_owned();
let mut symbols = vec![];
let mut footprints = vec![];
for entry in WalkDir::new(&root).follow_links(false).sort_by_file_name() {
let entry = entry?;
if !entry.file_type().is_file() {
continue;
}
let p = entry.path();
let ext = p
.extension()
.unwrap_or_default()
.to_string_lossy()
.to_lowercase();
if ext == "kicad_sym" || ext == "kicad_mod" {
let raw = fs::read_to_string(p).with_context(|| format!("reading {}", p.display()))?;
let node = sexpr::parse(&raw).with_context(|| format!("parsing {}", p.display()))?;
if ext == "kicad_sym" {
if node.head() != Some("kicad_symbol_lib") {
bail!("{} is not a symbol library", p.display());
}
symbols.push((p.to_owned(), node));
} else {
if !matches!(node.head(), Some("footprint" | "module")) {
bail!("{} is not a footprint", p.display());
}
footprints.push((p.to_owned(), node));
}
} else if entry
.file_name()
.to_string_lossy()
.to_lowercase()
.contains("how-to-import")
{
let mut s = String::new();
fs::File::open(p)?
.take(1024 * 1024)
.read_to_string(&mut s)
.ok();
if s.to_lowercase().contains("snapeda") {
provider = "SnapMagic".into();
}
}
}
if symbols.is_empty() && footprints.is_empty() {
bail!("no KiCad symbols or footprints found");
}
Ok(Bundle {
root,
symbols,
footprints,
provider,
_temp: temp,
})
}
+177
View File
@@ -0,0 +1,177 @@
use crate::config;
use anyhow::{Context, Result, bail};
use fs2::FileExt;
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
use std::{
collections::BTreeMap,
fs::{self, File, OpenOptions},
io::Write,
path::{Path, PathBuf},
};
pub fn hash(bytes: &[u8]) -> String {
format!("{:x}", Sha256::digest(bytes))
}
pub fn source_hash(path: &Path) -> Result<String> {
if path.is_file() {
return Ok(hash(&fs::read(path)?));
}
let mut h = Sha256::new();
for e in walkdir::WalkDir::new(path)
.follow_links(false)
.sort_by_file_name()
{
let e = e?;
if e.file_type().is_file() {
h.update(e.path().strip_prefix(path)?.to_string_lossy().as_bytes());
h.update([0]);
h.update(fs::read(e.path())?);
}
}
Ok(format!("{:x}", h.finalize()))
}
pub struct Lock {
_file: File,
}
impl Lock {
pub fn acquire() -> Result<Self> {
Self::named("writer.lock")
}
pub fn watcher() -> Result<Self> {
Self::named("watcher.lock")
}
fn named(name: &str) -> Result<Self> {
fs::create_dir_all(config::data_dir())?;
let f = OpenOptions::new()
.create(true)
.truncate(false)
.read(true)
.write(true)
.open(config::data_dir().join(name))?;
f.try_lock_exclusive().with_context(|| {
if name == "writer.lock" {
"another UniLoader import is active; retry shortly"
} else {
"another UniLoader watcher is already running"
}
})?;
Ok(Self { _file: f })
}
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct Record {
pub id: String,
pub source: PathBuf,
pub hash: String,
pub status: String,
pub destination: Option<String>,
pub project: Option<PathBuf>,
pub message: String,
#[serde(default)]
pub changed: Vec<PathBuf>,
#[serde(default)]
pub backups: Vec<PathBuf>,
}
impl Record {
pub fn new(source: PathBuf, hash: String) -> Self {
let id = hash[..16].to_owned();
Self {
id,
source,
hash,
status: "pending".into(),
destination: None,
project: None,
message: String::new(),
changed: vec![],
backups: vec![],
}
}
pub fn save(&self) -> Result<()> {
let dir = config::data_dir().join("imports");
fs::create_dir_all(&dir)?;
atomic(
&dir.join(format!("{}.json", self.id)),
&serde_json::to_vec_pretty(self)?,
)
}
}
pub fn records() -> Result<Vec<Record>> {
let d = config::data_dir().join("imports");
if !d.exists() {
return Ok(vec![]);
}
let mut out: Vec<Record> = vec![];
for e in fs::read_dir(d)? {
let p = e?.path();
if p.extension().is_some_and(|e| e == "json") {
out.push(
serde_json::from_slice(&fs::read(&p)?)
.with_context(|| format!("invalid history {}", p.display()))?,
);
}
}
out.sort_by(|a, b| a.id.cmp(&b.id));
Ok(out)
}
pub fn atomic(path: &Path, bytes: &[u8]) -> Result<()> {
let parent = path.parent().context("missing parent")?;
fs::create_dir_all(parent)?;
let mut tmp = tempfile::NamedTempFile::new_in(parent)?;
tmp.write_all(bytes)?;
tmp.as_file().sync_all()?;
tmp.persist(path)
.map_err(|e| e.error)
.with_context(|| format!("writing {}", path.display()))?;
Ok(())
}
#[derive(Default)]
pub struct Plan {
pub writes: BTreeMap<PathBuf, Vec<u8>>,
pub messages: Vec<String>,
pub directories: Vec<PathBuf>,
}
impl Plan {
pub fn put(&mut self, path: PathBuf, bytes: Vec<u8>) -> Result<()> {
if let Some(old) = self.writes.get(&path)
&& old != &bytes
{
bail!("conflicting staged writes to {}", path.display());
}
self.writes.insert(path, bytes);
Ok(())
}
pub fn commit(&self, record: &mut Record) -> Result<()> {
for dir in &self.directories {
fs::create_dir_all(dir)?;
}
let backup_root = config::data_dir().join("backups").join(format!(
"{}-{}",
record.id,
chrono::Utc::now().timestamp_millis()
));
for (p, bytes) in &self.writes {
if p.exists() {
let old = fs::read(p).with_context(|| format!("reading target {}", p.display()))?;
if old == *bytes {
continue;
}
fs::create_dir_all(&backup_root)?;
let backup = backup_root.join(format!(
"{}-{}",
record.backups.len(),
p.file_name().unwrap_or_default().to_string_lossy()
));
fs::write(&backup, old)?;
record.backups.push(backup);
}
record.message = format!("writing {}", p.display());
record.save()?;
atomic(p, bytes)?;
record.changed.push(p.clone());
record.save()?;
}
Ok(())
}
}
+306
View File
@@ -0,0 +1,306 @@
//! Merge imported symbol libraries without silently replacing existing symbols.
use crate::sexpr::Node;
use anyhow::{Context, Result, bail};
use sha2::{Digest, Sha256};
use std::collections::{BTreeMap, BTreeSet};
fn name(node: &Node) -> Result<&str> {
let name = node.value().context("symbol has no name")?;
if name.is_empty() || name == "." || name == ".." || name.contains(['/', '\\', ':', '\0']) {
bail!("invalid KiCad symbol name: {name:?}");
}
Ok(name)
}
fn find<'a>(library: &'a Node, name: &str) -> Option<&'a Node> {
library
.children()
.iter()
.find(|n| n.head() == Some("symbol") && n.value() == Some(name))
}
fn rename(node: &mut Node, old: &str, new: &str) {
if node.head() == Some("symbol")
&& let Some(current) = node.value()
{
let renamed = if current == old {
Some(new.to_owned())
} else {
current
.strip_prefix(&format!("{old}_"))
.map(|suffix| format!("{new}_{suffix}"))
};
if let Some(renamed) = renamed {
node.children_mut()[1] = Node::string(renamed);
}
}
if let Node::List(children) = node {
for child in children {
rename(child, old, new);
}
}
}
fn merge_one(
old: &str,
source: &BTreeMap<String, Node>,
resolved: &mut BTreeMap<String, String>,
visiting: &mut BTreeSet<String>,
current: &mut Node,
messages: &mut Vec<String>,
) -> Result<String> {
if let Some(final_name) = resolved.get(old) {
return Ok(final_name.clone());
}
if !visiting.insert(old.to_owned()) {
bail!("cyclic symbol inheritance involving {old}");
}
let mut symbol = source.get(old).context("incoming symbol missing")?.clone();
if let Some(parent) = symbol.field("extends") {
let parent = parent
.value()
.context("extends has no parent name")?
.to_owned();
let final_parent = if source.contains_key(&parent) {
merge_one(&parent, source, resolved, visiting, current, messages)?
} else if find(current, &parent).is_some() {
parent
} else {
bail!("pending: symbol {old} extends missing parent {parent}");
};
symbol.field_mut("extends").unwrap().children_mut()[1] = Node::string(final_parent);
}
let final_name = match find(current, old) {
None => {
current.children_mut().push(symbol);
old.to_owned()
}
Some(existing) if existing == &symbol => {
messages.push(format!("Skip identical symbol {old}"));
old.to_owned()
}
Some(_) => {
// Fingerprint the original name with already-resolved inheritance.
// Renaming this same content again therefore yields the same candidate.
let hash = format!("{:x}", Sha256::digest(symbol.to_string().as_bytes()));
let mut selected = None;
for n in 0..1000 {
let candidate = if n == 0 {
format!("{old}_{}", &hash[..8])
} else {
format!("{old}_{}_{n}", &hash[..8])
};
let mut renamed = symbol.clone();
rename(&mut renamed, old, &candidate);
match find(current, &candidate) {
Some(existing) if existing == &renamed => {
messages.push(format!("Skip identical renamed symbol {candidate}"));
selected = Some(candidate);
break;
}
Some(_) => continue,
None => {
messages.push(format!("Rename conflicting symbol {old} to {candidate}"));
current.children_mut().push(renamed);
selected = Some(candidate);
break;
}
}
}
selected.context("cannot allocate a conflict-free symbol name")?
}
};
visiting.remove(old);
resolved.insert(old.to_owned(), final_name.clone());
Ok(final_name)
}
pub fn merge(mut current: Node, incoming: Vec<Node>) -> Result<(Node, Vec<String>)> {
if current.head() != Some("kicad_symbol_lib") {
bail!("target is not a symbol library");
}
let mut messages = Vec::new();
let mut version = current
.field("version")
.and_then(Node::value)
.context("target symbol library missing version")?
.parse::<u64>()?;
for library in incoming {
if library.head() != Some("kicad_symbol_lib") {
bail!("incoming asset is not a symbol library");
}
version = version.max(
library
.field("version")
.and_then(Node::value)
.context("incoming symbol library missing version")?
.parse::<u64>()?,
);
for item in library.children().iter().skip(1).filter(|n| {
!matches!(
n.head(),
Some("symbol" | "version" | "generator" | "generator_version")
)
}) {
if current.children().contains(item) {
continue;
}
if current.children().iter().any(|n| n.head() == item.head()) {
bail!(
"pending: conflicting library-level metadata {:?}",
item.head()
);
}
current.children_mut().push(item.clone());
}
let mut source = BTreeMap::new();
let mut order = Vec::new();
for symbol in library
.children()
.iter()
.filter(|n| n.head() == Some("symbol"))
{
let old = name(symbol)?.to_owned();
if let Some(previous) = source.insert(old.clone(), symbol.clone()) {
if previous != *symbol {
bail!(
"pending: conflicting definitions of symbol {old} inside one incoming library"
);
}
} else {
order.push(old);
}
}
// This map belongs to one source library. Unrelated libraries can have
// different parents with the same name without retargeting each other.
let mut resolved = BTreeMap::new();
let mut visiting = BTreeSet::new();
for old in order {
merge_one(
&old,
&source,
&mut resolved,
&mut visiting,
&mut current,
&mut messages,
)?;
}
}
current.field_mut("version").unwrap().children_mut()[1] = Node::symbol(version.to_string());
Ok((current, messages))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::sexpr::parse;
fn library(symbols: &str) -> Node {
parse(&format!(
"(kicad_symbol_lib (version 20231120) (generator test) {symbols})"
))
.unwrap()
}
fn count(lib: &Node) -> usize {
lib.children()
.iter()
.filter(|n| n.head() == Some("symbol"))
.count()
}
#[test]
fn conflicts_rename_units_and_repeat_import_is_idempotent() {
let base = library(r#"(symbol "Part" (property "Value" "original"))"#);
let incoming = library(
r#"(symbol "Part" (property "Value" "changed") (symbol "Part_0_1" (future "Ω (keep)")))"#,
);
let (merged, _) = merge(base, vec![incoming.clone()]).unwrap();
assert_eq!(count(&merged), 2);
let renamed = merged
.children()
.iter()
.rfind(|n| n.head() == Some("symbol"))
.unwrap();
assert_eq!(
renamed.field("symbol").unwrap().value().unwrap(),
format!("{}_0_1", renamed.value().unwrap())
);
let (again, _) = merge(merged.clone(), vec![incoming]).unwrap();
assert_eq!(merged, again);
}
#[test]
fn inherited_conflicts_resolve_parent_first_and_reimport_is_idempotent() {
let base = library(
r#"(symbol "Parent" (property "Value" "original")) (symbol "Child" (extends "Parent"))"#,
);
let incoming = library(
r#"(symbol "Child" (extends "Parent")) (symbol "Parent" (property "Value" "changed"))"#,
);
let (merged, _) = merge(base, vec![incoming.clone()]).unwrap();
assert_eq!(count(&merged), 4);
let child = merged
.children()
.iter()
.find(|n| {
n.head() == Some("symbol") && n.value().is_some_and(|v| v.starts_with("Child_"))
})
.unwrap();
let parent = child.field("extends").unwrap().value().unwrap();
assert!(parent.starts_with("Parent_"));
assert!(find(&merged, parent).is_some());
assert_eq!(merge(merged.clone(), vec![incoming]).unwrap().0, merged);
}
#[test]
fn rename_maps_are_scoped_to_source_library() {
let base = library(r#"(symbol "Parent" (property "Value" "original"))"#);
let a = library(
r#"(symbol "Parent" (property "Value" "changed")) (symbol "ChildA" (extends "Parent"))"#,
);
let b = library(
r#"(symbol "Parent" (property "Value" "original")) (symbol "ChildB" (extends "Parent"))"#,
);
let (merged, _) = merge(base, vec![a, b]).unwrap();
assert!(
find(&merged, "ChildA")
.unwrap()
.field("extends")
.unwrap()
.value()
.unwrap()
.starts_with("Parent_")
);
assert_eq!(
find(&merged, "ChildB")
.unwrap()
.field("extends")
.unwrap()
.value(),
Some("Parent")
);
}
#[test]
fn rejects_missing_and_cyclic_inheritance() {
let base = library("");
assert!(
merge(
base.clone(),
vec![library(r#"(symbol "A" (extends "Missing"))"#)]
)
.is_err()
);
assert!(
merge(
base,
vec![library(
r#"(symbol "A" (extends "B")) (symbol "B" (extends "A"))"#
)]
)
.is_err()
);
}
#[test]
fn preserves_unknown_metadata_and_max_version() {
let base = library("");
let future = parse(r#"(kicad_symbol_lib (version 20261101) (future_library "Ω") (symbol "A" (unknown (uuid "meaningful"))))"#).unwrap();
let (merged, _) = merge(base, vec![future]).unwrap();
assert_eq!(merged.field("version").unwrap().value(), Some("20261101"));
assert_eq!(merged.field("future_library").unwrap().value(), Some("Ω"));
assert!(find(&merged, "A").unwrap().field("unknown").is_some());
}
}
+13
View File
@@ -0,0 +1,13 @@
[Unit]
Description=UniLoader KiCad component importer
PartOf=graphical-session.target
After=graphical-session.target
[Service]
Type=simple
ExecStart=%h/.local/bin/uniloader watch
Restart=on-failure
RestartSec=3
[Install]
WantedBy=graphical-session.target
+971
View File
@@ -0,0 +1,971 @@
//! End-to-end CLI fixtures: all filesystem and process state is isolated.
use std::{
fs,
io::Write,
path::{Path, PathBuf},
process::{Command, Output},
};
use tempfile::TempDir;
use zip::{ZipWriter, write::SimpleFileOptions};
#[path = "../src/sexpr.rs"]
#[allow(dead_code)]
mod sexpr;
use sexpr::{Node, parse};
struct Harness {
_tmp: TempDir,
root: PathBuf,
project: PathBuf,
config: PathBuf,
}
impl Harness {
fn new(policy: &str) -> Self {
let temp = tempfile::tempdir().unwrap();
let root = temp.path().to_owned();
let project = root.join("project/board.kicad_pro");
fs::create_dir_all(project.parent().unwrap()).unwrap();
fs::write(&project, "{}").unwrap();
let config = root.join("config.toml");
let h = Self {
_tmp: temp,
root,
project,
config,
};
h.policy(policy);
h
}
fn policy(&self, policy: &str) {
fs::write(&self.config, format!("[libraries]\npath = {:?}\nkicad_config = {:?}\n[matching]\nambiguous = {policy:?}\n", self.root.join("global").to_str().unwrap(), self.root.join("kicad-config").to_str().unwrap())).unwrap();
}
fn cmd(&self, args: &[&str]) -> Output {
Command::new(env!("CARGO_BIN_EXE_uniloader"))
.args([
"--config",
self.config.to_str().unwrap(),
"--noninteractive",
])
.args(args)
.env("HOME", self.root.join("home"))
.env("XDG_CONFIG_HOME", self.root.join("xdg-config"))
.env("XDG_DATA_HOME", self.root.join("xdg-data"))
.env("XDG_CACHE_HOME", self.root.join("xdg-cache"))
.env_remove("DISPLAY")
.env_remove("WAYLAND_DISPLAY")
.current_dir(&self.root)
.output()
.unwrap()
}
fn import(&self, source: &Path, extra: &[&str]) -> Output {
let mut args = vec![
"import",
source.to_str().unwrap(),
"--project",
self.project.to_str().unwrap(),
];
args.extend_from_slice(extra);
self.cmd(&args)
}
fn bundle(&self, name: &str, files: &[(&str, String)]) -> PathBuf {
let path = self.root.join(name);
fs::create_dir_all(&path).unwrap();
for (file, text) in files {
let p = path.join(file);
fs::create_dir_all(p.parent().unwrap()).unwrap();
fs::write(p, text).unwrap();
}
path
}
fn archive(&self, name: &str, files: &[(&str, String)]) -> PathBuf {
let path = self.root.join(name);
let mut zip = ZipWriter::new(fs::File::create(&path).unwrap());
for (file, text) in files {
zip.start_file(*file, SimpleFileOptions::default()).unwrap();
zip.write_all(text.as_bytes()).unwrap();
}
zip.finish().unwrap();
path
}
fn library(&self) -> Node {
parse(&fs::read_to_string(self.root.join("project/libraries/UniLoader.kicad_sym")).unwrap())
.unwrap()
}
fn footprints(&self) -> Vec<PathBuf> {
let dir = self.root.join("project/libraries/UniLoader.pretty");
if !dir.exists() {
return vec![];
}
let mut paths: Vec<_> = fs::read_dir(dir)
.unwrap()
.map(|e| e.unwrap().path())
.filter(|p| p.extension().is_some_and(|e| e == "kicad_mod"))
.collect();
paths.sort();
paths
}
}
fn ok(out: &Output) {
assert!(
out.status.success(),
"stdout:\n{}\nstderr:\n{}",
String::from_utf8_lossy(&out.stdout),
String::from_utf8_lossy(&out.stderr)
);
}
fn output(out: &Output) -> String {
format!(
"{}{}",
String::from_utf8_lossy(&out.stdout),
String::from_utf8_lossy(&out.stderr)
)
}
fn footprint(name: &str, width: f64, extra: &str) -> String {
format!(
r#"(footprint "{name}" (version 20240108) (generator "fixture") (layer "F.Cu") (pad "1" smd rect (at 0 0) (size {width} 1) (layers "F.Cu" "F.Paste" "F.Mask")) {extra})"#
)
}
fn symbol(name: &str, fp: &str) -> String {
format!(
r#"(kicad_symbol_lib (version 20231120) (generator "fixture") (symbol "{name}" (property "Reference" "U") (property "Value" "{name}") (property "Footprint" "{fp}") (symbol "{name}_0_1" (text "µ Ω (quoted)" (at 0 0 0) (effects (font (size 1 1)))))))"#
)
}
fn symbols(n: &Node) -> Vec<&Node> {
n.children()
.iter()
.filter(|n| n.head() == Some("symbol"))
.collect()
}
fn link(n: &Node) -> &str {
n.children()
.iter()
.find(|n| n.head() == Some("property") && n.value() == Some("Footprint"))
.unwrap()
.children()[2]
.atom()
.unwrap()
}
#[test]
fn imports_three_provider_layouts_and_preserves_original_archives() {
for (archive, sym_path, fp_path, readme) in [
(
"LIB_123.zip",
"SamacSys_Parts/KiCad/Parts.kicad_sym",
"SamacSys_Parts/KiCad/Package.kicad_mod",
"SamacSys",
),
(
"ul_123.zip",
"KiCAD/component.kicad_sym",
"KiCAD/footprints.pretty/Package.kicad_mod",
"UltraLibrarian",
),
(
"snapmagic.zip",
"Component.kicad_sym",
"Component.pretty/Package.kicad_mod",
"SnapEDA",
),
] {
let h = Harness::new("ask");
let source = h.archive(
archive,
&[
(sym_path, symbol("Part", "OldLib:Package")),
(fp_path, footprint("Package", 1.0, "")),
("How-to-import.txt", readme.into()),
],
);
ok(&h.import(&source, &[]));
assert!(source.is_file());
assert_eq!(h.footprints().len(), 1);
let lib = h.library();
assert_eq!(link(symbols(&lib)[0]), "UniLoader_Local:Package");
assert!(lib.to_string().contains("µ Ω (quoted)"));
assert_eq!(lib.field("version").unwrap().value(), Some("20231120"));
}
}
#[test]
fn exact_cross_provider_reuse_and_repeat_import_are_idempotent() {
let h = Harness::new("ask");
let a = h.archive(
"LIB_a.zip",
&[
("a.kicad_sym", symbol("First", "PkgA")),
("PkgA.kicad_mod", footprint("PkgA", 1.0, "(uuid a)")),
],
);
let b = h.archive(
"ul_b.zip",
&[
("b.kicad_sym", symbol("Second", "PkgB")),
("PkgB.kicad_mod", footprint("PkgB", 1.0, "(uuid b)")),
],
);
ok(&h.import(&a, &[]));
ok(&h.import(&b, &[]));
assert_eq!(h.footprints().len(), 1);
let lib = h.library();
assert_eq!(symbols(&lib).len(), 2);
assert!(
symbols(&lib)
.iter()
.all(|s| link(s) == "UniLoader_Local:PkgA")
);
ok(&h.import(&a, &[]));
ok(&h.import(&b, &["--force"]));
assert_eq!(symbols(&h.library()).len(), 2);
assert_eq!(h.footprints().len(), 1);
}
#[test]
fn uncertain_footprints_stay_pending_then_retry_with_separate_policy() {
let h = Harness::new("ask");
let first = h.bundle(
"first",
&[
("a.kicad_sym", symbol("First", "Package")),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
let second = h.bundle(
"second",
&[
("b.kicad_sym", symbol("Second", "Package")),
("Package.kicad_mod", footprint("Package", 2.0, "")),
],
);
ok(&h.import(&first, &[]));
let result = h.import(&second, &[]);
assert!(output(&result).contains("pending"), "{}", output(&result));
assert_eq!(h.footprints().len(), 1);
assert_eq!(symbols(&h.library()).len(), 1);
let listing = h.cmd(&["pending"]);
ok(&listing);
assert!(output(&listing).contains("second"));
let records = h.root.join("xdg-data/uniloader/imports");
let pending: serde_json::Value = fs::read_dir(records)
.unwrap()
.map(|e| {
serde_json::from_slice::<serde_json::Value>(&fs::read(e.unwrap().path()).unwrap())
.unwrap()
})
.find(|v| v["source"].as_str().is_some_and(|s| s.ends_with("second")))
.unwrap();
h.policy("separate");
ok(&h.cmd(&["retry", pending["id"].as_str().unwrap()]));
assert_eq!(h.footprints().len(), 2);
let lib = h.library();
let second = symbols(&lib)
.into_iter()
.find(|s| s.value() == Some("Second"))
.unwrap();
assert!(link(second).starts_with("UniLoader_Local:Package_"));
}
#[test]
fn dry_run_does_not_create_libraries_or_tables() {
let h = Harness::new("ask");
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("Part", "Package")),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
let result = h.import(&source, &["--dry-run"]);
ok(&result);
assert!(output(&result).contains("Would write"));
assert!(!h.root.join("project/libraries").exists());
assert!(!h.root.join("project/sym-lib-table").exists());
assert!(!h.root.join("project/fp-lib-table").exists());
}
#[test]
fn preserves_existing_tables_and_unknown_symbol_metadata() {
let h = Harness::new("ask");
fs::create_dir_all(h.root.join("other.pretty")).unwrap();
fs::write(h.root.join("project/fp-lib-table"),format!(r#"(fp_lib_table (version 7) (lib (name "Other") (type "KiCad") (uri "{}") (options "") (descr "Keep µ")))"#,h.root.join("other.pretty").display())).unwrap();
fs::write(h.root.join("project/sym-lib-table"),r#"(sym_lib_table (version 7) (lib (name "Other") (type "KiCad") (uri "${KIPRJMOD}/other.kicad_sym") (options "") (descr "Keep symbols")))"#).unwrap();
let mut sym = symbol("Part", "Package");
sym.insert_str(
sym.find("(symbol \"Part\"").unwrap(),
"(future_library (payload \"Ω (keep)\")) ",
);
let source = h.bundle(
"source",
&[
("a.kicad_sym", sym),
(
"Package.kicad_mod",
footprint("Package", 1.0, "(future_footprint (uuid meaningful))"),
),
],
);
ok(&h.import(&source, &[]));
assert!(h.library().field("future_library").is_some());
assert!(
fs::read_to_string(&h.footprints()[0])
.unwrap()
.contains("future_footprint")
);
for table in ["fp-lib-table", "sym-lib-table"] {
let table =
parse(&fs::read_to_string(h.root.join("project").join(table)).unwrap()).unwrap();
assert_eq!(
table
.children()
.iter()
.filter(|n| n.head() == Some("lib"))
.count(),
2
);
assert!(table.to_string().contains("Other"));
}
}
#[test]
fn repairs_broken_windows_models_and_reports_missing_dependencies() {
let h = Harness::new("ask");
let model = r#"(model "C:\\obsolete\\provider\\part.STEP" (offset (xyz 1 2 3)) (rotate (xyz 0 0 90)) (scale (xyz 1 1 1))) (model "missing.wrl")"#;
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("Part", "Package")),
("Package.kicad_mod", footprint("Package", 1.0, model)),
("deep/models/part.STEP", "STEP TEST CONTENT".into()),
],
);
let result = h.import(&source, &[]);
ok(&result);
assert!(output(&result).contains("Missing model"));
let fp = fs::read_to_string(&h.footprints()[0]).unwrap();
assert!(fp.contains("${KIPRJMOD}/libraries/models/part_"));
assert!(fp.contains("(xyz 1 2 3)"));
assert!(fp.contains("missing.wrl"));
let models: Vec<_> = fs::read_dir(h.root.join("project/libraries/models"))
.unwrap()
.collect();
assert_eq!(models.len(), 1);
}
#[test]
fn multi_symbol_footprint_bindings_do_not_use_first_footprint_for_all() {
let h = Harness::new("separate");
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("First", "PackageA")),
("b.kicad_sym", symbol("Second", "PackageB")),
("PackageA.kicad_mod", footprint("PackageA", 1.0, "")),
("PackageB.kicad_mod", footprint("PackageB", 2.0, "")),
],
);
ok(&h.import(&source, &[]));
let lib = h.library();
for (name, expected) in [
("First", "UniLoader_Local:PackageA"),
("Second", "UniLoader_Local:PackageB"),
] {
let s = symbols(&lib)
.into_iter()
.find(|s| s.value() == Some(name))
.unwrap();
assert_eq!(link(s), expected);
}
}
#[test]
fn creates_and_registers_empty_global_destination() {
let h = Harness::new("ask");
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("Part", "Package")),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
ok(&h.cmd(&["import", source.to_str().unwrap(), "--global"]));
for table in ["sym-lib-table", "fp-lib-table"] {
let text = fs::read_to_string(h.root.join("kicad-config").join(table)).unwrap();
assert!(text.contains("UniLoader"));
assert!(text.contains(h.root.join("global").to_str().unwrap()));
assert!(!text.contains("${KIPRJMOD}"));
}
assert!(!h.root.join("project/libraries").exists());
}
#[test]
fn same_plan_duplicate_footprints_with_models_reuse_one_copy() {
let h = Harness::new("ask");
let model = r#"(model "part.step" (offset (xyz 0 0 0)))"#;
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("First", "PackageA")),
("b.kicad_sym", symbol("Second", "PackageB")),
("PackageA.kicad_mod", footprint("PackageA", 1.0, model)),
("PackageB.kicad_mod", footprint("PackageB", 1.0, model)),
("part.step", "SAME STEP MODEL".into()),
],
);
ok(&h.import(&source, &[]));
assert_eq!(h.footprints().len(), 1);
let lib = h.library();
assert_eq!(symbols(&lib).len(), 2);
assert_eq!(link(symbols(&lib)[0]), link(symbols(&lib)[1]));
}
#[test]
fn symbol_conflict_renames_units_and_force_reimport_stays_idempotent() {
let h = Harness::new("separate");
let first = h.bundle(
"first",
&[
("a.kicad_sym", symbol("Part", "Package")),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
let modified = symbol("Part", "Package").replace("µ Ω (quoted)", "different symbol geometry");
let second = h.bundle(
"second",
&[
("a.kicad_sym", modified),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
ok(&h.import(&first, &[]));
ok(&h.import(&second, &[]));
let lib = h.library();
assert_eq!(symbols(&lib).len(), 2);
let renamed = symbols(&lib)
.into_iter()
.find(|s| s.value() != Some("Part"))
.unwrap();
let renamed_name = renamed.value().unwrap();
assert!(renamed_name.starts_with("Part_"));
assert_eq!(
renamed.field("symbol").unwrap().value().unwrap(),
format!("{renamed_name}_0_1")
);
ok(&h.import(&second, &["--force"]));
assert_eq!(symbols(&h.library()).len(), 2);
}
#[test]
fn portable_project_copies_exact_global_footprint_and_model() {
let h = Harness::new("ask");
let model = r#"(model "part.step" (offset (xyz 0 0 0)))"#;
let global = h.bundle(
"global-source",
&[
("a.kicad_sym", symbol("GlobalPart", "GlobalPackage")),
(
"GlobalPackage.kicad_mod",
footprint("GlobalPackage", 1.0, model),
),
("part.step", "PORTABLE MODEL".into()),
],
);
ok(&h.cmd(&["import", global.to_str().unwrap(), "--global"]));
let local = h.bundle(
"local-source",
&[
("a.kicad_sym", symbol("LocalPart", "LocalPackage")),
(
"LocalPackage.kicad_mod",
footprint("LocalPackage", 1.0, model),
),
("part.step", "PORTABLE MODEL".into()),
],
);
ok(&h.import(&local, &[]));
assert_eq!(h.footprints().len(), 1);
let text = fs::read_to_string(&h.footprints()[0]).unwrap();
assert!(text.contains("${KIPRJMOD}/libraries/models/"));
assert!(!text.contains(h.root.join("global").to_str().unwrap()));
let copied_models: Vec<_> = fs::read_dir(h.root.join("project/libraries/models"))
.unwrap()
.collect();
assert_eq!(copied_models.len(), 1);
assert_eq!(
link(symbols(&h.library())[0]),
"UniLoader_Local:GlobalPackage"
);
}
fn records(h: &Harness) -> Vec<serde_json::Value> {
fs::read_dir(h.root.join("xdg-data/uniloader/imports"))
.unwrap()
.map(|e| serde_json::from_slice(&fs::read(e.unwrap().path()).unwrap()).unwrap())
.collect()
}
fn append_config(h: &Harness, text: &str) {
let mut file = fs::OpenOptions::new().append(true).open(&h.config).unwrap();
writeln!(file, "{text}").unwrap();
}
#[test]
fn symbol_only_global_dependency_is_copied_into_portable_project() {
let h = Harness::new("ask");
let global = h.bundle(
"global-source",
&[
("a.kicad_sym", symbol("GlobalPart", "Package")),
(
"Package.kicad_mod",
footprint("Package", 1.0, r#"(model "part.step")"#),
),
("part.step", "GLOBAL MODEL".into()),
],
);
ok(&h.cmd(&["import", global.to_str().unwrap(), "--global"]));
let source = h.bundle(
"symbol-only",
&[(
"b.kicad_sym",
symbol("LocalPart", "UniLoader_Global:Package"),
)],
);
ok(&h.import(&source, &[]));
assert_eq!(h.footprints().len(), 1);
assert_eq!(link(symbols(&h.library())[0]), "UniLoader_Local:Package");
assert!(
fs::read_to_string(&h.footprints()[0])
.unwrap()
.contains("${KIPRJMOD}/libraries/models/")
);
assert_eq!(
fs::read_dir(h.root.join("project/libraries/models"))
.unwrap()
.count(),
1
);
}
#[test]
fn same_archive_keeps_completed_history_for_each_destination() {
let h = Harness::new("ask");
let source = h.archive(
"LIB_multi.zip",
&[
("a.kicad_sym", symbol("Part", "Package")),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
let second = h.root.join("second/second.kicad_pro");
fs::create_dir_all(second.parent().unwrap()).unwrap();
fs::write(&second, "{}").unwrap();
ok(&h.import(&source, &[]));
ok(&h.cmd(&[
"import",
source.to_str().unwrap(),
"--project",
second.to_str().unwrap(),
]));
let all = records(&h);
assert_eq!(all.len(), 2);
assert!(all.iter().all(|r| r["status"] == "completed"));
assert_ne!(all[0]["destination"], all[1]["destination"]);
let again = h.import(&source, &[]);
ok(&again);
assert!(output(&again).contains("Already imported"));
assert_eq!(records(&h).len(), 2);
}
#[test]
fn project_library_and_model_paths_resolve_after_directory_relocation() {
let h = Harness::new("ask");
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("Part", "Package")),
(
"Package.kicad_mod",
footprint("Package", 1.0, r#"(model "part.step")"#),
),
("part.step", "RELOCATED MODEL".into()),
],
);
ok(&h.import(&source, &[]));
let relocated = h.root.join("relocated project");
fs::rename(h.root.join("project"), &relocated).unwrap();
for file in ["sym-lib-table", "fp-lib-table"] {
let table = parse(&fs::read_to_string(relocated.join(file)).unwrap()).unwrap();
for lib in table.children().iter().filter(|n| n.head() == Some("lib")) {
let uri = lib.field("uri").unwrap().value().unwrap();
assert!(uri.starts_with("${KIPRJMOD}/"));
assert!(
relocated
.join(uri.strip_prefix("${KIPRJMOD}/").unwrap())
.exists()
);
}
}
let fp = parse(
&fs::read_to_string(relocated.join("libraries/UniLoader.pretty/Package.kicad_mod"))
.unwrap(),
)
.unwrap();
let uri = fp.field("model").unwrap().value().unwrap();
let path = relocated.join(uri.strip_prefix("${KIPRJMOD}/").unwrap());
assert_eq!(fs::read(path).unwrap(), b"RELOCATED MODEL");
let lib = parse(&fs::read_to_string(relocated.join("libraries/UniLoader.kicad_sym")).unwrap())
.unwrap();
assert_eq!(link(symbols(&lib)[0]), "UniLoader_Local:Package");
}
#[test]
fn disabled_footprint_import_reuses_registered_assets_and_rejects_dangling_links() {
let h = Harness::new("ask");
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("Part", "Package")),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
ok(&h.import(&source, &[]));
append_config(&h, "[options]\nfootprints = false");
let valid = h.bundle(
"valid",
&[("b.kicad_sym", symbol("Second", "UniLoader_Local:Package"))],
);
ok(&h.import(&valid, &[]));
let invalid = h.bundle(
"invalid",
&[("c.kicad_sym", symbol("Third", "Missing:NoSuchPackage"))],
);
let result = h.import(&invalid, &[]);
assert!(!result.status.success());
assert!(output(&result).contains("pending"));
assert_eq!(symbols(&h.library()).len(), 2);
assert_eq!(h.footprints().len(), 1);
}
#[test]
fn same_model_filename_with_different_content_gets_distinct_destinations() {
let h = Harness::new("separate");
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("First", "PackageA")),
("b.kicad_sym", symbol("Second", "PackageB")),
(
"PackageA.kicad_mod",
footprint("PackageA", 1.0, r#"(model "a/part.step")"#),
),
(
"PackageB.kicad_mod",
footprint("PackageB", 2.0, r#"(model "b/part.step")"#),
),
("a/part.step", "MODEL A".into()),
("b/part.step", "MODEL B".into()),
],
);
ok(&h.import(&source, &[]));
let mut model_contents: Vec<_> = fs::read_dir(h.root.join("project/libraries/models"))
.unwrap()
.map(|e| fs::read(e.unwrap().path()).unwrap())
.collect();
model_contents.sort();
assert_eq!(
model_contents,
vec![b"MODEL A".to_vec(), b"MODEL B".to_vec()]
);
let paths: Vec<_> = h
.footprints()
.iter()
.map(|p| {
parse(&fs::read_to_string(p).unwrap())
.unwrap()
.field("model")
.unwrap()
.value()
.unwrap()
.to_owned()
})
.collect();
assert_eq!(paths.len(), 2);
assert_ne!(paths[0], paths[1]);
}
#[test]
fn retry_keeps_reviewed_project_when_default_destination_changes() {
let h = Harness::new("ask");
let first = h.bundle(
"first",
&[
("a.kicad_sym", symbol("First", "Package")),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
let second = h.bundle(
"second",
&[
("b.kicad_sym", symbol("Second", "Package")),
("Package.kicad_mod", footprint("Package", 2.0, "")),
],
);
ok(&h.import(&first, &[]));
assert!(!h.import(&second, &[]).status.success());
let pending = records(&h)
.into_iter()
.find(|r| r["status"] == "pending")
.unwrap();
let other = h.root.join("other/other.kicad_pro");
fs::create_dir_all(other.parent().unwrap()).unwrap();
fs::write(&other, "{}").unwrap();
h.policy("separate");
append_config(
&h,
&format!(
"[destination]\npolicy = \"project\"\nproject = {:?}",
other.to_str().unwrap()
),
);
ok(&h.cmd(&["retry", pending["id"].as_str().unwrap()]));
assert_eq!(h.footprints().len(), 2);
assert!(!other.parent().unwrap().join("libraries").exists());
}
#[test]
fn registered_footprint_uses_filename_as_library_identifier() {
let h = Harness::new("ask");
let pretty = h.root.join("registered.pretty");
fs::create_dir_all(&pretty).unwrap();
fs::write(
pretty.join("Actual.kicad_mod"),
footprint("InternalOldName", 1.0, ""),
)
.unwrap();
fs::write(h.root.join("project/fp-lib-table"),format!(r#"(fp_lib_table (lib (name "External") (type "KiCad") (uri "{}") (options "") (descr "")))"#,pretty.display())).unwrap();
// Keep the library inside the project to permit in-place portable reuse.
let local_pretty = h.root.join("project/registered.pretty");
fs::rename(&pretty, &local_pretty).unwrap();
fs::write(h.root.join("project/fp-lib-table"),format!(r#"(fp_lib_table (lib (name "External") (type "KiCad") (uri "{}") (options "") (descr "")))"#,local_pretty.display())).unwrap();
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("Part", "Incoming")),
("Incoming.kicad_mod", footprint("Incoming", 1.0, "")),
],
);
ok(&h.import(&source, &[]));
assert_eq!(link(symbols(&h.library())[0]), "External:Actual");
}
#[test]
fn disabled_footprints_cannot_reference_an_unregistered_managed_directory() {
let h = Harness::new("ask");
append_config(&h, "[options]\nfootprints = false");
let pretty = h.root.join("project/libraries/UniLoader.pretty");
fs::create_dir_all(&pretty).unwrap();
fs::write(
pretty.join("Package.kicad_mod"),
footprint("Package", 1.0, ""),
)
.unwrap();
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("Part", "Package")),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
let result = h.import(&source, &[]);
// Registration without copying is also acceptable, but a successful import
// must never write a symbol link whose library nickname cannot be resolved.
if result.status.success() {
let table = parse(
&fs::read_to_string(h.root.join("project/fp-lib-table"))
.expect("successful import needs footprint registration"),
)
.unwrap();
assert!(table.children().iter().any(|n| n.head() == Some("lib")
&& n.field("name").and_then(Node::value) == Some("UniLoader_Local")));
} else {
assert!(output(&result).contains("pending"));
assert!(
!h.root
.join("project/libraries/UniLoader.kicad_sym")
.exists()
);
}
}
#[test]
fn qualified_existing_reference_is_not_silently_retargeted_by_bundle_basename() {
let h = Harness::new("separate");
let pretty = h.root.join("project/existing.pretty");
fs::create_dir_all(&pretty).unwrap();
fs::write(
pretty.join("Package.kicad_mod"),
footprint("Package", 5.0, ""),
)
.unwrap();
fs::write(h.root.join("project/fp-lib-table"),format!(r#"(fp_lib_table (lib (name "Existing") (type "KiCad") (uri "{}") (options "") (descr "")))"#,pretty.display())).unwrap();
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("First", "Incoming:Package")),
("b.kicad_sym", symbol("Second", "Existing:Package")),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
let result = h.import(&source, &[]);
if result.status.success() {
let lib = h.library();
let second = symbols(&lib)
.into_iter()
.find(|s| s.value() == Some("Second"))
.unwrap();
assert_eq!(link(second), "Existing:Package");
} else {
assert!(output(&result).contains("pending"));
}
}
#[test]
fn doctor_migrates_both_scopes_and_project_references_with_backups() {
let h = Harness::new("separate");
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("Part", "Package")),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
ok(&h.cmd(&["import", source.to_str().unwrap(), "--global"]));
ok(&h.import(&source, &[]));
// Recreate v0.2.1 registrations and stored footprint bindings.
let paths = [
"kicad-config/sym-lib-table",
"kicad-config/fp-lib-table",
"global/UniLoader.kicad_sym",
"project/sym-lib-table",
"project/fp-lib-table",
"project/libraries/UniLoader.kicad_sym",
];
for path in paths {
let path = h.root.join(path);
let text = fs::read_to_string(&path)
.unwrap()
.replace("UniLoader_Global", "UniLoader")
.replace("UniLoader_Local", "UniLoader");
fs::write(path, text).unwrap();
}
let sch = h.root.join("project/board.kicad_sch");
fs::write(&sch, r#"(kicad_sch (lib_symbols (symbol "UniLoader:Part" (property "Footprint" "UniLoader:Package"))) (symbol (lib_id "UniLoader:Part") (property "Footprint" "UniLoader:Package") (property "Description" "UniLoader:keep")))"#).unwrap();
let pcb = h.root.join("project/board.kicad_pcb");
fs::write(
&pcb,
r#"(kicad_pcb (footprint "UniLoader:Package" (property "Value" "keep")))"#,
)
.unwrap();
let before = fs::read(&sch).unwrap();
let args = ["doctor", "--project", h.project.to_str().unwrap()];
let preview = h.cmd(&args);
ok(&preview);
assert!(output(&preview).contains("Would repair"));
assert_eq!(fs::read(&sch).unwrap(), before);
// Imports must not silently retain a legacy registration.
let blocked = h.import(&source, &["--force"]);
assert!(!blocked.status.success());
assert!(output(&blocked).contains("doctor"));
let fixed = h.cmd(&["doctor", "--project", h.project.to_str().unwrap(), "--fix"]);
ok(&fixed);
assert!(output(&fixed).contains("Backup:"));
assert!(
walkdir::WalkDir::new(h.root.join("xdg-data/uniloader/backups"))
.into_iter()
.filter_map(Result::ok)
.any(|entry| entry.file_type().is_file() && fs::read(entry.path()).unwrap() == before)
);
assert!(
fs::read_to_string(&sch)
.unwrap()
.contains("UniLoader_Local:Part")
);
assert!(fs::read_to_string(&sch).unwrap().contains("UniLoader:keep"));
assert!(
fs::read_to_string(&pcb)
.unwrap()
.contains("UniLoader_Local:Package")
);
assert_eq!(link(symbols(&h.library())[0]), "UniLoader_Local:Package");
let global =
parse(&fs::read_to_string(h.root.join("global/UniLoader.kicad_sym")).unwrap()).unwrap();
assert_eq!(link(symbols(&global)[0]), "UniLoader_Global:Package");
let again = h.cmd(&["doctor", "--project", h.project.to_str().unwrap(), "--fix"]);
ok(&again);
assert!(output(&again).contains("No nickname repairs needed"));
ok(&h.import(&source, &["--force"]));
}
#[test]
fn doctor_refuses_occupied_nickname_before_writing_anything() {
let h = Harness::new("separate");
fs::create_dir_all(h.root.join("kicad-config")).unwrap();
let table = h.root.join("kicad-config/fp-lib-table");
let text = format!(
r#"(fp_lib_table (lib (name "UniLoader") (type "KiCad") (uri "{}")) (lib (name "UniLoader_Global") (type "KiCad") (uri "/unrelated.pretty") (options "keep")))"#,
h.root.join("global/UniLoader.pretty").display()
);
fs::write(&table, &text).unwrap();
let result = h.cmd(&["doctor", "--global", "--fix"]);
assert!(!result.status.success());
assert!(output(&result).contains("occupied"));
assert_eq!(fs::read_to_string(table).unwrap(), text);
}
#[test]
fn doctor_updates_global_references_in_projects_after_global_migration() {
let h = Harness::new("separate");
let source = h.bundle(
"source",
&[
("a.kicad_sym", symbol("Part", "Package")),
("Package.kicad_mod", footprint("Package", 1.0, "")),
],
);
ok(&h.cmd(&["import", source.to_str().unwrap(), "--global"]));
for path in [
"kicad-config/fp-lib-table",
"kicad-config/sym-lib-table",
"global/UniLoader.kicad_sym",
] {
let path = h.root.join(path);
fs::write(
&path,
fs::read_to_string(&path)
.unwrap()
.replace("UniLoader_Global", "UniLoader_2"),
)
.unwrap();
}
ok(&h.cmd(&["doctor", "--global", "--fix"]));
let sch = h.root.join("project/board.kicad_sch");
let text = r#"(kicad_sch (symbol (lib_id "UniLoader_2:Part") (property "Footprint" "UniLoader_2:Package")))"#;
fs::write(&sch, text).unwrap();
ok(&h.cmd(&["doctor", "--project", h.project.to_str().unwrap(), "--fix"]));
assert!(
fs::read_to_string(&sch)
.unwrap()
.contains("UniLoader_Global:Part")
);
// An unrelated local registration must retain its existing resolution.
fs::write(&sch, text).unwrap();
fs::write(
h.root.join("project/sym-lib-table"),
r#"(sym_lib_table (lib (name "UniLoader_2") (type "KiCad") (uri "/unrelated.kicad_sym")))"#,
)
.unwrap();
ok(&h.cmd(&["doctor", "--project", h.project.to_str().unwrap(), "--fix"]));
assert!(
fs::read_to_string(&sch)
.unwrap()
.contains("\"UniLoader_2:Part\"")
);
}
+81
View File
@@ -0,0 +1,81 @@
//! The native UI uses the installed binary and never needs a scripting runtime.
use std::{
fs,
io::Write,
process::{Command, Stdio},
};
fn internal_dialog(request: &str) -> (std::process::Output, tempfile::TempDir) {
let temp = tempfile::tempdir().unwrap();
let mut child = Command::new(env!("CARGO_BIN_EXE_uniloader"))
.args([
"--config",
temp.path()
.join("must-not-be-created/config.toml")
.to_str()
.unwrap(),
"__dialog",
])
.env("HOME", temp.path())
.env("XDG_CONFIG_HOME", temp.path().join("config"))
.env("XDG_DATA_HOME", temp.path().join("data"))
// The binary must not search PATH for an interpreter or companion helper.
.env("PATH", temp.path().join("empty-path"))
.env_remove("DISPLAY")
.env_remove("WAYLAND_DISPLAY")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.unwrap();
child
.stdin
.take()
.unwrap()
.write_all(request.as_bytes())
.unwrap();
(child.wait_with_output().unwrap(), temp)
}
#[test]
fn no_display_is_pending_without_config_or_external_runtime() {
for request in [
r#"{"mode":"destination","projects":[],"component":"part"}"#,
r#"{"mode":"compare","incoming":"(footprint A)","candidates":[]}"#,
r#"{"mode":"file","title":"model","paths":["/a.step","/b.step"]}"#,
] {
let (output, temp) = internal_dialog(request);
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
assert_eq!(
serde_json::from_slice::<serde_json::Value>(&output.stdout).unwrap(),
serde_json::Value::Null
);
assert_eq!(
fs::read_dir(temp.path()).unwrap().count(),
0,
"native helper must not load config or write state"
);
}
}
#[test]
fn malformed_internal_request_is_rejected_without_writes() {
let (output, temp) = internal_dialog("{bad JSON");
assert!(!output.status.success());
assert!(String::from_utf8_lossy(&output.stderr).contains("invalid native dialog request"));
assert_eq!(fs::read_dir(temp.path()).unwrap().count(), 0);
}
#[test]
fn internal_window_command_is_not_a_user_command() {
let output = Command::new(env!("CARGO_BIN_EXE_uniloader"))
.arg("--help")
.output()
.unwrap();
assert!(output.status.success());
assert!(!String::from_utf8_lossy(&output.stdout).contains("__dialog"));
}
+217
View File
@@ -0,0 +1,217 @@
//! Actual filesystem watcher subprocess: no display, user state or KiCad needed.
use std::{
fs,
io::{Cursor, Write},
path::{Path, PathBuf},
process::{Child, Command, Stdio},
thread,
time::{Duration, Instant, SystemTime},
};
use zip::{ZipWriter, write::SimpleFileOptions};
struct Watcher(Child);
impl Drop for Watcher {
fn drop(&mut self) {
let _ = self.0.kill();
let _ = self.0.wait();
}
}
fn until(label: &str, log: &Path, mut check: impl FnMut() -> bool) {
let start = Instant::now();
while start.elapsed() < Duration::from_secs(5) {
if check() {
return;
}
thread::sleep(Duration::from_millis(40));
}
panic!(
"timed out waiting for {label}; watcher log:\n{}",
fs::read_to_string(log).unwrap_or_default()
);
}
fn archive(name: &str, pad: usize) -> Vec<u8> {
let mut zip = ZipWriter::new(Cursor::new(Vec::new()));
let fp = format!(
r#"(footprint "{name}" (version 20240108) (layer "F.Cu") (pad "{pad}" smd rect (at 0 0) (size 1 1) (layers "F.Cu" "F.Paste" "F.Mask")))"#
);
let sym = format!(
r#"(kicad_symbol_lib (version 20231120) (generator "watch-fixture") (symbol "{name}" (property "Reference" "U") (property "Footprint" "{name}")))"#
);
for (file, data) in [
(format!("{name}.kicad_mod"), fp),
(format!("{name}.kicad_sym"), sym),
] {
zip.start_file(file, SimpleFileOptions::default()).unwrap();
zip.write_all(data.as_bytes()).unwrap();
}
zip.finish().unwrap().into_inner()
}
fn completed(root: &Path) -> usize {
let Ok(entries) = fs::read_dir(root.join("data/uniloader/imports")) else {
return 0;
};
entries
.flatten()
.filter(|entry| {
fs::read(entry.path())
.ok()
.and_then(|b| serde_json::from_slice::<serde_json::Value>(&b).ok())
.is_some_and(|record| record["status"] == "completed")
})
.count()
}
fn snapshot(root: &Path) -> Vec<(PathBuf, Vec<u8>, SystemTime)> {
let mut files = vec![];
for tree in [root.join("project"), root.join("data/uniloader/imports")] {
for entry in walkdir::WalkDir::new(tree).into_iter().flatten() {
if entry.file_type().is_file() {
files.push((
entry.path().to_owned(),
fs::read(entry.path()).unwrap(),
entry.metadata().unwrap().modified().unwrap(),
));
}
}
}
files.sort_by(|a, b| a.0.cmp(&b.0));
files
}
#[test]
fn watcher_handles_partial_and_renamed_downloads_without_blocking_or_reimporting() {
let temp = tempfile::tempdir().unwrap();
let root = temp.path();
let downloads = root.join("downloads");
fs::create_dir_all(&downloads).unwrap();
let project = root.join("project/board.kicad_pro");
fs::create_dir_all(project.parent().unwrap()).unwrap();
fs::write(&project, "{}").unwrap();
let config = root.join("config.toml");
fs::write(&config, format!("[libraries]\npath = {:?}\nkicad_config = {:?}\n[watch]\ndirectories = [{:?}]\nsettle_seconds = 0\nretry_seconds = 2\n[destination]\npolicy = \"project\"\nproject = {:?}\n[matching]\nambiguous = \"separate\"\n[ui]\nenabled = false\n", root.join("global"), root.join("kicad"), downloads, project)).unwrap();
let log = root.join("watch.log");
let spawn = || {
let logfile = fs::File::create(&log).unwrap();
Watcher(
Command::new(env!("CARGO_BIN_EXE_uniloader"))
.args([
"--config",
config.to_str().unwrap(),
"--noninteractive",
"watch",
])
.env("HOME", root.join("home"))
.env("XDG_CONFIG_HOME", root.join("config"))
.env("XDG_DATA_HOME", root.join("data"))
.env("XDG_CACHE_HOME", root.join("cache"))
.env_remove("DISPLAY")
.env_remove("WAYLAND_DISPLAY")
.env_remove("DBUS_SESSION_BUS_ADDRESS")
.current_dir(root)
.stdin(Stdio::null())
.stdout(Stdio::from(logfile.try_clone().unwrap()))
.stderr(Stdio::from(logfile))
.spawn()
.unwrap(),
)
};
let watcher = spawn();
until("watcher readiness", &log, || {
fs::read_to_string(&log)
.unwrap_or_default()
.contains("Watching")
});
// Directory scans emit Linux inotify OPEN events themselves. Idle watching
// must block instead of repeatedly scanning in response to its own reads.
#[cfg(target_os = "linux")]
{
let runtime = || {
fs::read_to_string(format!("/proc/{}/schedstat", watcher.0.id()))
.unwrap()
.split_whitespace()
.next()
.unwrap()
.parse::<u64>()
.unwrap()
};
let before = runtime();
thread::sleep(Duration::from_secs(1));
let cpu = Duration::from_nanos(runtime() - before);
assert!(
cpu < Duration::from_millis(200),
"idle watcher spent {cpu:?} on CPU in one second"
);
}
let first = archive("First", 1);
let second = archive("Independent", 2);
let third = archive("Renamed", 3);
// A corrupt/incomplete final filename must not block an independent ZIP.
fs::write(downloads.join("partial.zip"), &first[..20]).unwrap();
fs::write(downloads.join("independent.zip"), &second).unwrap();
// A completed .part is still not a component until its final rename.
fs::write(downloads.join("renamed.zip.part"), &third).unwrap();
until(
"independent component despite partial archive",
&log,
|| completed(root) == 1,
);
assert!(
!root
.join("project/libraries/UniLoader.pretty/First.kicad_mod")
.exists()
);
assert!(
!root
.join("project/libraries/UniLoader.pretty/Renamed.kicad_mod")
.exists()
);
fs::write(downloads.join("partial.zip.part"), &first).unwrap();
fs::rename(
downloads.join("partial.zip.part"),
downloads.join("partial.zip"),
)
.unwrap();
fs::rename(
downloads.join("renamed.zip.part"),
downloads.join("renamed.zip"),
)
.unwrap();
until("both renamed components", &log, || completed(root) == 3);
for (file, bytes) in [
("partial.zip", &first),
("independent.zip", &second),
("renamed.zip", &third),
] {
assert_eq!(
&fs::read(downloads.join(file)).unwrap(),
bytes,
"source archive must be preserved"
);
}
let before = snapshot(root);
// Duplicate creation and write events, including a new pathname for the
// same archive content, must not edit libraries/history again.
fs::write(downloads.join("duplicate.zip"), &first).unwrap();
fs::write(downloads.join("independent.zip"), &second).unwrap();
thread::sleep(Duration::from_millis(800));
assert_eq!(completed(root), 3);
assert_eq!(before, snapshot(root));
drop(watcher);
let restarted = spawn();
until("watcher restart", &log, || {
fs::read_to_string(&log)
.unwrap_or_default()
.contains("Watching")
});
thread::sleep(Duration::from_millis(800));
assert_eq!(completed(root), 3);
assert_eq!(
before,
snapshot(root),
"restart must not rewrite completed imports"
);
drop(restarted);
}