Initial commit

2026-05-22 18:14:35 +02:00
commit d6900dbf80
9 changed files with 2368 additions and 0 deletions
--- a/STEPS.md
+++ b/STEPS.md
@@ -0,0 +1,764 @@
+# dpg-map Rebuild Steps
+
+This file replaces the old long step list.
+
+There is no Step 0. Initial setup is listed separately, then implementation starts at Step 1.
+
+## Workflow rules
+
+1. Use `uv` for all Python package and dependency management.
+2. Always read `FEATURES.md`, `ARCHITECTURE.md`, and `AGENTS.md` before making code changes.
+3. Keep `AGENTS.md` as a rolling log of what has been done, what is broken, and what comes next.
+4. Update `README.md` whenever public behaviour or examples change.
+5. After every step:
+   - update `AGENTS.md`
+   - run relevant checks
+   - commit to git
+6. Do not casually change public API once introduced.
+7. Public runtime APIs must be safe to call from background threads unless explicitly documented otherwise.
+8. Dear PyGui calls must happen on the GUI thread only.
+9. Prefer stable, boring implementation over clever rendering tricks.
+
+## Initial setup instructions
+
+Run these before Step 1 when starting from a clean repository:
+
+```bash
+mkdir dpg-map
+cd dpg-map
+git init
+uv init --package dpg-map
+uv add dearpygui pillow platformdirs requests
+uv add --dev pytest ruff pyright
+```
+
+Create this structure:
+
+```text
+src/dpg_map/
+  __init__.py
+  api.py
+  widget.py
+  state.py
+  commands.py
+  renderer.py
+  draw_layers.py
+  interaction.py
+  overlays.py
+  providers.py
+  projection.py
+  tiles.py
+  cache.py
+  sizing.py
+  diagnostics.py
+  types.py
+  exceptions.py
+examples/
+tests/
+FEATURES.md
+ARCHITECTURE.md
+STEPS.md
+AGENTS.md
+README.md
+```
+
+Recommended `pyproject.toml` additions:
+
+```toml
+[tool.ruff]
+line-length = 100
+target-version = "py311"
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B", "SIM"]
+
+[tool.pyright]
+typeCheckingMode = "basic"
+```
+
+Create `AGENTS.md` with:
+
+```markdown
+# AGENTS.md
+
+## Current status
+
+Rebuild initialized.
+
+## Completed steps
+
+None yet.
+
+## Current step
+
+Step 1 — Public API contract and pure core.
+
+## Design decisions
+
+- Package is managed with uv.
+- Public import is `import dpg_map as dpgm`.
+- Dear PyGui calls are GUI-thread-only.
+- Runtime public calls enqueue commands or update logical state.
+- Overlay updates must not reset center/zoom.
+- The widget uses child_window + measured-size drawlist.
+- Tiles use a memory cache and persistent disk cache.
+- Tile providers are interchangeable.
+
+## Known issues
+
+None yet.
+
+## Commands used
+
+None yet.
+
+## Next action
+
+Implement Step 1.
+```
+
+Initial commit:
+
+```bash
+uv run python -c "import dpg_map; print(dpg_map.__name__)"
+uv run ruff check .
+uv run ruff format .
+git add .
+git commit -m "initial uv project setup"
+```
+
+## Step 1 — Public API contract and pure core
+
+Goal: lock the public API surface and implement pure non-DPG components first.
+
+Tasks:
+
+1. Define public exports in `__init__.py`.
+
+Required API names:
+
+```python
+configure
+
+TileProvider
+register_provider
+unregister_provider
+get_provider
+list_providers
+
+map_widget
+
+set_center
+get_center
+set_zoom
+get_zoom
+set_view
+fit_bounds
+screen_to_latlon
+latlon_to_screen
+
+add_marker
+add_polyline
+add_trajectory
+update_marker
+update_polyline
+update_trajectory
+set_marker_position
+set_marker_label
+set_polyline_points
+set_overlay_show
+delete_overlay
+
+add_layer
+show_layer
+hide_layer
+clear_layer
+clear_map
+
+set_provider
+
+clear_memory_cache
+clear_disk_cache
+get_cache_stats
+get_map_debug_state
+```
+
+2. Add exceptions in `exceptions.py`.
+
+3. Add common types in `types.py`.
+
+4. Implement `TileProvider` and provider registry in `providers.py`.
+
+5. Implement Web Mercator projection in `projection.py`.
+
+6. Implement initial cache dataclasses and `CacheStats`.
+
+7. Stub GUI-dependent public functions so imports succeed.
+
+8. Add tests for:
+   - package imports
+   - provider registration
+   - provider URL building
+   - invalid provider templates
+   - projection roundtrips
+   - cache stats dataclass construction
+
+Checks:
+
+```bash
+uv run pytest
+uv run ruff check .
+uv run ruff format --check .
+uv run pyright
+```
+
+Acceptance criteria:
+
+- public API imports cleanly
+- provider system works
+- projection tests pass
+- no Dear PyGui context is required for tests
+- API names are fixed before renderer work begins
+
+Commit:
+
+```bash
+git add .
+git commit -m "step 1: lock public api and pure core"
+```
+
+## Step 2 — Thread-safe state, commands, overlays, and cache model
+
+Goal: build the logical state model before rendering anything.
+
+Tasks:
+
+1. Implement `DpgMapConfig` and `configure(...)`.
+
+2. Implement `MapState`.
+
+3. Implement global map registry and current map context stack.
+
+4. Implement `DirtyFlags`.
+
+5. Implement `MapCommand`, `CommandKind`, and `MapCommandQueue`.
+
+6. Implement coalescing rules:
+   - latest overlay update per overlay wins
+   - latest view update wins
+   - structural commands preserve order
+
+7. Implement logical overlay dataclasses:
+   - `Overlay`
+   - `MarkerOverlay`
+   - `PolylineOverlay`
+   - `TrajectoryOverlay`
+   - `LayerState`
+
+8. Implement public overlay creation/update functions so they:
+   - resolve map context or `map_tag`
+   - validate inputs
+   - copy coordinate sequences
+   - enqueue commands or mutate logical model under lock
+   - never call Dear PyGui
+
+9. Implement cache model:
+   - memory cache metadata
+   - disk path generation
+   - disk metadata read/write
+   - disk size scanning
+   - prune planning without deleting active files yet
+
+10. Add tests for:
+    - command coalescing
+    - overlay update does not alter center/zoom
+    - trajectory input lists are copied
+    - mismatched lat/lon lengths raise
+    - layer state
+    - disk cache path generation
+    - disk cache prune ordering
+
+Checks:
+
+```bash
+uv run pytest
+uv run ruff check .
+uv run ruff format --check .
+uv run pyright
+```
+
+Acceptance criteria:
+
+- runtime overlay update functions are thread-safe at the logical level
+- no public overlay function calls Dear PyGui
+- center/zoom cannot be changed by overlay commands
+- cache has separate memory and disk configuration
+- persistent disk cache layout is decided
+
+Commit:
+
+```bash
+git add .
+git commit -m "step 2: add thread safe state commands and cache model"
+```
+
+## Step 3 — Widget shell, sizing system, and GUI-thread frame pump
+
+Goal: create a stable Dear PyGui widget shell before loading real tiles.
+
+Tasks:
+
+1. Implement `map_widget(...)`.
+
+2. Internally create:
+
+```text
+child_window
+  drawlist
+```
+
+3. Child window receives requested sizing values.
+
+4. Drawlist receives concrete measured size only.
+
+5. Implement `sizing.py`:
+   - measure child content region
+   - track last non-zero size
+   - resize drawlist
+   - detect hidden/show-later transitions
+   - avoid permanent zero-size collapse
+
+6. Implement the renderer frame pump:
+   - schedules frame callbacks
+   - drains commands on GUI thread
+   - updates size
+   - draws background and attribution placeholder
+   - exposes debug state
+
+7. Implement map interaction hit-test rectangle calculation, but not full panning yet.
+
+8. Add examples:
+   - `examples/basic_map.py`
+   - `examples/sizing_window.py`
+   - `examples/sizing_child.py`
+   - `examples/sizing_table.py`
+   - `examples/hidden_tab.py`
+
+9. Add tests where possible without opening a GUI:
+   - sizing state transitions
+   - zero-size does not overwrite last non-zero size
+   - dirty flags from resize
+   - command drain ordering
+
+Manual checks:
+
+```bash
+uv run python examples/basic_map.py
+uv run python examples/sizing_window.py
+uv run python examples/sizing_child.py
+uv run python examples/sizing_table.py
+uv run python examples/hidden_tab.py
+```
+
+Automated checks:
+
+```bash
+uv run pytest
+uv run ruff check .
+uv run ruff format --check .
+uv run pyright
+```
+
+Acceptance criteria:
+
+- widget appears
+- map area fills its child container correctly
+- drawlist size matches measured child content size
+- hidden tab example loads/render placeholder after becoming visible
+- no map gets stuck at zero size
+- public API remains unchanged
+
+Commit:
+
+```bash
+git add .
+git commit -m "step 3: add widget shell sizing and frame pump"
+```
+
+## Step 4 — Tile manager, persistent cache, and asynchronous loading
+
+Goal: implement stable tile loading and both cache limits before advanced overlays.
+
+Tasks:
+
+1. Implement `TileID`, `TileStatus`, `Tile`, and `TileManager`.
+
+2. Implement visible tile calculation.
+
+3. Implement persistent disk cache:
+   - provider-namespaced paths
+   - metadata files
+   - `last_accessed_at`
+   - disk size scanning
+   - LRU pruning to `disk_cache_max_bytes`
+
+4. Implement memory cache:
+   - LRU or approximate LRU
+   - `memory_cache_max_tiles`
+   - visible-tile protection
+   - deferred GUI-thread texture deletion
+
+5. Implement worker thread/tile queue:
+   - read disk cache
+   - fetch HTTP if missing
+   - decode tile image
+   - return tile result with map generation and provider name
+   - never call Dear PyGui
+
+6. Renderer GUI thread:
+   - processes tile results
+   - ignores stale generation/provider results
+   - creates textures
+   - deletes evicted textures
+   - draws tile layer
+
+7. Ensure OpenStreetMap provider uses configured User-Agent or warns/raises clearly.
+
+8. Add `examples/cache_stress.py`.
+
+Manual checks:
+
+```bash
+uv run python examples/basic_map.py
+uv run python examples/cache_stress.py
+```
+
+Automated checks:
+
+```bash
+uv run pytest
+uv run ruff check .
+uv run ruff format --check .
+uv run pyright
+```
+
+Acceptance criteria:
+
+- OSM tiles display
+- GUI does not freeze while tiles load
+- only visible tiles plus margin are requested
+- returning to recently viewed area uses memory cache
+- restarting app uses disk cache
+- disk cache prunes to configured limit
+- stale tile results are ignored after provider/generation change
+- texture deletion happens on GUI thread
+
+Commit:
+
+```bash
+git add .
+git commit -m "step 4: add async tiles and persistent cache"
+```
+
+## Step 5 — Interaction: pan, zoom, and view commands
+
+Goal: make interaction stable before adding live overlays.
+
+Tasks:
+
+1. Implement left mouse drag panning.
+
+2. Implement mouse wheel zoom.
+
+3. Zoom around cursor where practical.
+
+4. Implement programmatic view commands:
+   - `set_center`
+   - `set_zoom`
+   - `set_view`
+   - `fit_bounds`
+   - `screen_to_latlon`
+   - `latlon_to_screen`
+
+5. Interaction must:
+   - use concrete map rectangle
+   - work in nested containers
+   - not depend only on child-window hover state
+   - not reset when overlay commands are queued
+   - not recenter from tile results
+
+6. Add debug fields for:
+   - active drag
+   - last mouse position
+   - current center
+   - current zoom
+   - dirty flags
+
+Manual checks:
+
+```bash
+uv run python examples/basic_map.py
+uv run python examples/sizing_child.py
+uv run python examples/sizing_table.py
+```
+
+Automated checks:
+
+```bash
+uv run pytest
+uv run ruff check .
+uv run ruff format --check .
+uv run pyright
+```
+
+Acceptance criteria:
+
+- dragging pans smoothly
+- mouse wheel zooms
+- tile grid remains aligned
+- map works in nested containers
+- view commands work
+- no unexpected recenter occurs from tile loading
+
+Commit:
+
+```bash
+git add .
+git commit -m "step 5: add stable pan zoom and view commands"
+```
+
+## Step 6 — Overlay rendering and runtime update stress tests
+
+Goal: implement overlays in a way that cannot break interaction.
+
+Tasks:
+
+1. Implement marker rendering.
+
+2. Implement polyline rendering.
+
+3. Implement trajectory rendering.
+
+4. Implement overlay layer rendering separate from tile layer.
+
+5. Overlay updates must set only `OVERLAYS` dirty unless a structural map operation requires more.
+
+6. Overlay draw rebuild must not:
+   - clear tile textures
+   - clear tile draw state
+   - reset center
+   - reset zoom
+   - reset active drag state
+   - rebuild the whole map unnecessarily
+
+7. Implement:
+   - `add_marker`
+   - `add_polyline`
+   - `add_trajectory`
+   - `update_marker`
+   - `update_polyline`
+   - `update_trajectory`
+   - `set_marker_position`
+   - `set_marker_label`
+   - `set_polyline_points`
+   - `set_overlay_show`
+   - `delete_overlay`
+
+8. Add examples:
+   - `examples/markers_live_thread.py`
+   - `examples/trajectory_live_thread.py`
+
+The live examples must update overlays from a background thread while the user drags and zooms.
+
+Manual stress checks:
+
+```bash
+uv run python examples/markers_live_thread.py
+uv run python examples/trajectory_live_thread.py
+```
+
+While examples run:
+
+- drag continuously for at least 30 seconds
+- zoom in and out
+- verify no flicker through blank map
+- verify no random recentering
+- verify overlay remains geographically attached
+- verify update thread does not block badly
+
+Automated checks:
+
+```bash
+uv run pytest
+uv run ruff check .
+uv run ruff format --check .
+uv run pyright
+```
+
+Acceptance criteria:
+
+- marker updates at 20–60 Hz do not break dragging
+- trajectory updates at 10–30 Hz do not break dragging
+- overlay updates are coalesced
+- center/zoom remain stable unless explicitly changed
+- overlay update APIs are safe from background threads
+- tile layer is not destroyed by overlay updates
+
+Commit:
+
+```bash
+git add .
+git commit -m "step 6: add stable overlays and live update stress tests"
+```
+
+## Step 7 — Layers, provider switching, and clearing APIs
+
+Goal: add structural map operations after rendering stability is proven.
+
+Tasks:
+
+1. Implement layer operations:
+   - `add_layer`
+   - `show_layer`
+   - `hide_layer`
+   - `clear_layer`
+   - `clear_map`
+
+2. Implement provider switching:
+   - `set_provider`
+
+3. Provider switch must:
+   - keep overlays
+   - keep center
+   - clamp zoom if needed
+   - increment map generation
+   - clear old provider tile draw state
+   - ignore stale tile results
+   - redraw attribution
+   - not perform Dear PyGui work from caller thread
+
+4. Implement cache control:
+   - `clear_memory_cache`
+   - `clear_disk_cache`
+   - `get_cache_stats`
+
+5. Add examples:
+   - `examples/custom_provider.py`
+   - extend `examples/cache_stress.py`
+
+Manual checks:
+
+```bash
+uv run python examples/custom_provider.py
+uv run python examples/cache_stress.py
+```
+
+Automated checks:
+
+```bash
+uv run pytest
+uv run ruff check .
+uv run ruff format --check .
+uv run pyright
+```
+
+Acceptance criteria:
+
+- layers can be shown/hidden/cleared
+- provider switch keeps overlays
+- provider switch does not render stale old-provider tiles
+- cache clear functions do not call Dear PyGui from non-GUI threads
+- memory/disk cache stats are usable
+
+Commit:
+
+```bash
+git add .
+git commit -m "step 7: add layers provider switching and cache controls"
+```
+
+## Step 8 — Documentation, hardening, and internal release
+
+Goal: make the package usable as a dependency in DataFlux or another project.
+
+Tasks:
+
+1. Update `README.md` with:
+   - uv install
+   - local editable dependency
+   - basic map
+   - sizing examples
+   - live marker update
+   - live trajectory update
+   - custom provider
+   - memory/disk cache config
+   - OpenStreetMap attribution/User-Agent note
+   - thread-safety contract
+
+2. Add docstrings for all public functions.
+
+3. Review public API exports.
+
+4. Add tests for failure cases:
+   - unknown map
+   - unknown overlay
+   - unknown provider
+   - invalid coordinates
+   - mismatched lat/lon lengths
+   - empty trajectory
+   - clear deleted overlay
+   - provider switch while tiles loading
+   - overlay update while dragging model state
+
+5. Run all examples manually.
+
+6. Test local editable install from another project:
+
+```bash
+cd ../some-test-project
+uv add -e ../dpg-map
+uv run python -c "import dpg_map as dpgm; print(dpgm.list_providers())"
+```
+
+7. Bump version.
+
+Suggested initial rebuilt beta:
+
+```text
+0.3.0b1
+```
+
+8. Tag release:
+
+```bash
+git tag v0.3.0b1
+```
+
+Final checks:
+
+```bash
+uv sync
+uv run pytest
+uv run ruff check .
+uv run ruff format --check .
+uv run pyright
+```
+
+Acceptance criteria:
+
+- all examples run
+- local editable dependency works
+- live overlay stress examples are stable
+- cache survives restart
+- disk cache limit works
+- sizing examples work
+- README accurately documents thread-safety and cache behaviour
+- AGENTS.md accurately describes status
+
+Commit:
+
+```bash
+git add .
+git commit -m "step 8: harden docs and prepare rebuilt beta"
+git tag v0.3.0b1
+```