Watch selected folders and reconcile bounded changes with media freshness guards

docs/AUTOMATIC-INDEXING.md

@@ -0,0 +1,89 @@
+# Automatic folder updates
+
+The desktop now registers saved folders at startup and reconciles them in the
+background. Native notifications use pinned `notify` 8.2.0. They are hints:
+[notify documents](https://docs.rs/notify/8.2.0/notify/) missed events, network
+filesystem limitations and platform-specific editor behavior. Periodic scans
+remain necessary; a native watch failure keeps polling and reports the issue.
+
+## Scheduling and bounds
+
+- At most 64 roots and 128 retained paths per root; overflow becomes a full
+  reconciliation rather than an unbounded event queue.
+- A 750 ms quiet period coalesces bursts, with a five-second maximum delay.
+- Startup reconciles saved folders; ordinary verification runs every five minutes.
+- Busy manual indexing retains queued work. Failed scans retry with backoff from
+  five seconds to one minute. Events arriving during a scan remain pending.
+- Read/access-time events are ignored; native rescan flags still take priority.
+- Missing roots keep existing records and retry attachment. Successful
+  reattachment after a prior watch requests verification of missed media events.
+- Callback panics become a retryable failure. Shutdown signals cancellation,
+  wakes the owned thread and joins it without holding its state mutex.
+
+Scans own separate SQLite connections. They do not hold the foreground search
+mutex during traversal or extraction. A changed scan prepares its reader and
+briefly swaps it into the desktop. UI events refresh counts and the active query,
+reject results from before the change, and request normal semantic indexing.
+The original input/first-response timing survives an automatic query refresh.
+
+## Targeted reconciliation
+
+`SearchIndex::index_paths_with_options` handles bounded absolute normalized paths.
+Files update individually; directory hints reconcile only that subtree. Missing
+paths remove only records within their root and path boundary. Parent/child hints
+are coalesced. Outside paths, traversal and symlink ancestors are rejected.
+Read/traversal failure or cancellation rolls back the complete batch.
+
+The cancellable folder and path variants check between entries and before commit.
+An individual bounded parser input can finish before cancellation is observed.
+Parser process isolation and bounded shutdown time still require further work.
+
+Absolute exclusions complement directory-name exclusions. The desktop excludes
+its data, model and worker directories from both scans and notifications.
+Decoded video frames use owned scratch directories under application data,
+so temporary frames cannot feed back into the index when a home folder is watched.
+
+## Media freshness
+
+Core schema 13 adds `files.change_revision`, initially zero for existing records.
+Targeted media hints increment it even when file size and timestamp are unchanged,
+and remove old media tags. Full native-rescan/overflow hints take the targeted
+root path; ordinary periodic scans retain the metadata fast path for media.
+
+Semantic schema 8 records the matching `file_change_revision` on media chunks.
+An old media vector is ineligible after a hint until genuine byte-verified indexing
+completes. Unchanged media can reuse completed vectors with the current revision;
+changed bytes require new Google inference. Metadata probing captures the revision
+before work and writes only if it still matches, preventing a stale probe from
+restoring tags after a concurrent event. Generation activation remains guarded
+by the original corpus revision and actual source SHA validation.
+
+Periodic scans hash supported bounded document inputs, but do not hash every
+large media file. A same-stat media overwrite with no observed native hint remains
+a verification limitation; explicit semantic rebuilds verify actual media bytes.
+
+## Executed validation and remaining gates
+
+Windows native watcher tests observed real create/update/rename/delete events.
+The final Windows workspace run passed 123 tests (19 explicitly ignored), and
+the desktop suite passed 28 tests. Frontend build, 39 tests, formatting and strict
+workspace/desktop Clippy passed. The real CPU FFmpeg decoder regression passed
+with frame paths under the configured owned scratch parent.
+Coordinator tests cover overflow, rescan, busy retention, concurrent events,
+periodic scheduling, panic recovery and shutdown. Core tests cover same-stat
+documents/media, subtree and boundary deletion, exclusions, migrations, rollback,
+concurrent readers and cancellation after a write transaction starts.
+
+The real Google mixed-media CPU regression passed in 83.70 seconds. It verified
+that a same-stat image hint hides old vectors, changed image bytes cause one new
+image encode, unchanged hinted image/audio sources reuse vectors, and current
+revision tokens restore eligibility. This is correctness evidence on a tiny
+fixture, not relevance quality or search/indexing throughput evidence.
+
+Linux and macOS native watcher execution, root disappearance/reappearance on
+real removable/network storage, native UI refresh interaction, representative
+large-corpus contention and power-loss behavior remain open. Full reconciliation
+still holds a write transaction during extraction and can contend with semantic
+writes; targeted batches reduce that work but do not establish a universal
+latency bound. Resource admission currently covers neural inference, not all
+filesystem, decoder, parser or ANN work.

docs/INCREMENTAL-SEMANTIC.md

@@ -1,7 +1,7 @@
 # Incremental semantic indexing and recovery
 
 Status: completed-source reuse and compatible staging recovery implemented in
-schema 7. Unchanged vectors are copied with current citations; the ANN graph
+schema 8 (completed-source tasks were introduced in schema 7). Unchanged vectors are copied with current citations; the ANN graph
 is rebuilt. Incomplete source groups are discarded before retry. The previous
 ready generation and its reader policy remain available on failures.
 

docs/ROADMAP.md

@@ -40,6 +40,12 @@ Status: implementation started; no production release yet. Windows VM tests use
 - [ ] Folder selection, exclusion controls, resource budgets and indexing status.
 - [ ] Global shortcut, opening/revealing source files, PDF page navigation.
 - [ ] Watchers and reconciliation across Windows, macOS and Linux.
+  - [x] Native notification coordinator, bounded targeted reconciliation,
+    periodic verification and CPU-tested Windows create/update/rename/delete.
+  - [x] Durable same-stat media hint invalidation, guarded metadata writes and
+    real Google reindex/reuse regression; owned decoder scratch avoids feedback.
+  - [ ] Native Linux/macOS events, actual UI refresh and representative storage
+    failure/contention checks; see [automatic indexing](AUTOMATIC-INDEXING.md).
 
 ## 4. Media
 - [x] Genuine Google image, audio and video embeddings integrated with the shared