Preserve Unicode Windows arguments when loading native models

docs/WINDOWS-UNICODE.md

@@ -0,0 +1,62 @@
+# Windows Unicode model paths
+
+The native worker must preserve filenames from Windows command-line arguments.
+Windows narrow CRT `char** argv` uses the active ANSI code page; the pinned
+llama/ggml model loader expects UTF-8 filenames. UTF-8 JSON requests alone do
+not correct the model path supplied at startup.
+
+## Observed failure, 2026-10-13
+
+An actual CPU query-role startup used a hardlink to the verified Google backbone
+under `Gemäß_日本_😀`. The model bytes were unchanged (309,855,456 bytes,
+SHA256 `2188ac1deca4b77dffefd603c2776a9d76d9d74ec01841392982ebb840b09135`).
+On this Windows VM the active ANSI code page was 1252. The normal worker with
+SHA256 `1272797af94c21db1ccabaa3ad996ab57d64f1bf9d33280c61d65c51360deab7`
+exited with code 1 before a ready handshake. `gguf_init_from_file` reported an
+invalid argument with the Japanese and emoji components replaced by question
+marks, and the German component no longer valid UTF-8. The captured raw stderr
+is preserved locally in `.searchi/qa/windows-unicode-model-before-stderr.bin`.
+
+## Implementation
+
+Windows enters through `wmain`, converts the original UTF-16 arguments to UTF-8
+with `WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS)`, then calls the shared
+worker entry. Invalid Unicode fails explicitly. Unix continues to pass its
+existing byte arguments through `main`. Normal CPU/GPU selection, protocol,
+model validation and role behavior remain unchanged. MinGW receives its required
+`-municode` link option, but a MinGW build has not been executed or certified.
+
+After the fix, normal MSVC worker SHA256
+`c01cb3ec13cfca722ee768e8350e1399a8b9a4d14a24a35adc3473a5cfc82a73`
+passed the real Unicode query/indexing regression (21.03 s). A strengthened
+regression also relocated the worker and every adjacent DLL as hardlinks under
+the Unicode folder: actual query and full document indexing passed (22.45 s),
+with ASCII/Unicode query-vector cosine 1.00000001. The temporary fixture names
+were verified absent afterward and the original backbone remained intact.
+The unchanged query-context/parity/boundary regression passed again (25.83 s).
+
+A separate native after-probe reported query role, 512-token context, CPU,
+`ggml-cpu-cascadelake.dll`, no Vulkan module, and a normalized 768-dimensional
+vector. Metadata is retained in `.searchi/qa/windows-unicode-model-after.json`.
+The ad-hoc model hardlink was removed only after verifying its absolute path
+under the owned QA root and that the directory contained exactly that link.
+The original verified model remains present. No GPU was executed.
+
+The actual regression creates a uniquely owned temporary directory on the
+model's volume, hardlinks the backbone under a Unicode child, relocates adjacent
+runtime files as hardlinks, and compares
+ASCII/Unicode CPU query vectors. It also opens the full indexing role through
+that Unicode path and encodes a real document. Temporary-name cleanup leaves
+the original verified model intact. No duplicate model download or copy is
+needed; hardware acceleration is not executed in this VM.
+
+```powershell
+$env:SEARCHI_FORCE_CPU = '1'
+$env:SEARCHI_GEMMA_MODEL_DIR = 'PATH/TO/VERIFIED/MODELS'
+$env:SEARCHI_GEMMA_WORKER = 'PATH/TO/CURRENT/searchi-gemma-worker.exe'
+cargo test -p searchi-semantic --test unicode_windows_cpu --locked -- --ignored --nocapture
+```
+
+Native package restaging and installed Unicode-profile paths require their own
+checks. This focused model-path regression does not certify every filesystem,
+codec, launcher or installer path, or any GPU hardware.