karl
b8400c4fbf
2.6 KiB
Google worker image decoding
The production adapter verifies model/projector SHA256 and performs genuine
Google MTMD image/audio inference. Images are decoded with established Rust
libraries, not filename captions. WorkerInfo.media_supported comes from the
native handshake; absent fields on older text workers mean false. Media APIs
return an explicit runtime repair error for such workers while text remains
compatible.
JPEG sources up to 64 Mi pixels (including 48 MP camera photos), 16,384 pixels
per side, and 64 MiB compressed size are supported. For JPEG above 16 Mi pixels,
image-rs/jpeg-decoder 0.3.2 applies scaled IDCT before final model resizing.
Its default features are disabled, so this path adds no Rayon worker pool.
The source dimensions are checked before pixels are decoded. The secondary
header must match the first to detect file changes. RGB24 and grayscale camera
JPEGs are supported on this scaled path; unusual large CMYK/16-bit JPEGs fail
explicitly rather than silently taking an unbounded full-resolution fallback.
The published decoder offers 1/8, 1/4, 1/2, and full-scale decoding. It is in maintenance mode; the current image 0.25 zune JPEG adapter exposes no scaled IDCT interface. See the decoder API and upstream maintenance note. Small JPEGs retain the existing image/zune path and pinned parity behavior.
PNG, WebP, and BMP remain capped at 16 Mi pixels with explicit diagnostics.
Both paths enforce a 256 MiB advertised decoding-buffer ceiling and apply
EXIF orientation. That ceiling is a buffer/output bound, not an absolute
process RSS guarantee: compressed data, codec working state, and resizing
have additional allocations. Extreme/progressive files need broader resource
profiling. The media fingerprint includes largejpeg-scaled-v2; generations
must additionally include visual token budget. Text fingerprints are unchanged.
An executed Windows CPU release test encodes and decodes an actual 8000x6000 JPEG with EXIF rotation, verifies orientation and region colors, and resizes it to a 280-token grid. Decode plus resize took 176–220 ms and produced 672x912 RGB (1,838,592 bytes). A sampled whole-test peak was 146.86 MiB, including 144 MB of source fixture construction; it is not a decoder-only measurement or a semantic quality benchmark. An 81 MP JPEG header is rejected before pixel decode in the default tests.
cargo test --release -p searchi-semantic large_phone_jpeg -- --ignored --nocapture
cargo test -p searchi-semantic --lib