karl
b8400c4fbf
Core CI / core (ubuntu-latest) (push) Failing after 3m31s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Successful in 5m39s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s
44 lines
2.6 KiB
Markdown
44 lines
2.6 KiB
Markdown
# Google worker image decoding
|
||
|
||
The production adapter verifies model/projector SHA256 and performs genuine
|
||
Google MTMD image/audio inference. Images are decoded with established Rust
|
||
libraries, not filename captions. `WorkerInfo.media_supported` comes from the
|
||
native handshake; absent fields on older text workers mean false. Media APIs
|
||
return an explicit runtime repair error for such workers while text remains
|
||
compatible.
|
||
|
||
JPEG sources up to 64 Mi pixels (including 48 MP camera photos), 16,384 pixels
|
||
per side, and 64 MiB compressed size are supported. For JPEG above 16 Mi pixels,
|
||
`image-rs/jpeg-decoder` 0.3.2 applies scaled IDCT before final model resizing.
|
||
Its default features are disabled, so this path adds no Rayon worker pool.
|
||
The source dimensions are checked before pixels are decoded. The secondary
|
||
header must match the first to detect file changes. RGB24 and grayscale camera
|
||
JPEGs are supported on this scaled path; unusual large CMYK/16-bit JPEGs fail
|
||
explicitly rather than silently taking an unbounded full-resolution fallback.
|
||
|
||
The published decoder offers 1/8, 1/4, 1/2, and full-scale decoding. It is in
|
||
maintenance mode; the current image 0.25 zune JPEG adapter exposes no scaled
|
||
IDCT interface. See the [decoder API](https://docs.rs/jpeg-decoder/0.3.2/jpeg_decoder/struct.Decoder.html)
|
||
and [upstream maintenance note](https://github.com/image-rs/jpeg-decoder).
|
||
Small JPEGs retain the existing image/zune path and pinned parity behavior.
|
||
|
||
PNG, WebP, and BMP remain capped at 16 Mi pixels with explicit diagnostics.
|
||
Both paths enforce a 256 MiB advertised decoding-buffer ceiling and apply
|
||
EXIF orientation. That ceiling is a buffer/output bound, not an absolute
|
||
process RSS guarantee: compressed data, codec working state, and resizing
|
||
have additional allocations. Extreme/progressive files need broader resource
|
||
profiling. The media fingerprint includes `largejpeg-scaled-v2`; generations
|
||
must additionally include visual token budget. Text fingerprints are unchanged.
|
||
|
||
An executed Windows CPU release test encodes and decodes an actual 8000x6000
|
||
JPEG with EXIF rotation, verifies orientation and region colors, and resizes it
|
||
to a 280-token grid. Decode plus resize took 176–220 ms and produced 672x912
|
||
RGB (1,838,592 bytes). A sampled whole-test peak was 146.86 MiB, including
|
||
144 MB of source fixture construction; it is not a decoder-only measurement
|
||
or a semantic quality benchmark. An 81 MP JPEG header is rejected before
|
||
pixel decode in the default tests.
|
||
|
||
```powershell
|
||
cargo test --release -p searchi-semantic large_phone_jpeg -- --ignored --nocapture
|
||
cargo test -p searchi-semantic --lib
|
||
``` |