44 lines
2.6 KiB
Markdown
44 lines
2.6 KiB
Markdown
|
# Google worker image decoding
|
|||
|
|
|
|||
|
|
The production adapter verifies model/projector SHA256 and performs genuine
|
|||
|
|
Google MTMD image/audio inference. Images are decoded with established Rust
|
|||
|
|
libraries, not filename captions. `WorkerInfo.media_supported` comes from the
|
|||
|
|
native handshake; absent fields on older text workers mean false. Media APIs
|
|||
|
|
return an explicit runtime repair error for such workers while text remains
|
|||
|
|
compatible.
|
|||
|
|
|
|||
|
|
JPEG sources up to 64 Mi pixels (including 48 MP camera photos), 16,384 pixels
|
|||
|
|
per side, and 64 MiB compressed size are supported. For JPEG above 16 Mi pixels,
|
|||
|
|
`image-rs/jpeg-decoder` 0.3.2 applies scaled IDCT before final model resizing.
|
|||
|
|
Its default features are disabled, so this path adds no Rayon worker pool.
|
|||
|
|
The source dimensions are checked before pixels are decoded. The secondary
|
|||
|
|
header must match the first to detect file changes. RGB24 and grayscale camera
|
|||
|
|
JPEGs are supported on this scaled path; unusual large CMYK/16-bit JPEGs fail
|
|||
|
|
explicitly rather than silently taking an unbounded full-resolution fallback.
|
|||
|
|
|
|||
|
|
The published decoder offers 1/8, 1/4, 1/2, and full-scale decoding. It is in
|
|||
|
|
maintenance mode; the current image 0.25 zune JPEG adapter exposes no scaled
|
|||
|
|
IDCT interface. See the [decoder API](https://docs.rs/jpeg-decoder/0.3.2/jpeg_decoder/struct.Decoder.html)
|
|||
|
|
and [upstream maintenance note](https://github.com/image-rs/jpeg-decoder).
|
|||
|
|
Small JPEGs retain the existing image/zune path and pinned parity behavior.
|
|||
|
|
|
|||
|
|
PNG, WebP, and BMP remain capped at 16 Mi pixels with explicit diagnostics.
|
|||
|
|
Both paths enforce a 256 MiB advertised decoding-buffer ceiling and apply
|
|||
|
|
EXIF orientation. That ceiling is a buffer/output bound, not an absolute
|
|||
|
|
process RSS guarantee: compressed data, codec working state, and resizing
|
|||
|
|
have additional allocations. Extreme/progressive files need broader resource
|
|||
|
|
profiling. The media fingerprint includes `largejpeg-scaled-v2`; generations
|
|||
|
|
must additionally include visual token budget. Text fingerprints are unchanged.
|
|||
|
|
|
|||
|
|
An executed Windows CPU release test encodes and decodes an actual 8000x6000
|
|||
|
|
JPEG with EXIF rotation, verifies orientation and region colors, and resizes it
|
|||
|
|
to a 280-token grid. Decode plus resize took 176–220 ms and produced 672x912
|
|||
|
|
RGB (1,838,592 bytes). A sampled whole-test peak was 146.86 MiB, including
|
|||
|
|
144 MB of source fixture construction; it is not a decoder-only measurement
|
|||
|
|
or a semantic quality benchmark. An 81 MP JPEG header is rejected before
|
|||
|
|
pixel decode in the default tests.
|
|||
|
|
|
|||
|
|
```powershell
|
|||
|
|
cargo test --release -p searchi-semantic large_phone_jpeg -- --ignored --nocapture
|
|||
|
|
cargo test -p searchi-semantic --lib
|
|||
|
|
```
|