feat: add multimodal search and granular performance diagnostics
Core CI / core (ubuntu-latest) (push) Failing after 3m31s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Successful in 5m39s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s
Core CI / core (ubuntu-latest) (push) Failing after 3m31s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Successful in 5m39s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s
This commit is contained in:
1 parent
d3bb8c7618
commit
b8400c4fbf
110 files changed
+79395
-250
No files matched your search
@@ -0,0 +1,44 @@
|
||||
# Google worker image decoding
|
||||
|
||||
The production adapter verifies model/projector SHA256 and performs genuine
|
||||
Google MTMD image/audio inference. Images are decoded with established Rust
|
||||
libraries, not filename captions. `WorkerInfo.media_supported` comes from the
|
||||
native handshake; absent fields on older text workers mean false. Media APIs
|
||||
return an explicit runtime repair error for such workers while text remains
|
||||
compatible.
|
||||
|
||||
JPEG sources up to 64 Mi pixels (including 48 MP camera photos), 16,384 pixels
|
||||
per side, and 64 MiB compressed size are supported. For JPEG above 16 Mi pixels,
|
||||
`image-rs/jpeg-decoder` 0.3.2 applies scaled IDCT before final model resizing.
|
||||
Its default features are disabled, so this path adds no Rayon worker pool.
|
||||
The source dimensions are checked before pixels are decoded. The secondary
|
||||
header must match the first to detect file changes. RGB24 and grayscale camera
|
||||
JPEGs are supported on this scaled path; unusual large CMYK/16-bit JPEGs fail
|
||||
explicitly rather than silently taking an unbounded full-resolution fallback.
|
||||
|
||||
The published decoder offers 1/8, 1/4, 1/2, and full-scale decoding. It is in
|
||||
maintenance mode; the current image 0.25 zune JPEG adapter exposes no scaled
|
||||
IDCT interface. See the [decoder API](https://docs.rs/jpeg-decoder/0.3.2/jpeg_decoder/struct.Decoder.html)
|
||||
and [upstream maintenance note](https://github.com/image-rs/jpeg-decoder).
|
||||
Small JPEGs retain the existing image/zune path and pinned parity behavior.
|
||||
|
||||
PNG, WebP, and BMP remain capped at 16 Mi pixels with explicit diagnostics.
|
||||
Both paths enforce a 256 MiB advertised decoding-buffer ceiling and apply
|
||||
EXIF orientation. That ceiling is a buffer/output bound, not an absolute
|
||||
process RSS guarantee: compressed data, codec working state, and resizing
|
||||
have additional allocations. Extreme/progressive files need broader resource
|
||||
profiling. The media fingerprint includes `largejpeg-scaled-v2`; generations
|
||||
must additionally include visual token budget. Text fingerprints are unchanged.
|
||||
|
||||
An executed Windows CPU release test encodes and decodes an actual 8000x6000
|
||||
JPEG with EXIF rotation, verifies orientation and region colors, and resizes it
|
||||
to a 280-token grid. Decode plus resize took 176–220 ms and produced 672x912
|
||||
RGB (1,838,592 bytes). A sampled whole-test peak was 146.86 MiB, including
|
||||
144 MB of source fixture construction; it is not a decoder-only measurement
|
||||
or a semantic quality benchmark. An 81 MP JPEG header is rejected before
|
||||
pixel decode in the default tests.
|
||||
|
||||
```powershell
|
||||
cargo test --release -p searchi-semantic large_phone_jpeg -- --ignored --nocapture
|
||||
cargo test -p searchi-semantic --lib
|
||||
```
|
||||
Reference in new issue
Block a user