karl
caf6a2f82e
Core CI / core (ubuntu-latest) (push) Failing after 3m27s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Successful in 5m45s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s
15 lines
838 B
Markdown
15 lines
838 B
Markdown
# Retrieval regression corpus
|
|
|
|
This synthetic seed corpus contains 20 documents and 24 predetermined queries:
|
|
exact filenames, error/invoice identifiers, remembered concepts and English/German
|
|
cross-language retrieval. Each query labels one expected file. Supplier-contract
|
|
termination and meeting cancellation deliberately share words but differ in intent.
|
|
|
|
Report top-1 accuracy, recall@10, per-query rank and latency. Keep failures in
|
|
reports; do not rewrite expected answers or remove difficult queries to make a
|
|
model pass. Evaluate lexical, semantic and hybrid modes on the same corpus.
|
|
|
|
This corpus is a regression fixture, not a representative user dataset. Production
|
|
quality needs a larger independently labelled corpus, varied document lengths,
|
|
PDF/OCR/media cases, ambiguous/no-answer queries and measured filter correctness.
|