pplx-embed-v2-late
Open late-interaction embeddings that index text, images and PDF pages in one space — no OCR step.
An embedding model — software that turns text, images or PDF pages into lists of numbers, so a computer can search by meaning instead of by keyword. This one is unusual twice over: it keeps a small vector for every word rather than one per document, and it reads a scanned page as a picture instead of running OCR on it first.
What you build with it
- Search a pile of PDFs, contracts or slide decks where keyword search gives up
- Build a RAG pipeline that can quote the exact clause, not just the right document
- Index screenshots, diagrams and text together in one place
- Delete an OCR step that keeps mangling tables
What it gets you
- Finds the specific line, not just a roughly relevant page
- No OCR stage, so no OCR failures downstream
- MIT licence — runs on your own hardware, no licence conversation
- Index once with the 9B model, then query cheaply with the 0.6B
- Sizes
- 0.6B · 9B
- Licence
- MIT
- Vector
- 128-dim per token
- Scoring
- MaxSim (ColBERT-style)
- Modalities
- Text · images · PDF pages
Straight from the source
Why it matters
Most RAG pipelines squeeze a whole document into a single vector and lose everything that did not survive the average. These models keep one small vector per token instead, which is the difference between "this page is broadly about Kafka" and "this exact line mentions exactly-once semantics". They are MIT-licensed, so the whole thing can run on your own hardware.
What late interaction actually changes
A dense embedding model reads a document and emits one vector. Retrieval is then a single dot product per document — fast, cheap, and lossy, because a 200-page PDF and a one-line note both collapse to the same shape.
Late interaction keeps a vector per token — here, 128 dimensions each. At query time every query token is matched against its best-matching document token, and those maxima are summed. That is MaxSim, and it is the same idea ColBERT introduced, applied to a modern multimodal encoder.
The practical effect is that fine-grained matches survive. A query about one clause in a contract can find that clause, rather than losing it inside the average of the whole contract.
The OCR step disappears
The models embed images and rendered PDF pages into the same space as text. A scanned page is embedded as a picture of a page, not as the output of an OCR pass.
That removes an entire failure surface. In most document pipelines OCR is where tables lose their structure, multi-column layouts interleave, and handwriting quietly becomes noise — and every downstream retrieval error inherits those mistakes.
The two sizes share one embedding space
This is the detail worth building around: the 0.6B model can query an index that the 9B model built. The expensive pass over the corpus happens once with the big model; every query afterwards runs on the small one.
For a self-hosted setup that means the heavy GPU is a batch job, not a dependency of your request path.
On Perplexity's own ViDoRe v3 image-retrieval numbers, 0.6B indexing and querying scores 62.3% and the 9B doing both scores 65.2%. On MADQA the 9B scores 92.4%, just behind the reported 93.4% of Mixedbread Agentic Search.
The cost you pay for it
A vector per token is a lot of vectors. Published measurements put multi-vector indexes at roughly 20× to 64× the size of a dense index over the same corpus.
That is the whole trade. On a corpus of a few thousand documents it is irrelevant. On tens of millions it is the design constraint, and the usual answer is a two-stage pipeline: cheap dense retrieval for the first few hundred candidates, late interaction to rerank them.
Reach for it when
- Your corpus is PDFs, scans, slides or screenshots and OCR keeps mangling them
- Queries are specific and precision matters more than index size
- You want weights you can self-host without a licence conversation
- You can afford a nightly batch index on a big GPU and want cheap queries after
Watch out for
- Index size — budget 20–64× a dense index before committing
- Most vector databases are built around one vector per document; check multi-vector support first
- The benchmark numbers are the vendor's own; re-run them on your corpus