---
title: "What Are Document Embeddings? A Practical Guide"
description: "Document embeddings turn documents into vectors so search finds them by meaning. How they work, chunking, choosing a model, and semantic vs keyword search."
url: https://www.docsumo.com/blog/document-embeddings
updated: 2026-09-26
---

# Document embeddings: what they are, how they work and which model to use

For developers and data teams building search, clustering or RAG over documents: how embeddings work, how to chunk and store them, and how to pick a model.

[Sagnik Chakraborty](https://www.docsumo.com/blog-authors/sagnik-chakraborty) · Updated Sep 26, 2026 · 4 min read

## Key takeaways

- **Document embeddings** are dense numerical vectors that capture what a document means, so similar documents sit close together even when they use different words.
- **Semantic search** compares a query's embedding with document embeddings; keyword search matches exact terms. Most production systems combine the two (hybrid search).
- Long documents are **split into chunks** before embedding, because every model has an input limit; chunk boundaries and overlap affect results more than most model choices.
- Pick a model by **testing it on your own documents and queries**: context length, languages, hosting (API or self-hosted) and cost matter as much as leaderboard scores.

**Document embeddings** are numerical vectors that represent what a document means, so documents with similar content sit close together even when they use different words. An embedding model turns each document, or each chunk of it, into a vector, and a vector database stores those vectors for semantic search, clustering, classification and similarity matching. That lets search find related documents by concept, where keyword search finds only exact word matches.

This guide covers how embeddings compare with keyword search, how they're made and searched, and how to choose a model.

## What are document embeddings?

An embedding model reads text and outputs a fixed-length list of numbers, often 384 to 3,072 of them. The model learned from large amounts of text which words and phrases mean similar things, so "indemnification", "liability" and "hold harmless" land near each other, while unrelated documents land far apart. Embedding quality depends on the model, what it was trained on, and how you apply it to your documents.

### Semantic search

Find passages by meaning, not exact wording.

### Clustering

Group similar reports, tickets or contracts without labeling them first.

### Classification

Train a light classifier on embeddings to sort documents by type or topic.

### Duplicate detection

Spot near-identical documents that differ in formatting or wording.

### Retrieval for RAG

Fetch the right chunks for a language model to answer from.

## Keyword search vs semantic search

Keyword search matches the words in the query. Say a legal team searches its contracts for "indemnification obligations related to data breaches": a keyword index returns contracts that use those words and misses those that say "hold harmless" or "cybersecurity incident" for the same obligation. Semantic search embeds the query and returns the nearest documents by meaning, so it finds both.

| Aspect | Keyword search | Semantic search |
| --- | --- | --- |
| Matching | Exact or fuzzy terms | Similarity of meaning |
| Synonyms | Only with manual synonym lists | Handled by the embeddings |
| Query style | Keywords, Boolean operators, exact phrases | Natural-language questions |
| Zero results | Common for unusual wording | Rare; returns the closest matches |
| Cost | Low | Higher: embedding every chunk, storing vectors, embedding each query |
| Best for | IDs, names, exact phrases | Exploratory search, questions, varied wording |

Neither wins everywhere: semantic search can miss an exact invoice or policy number that keyword search finds at once. Most production systems run both and merge the results, which is called hybrid search.

## How document embeddings work

Contracts, Reports, OCR text, Emails → Embeddings: 01 Split into chunks, 02 Encode with a model, 03 Store vectors, 04 Find nearest neighbors → **Ranked passages**

*From documents to semantic search*

**Tokenize and encode.** The model splits text into tokens (words or word pieces), processes them through its layers and outputs one vector per chunk. Sizes vary: BERT-base produces 768 dimensions, and OpenAI's text-embedding-3-small and -large produce 1,536 and 3,072 by default, which can be shortened to save storage.

**Chunk long documents.** Every model has an input limit: 512 tokens for BERT, 8,192 for BGE-M3 and OpenAI's text-embedding-3 models. A 50-page contract has to be split.

1. **Split at natural boundaries**: Sections, headings or paragraphs, rather than a fixed word count that cuts sentences in half.
2. **Overlap the chunks**: A little overlap, such as 50 tokens on a 400-token chunk, keeps ideas that cross a boundary; more overlap means more vectors to store.
3. **Keep the source**: Store each chunk's document ID and page, so a result links back to where it came from.
4. **Return chunks, then documents**: Show the top 5 to 10 chunks, or group them by document.

**Store and search.** Vectors go into a vector database such as Pinecone, Weaviate, Milvus or pgvector for Postgres. Similarity is usually cosine similarity, and approximate nearest neighbor indexes such as HNSW or IVF keep queries fast across millions of vectors, trading a little accuracy for speed. What score counts as "similar" depends on the model, so set thresholds on pairs from your own documents.

## Which embedding model works best for documents?

### BERT-style encoders

Small enough to fine-tune on your own labeled data; you pool token vectors into one document vector.

### Sentence Transformers (SBERT)

A Python library of models trained so that similar sentences and passages get similar vectors; open and self-hostable.

### BGE-M3

An open model (MIT license) for dense, sparse and multi-vector retrieval in 100+ languages, with inputs up to 8,192 tokens.

### API models

OpenAI's text-embedding-3 models, and Cohere's Embed v4, which embeds text, images and mixed PDFs with a 128,000-token context.

- **Test on your documents and queries**: Build 50 to 100 real queries with known right answers and compare which model ranks them highest, using recall@k or mean reciprocal rank (MRR).
- **Match context length to your chunks**: A short input limit forces smaller chunks.
- **Check languages**: If your documents aren't all in English, test each language you have.
- **Decide where it runs**: Self-host an open model when documents can't leave your infrastructure; use an API when they can.
- **Price it at your volume**: Per-token API fees against the cost of running your own servers, including every time you re-embed.
- **Plan for OCR text**: Scanned documents need OCR first, and OCR errors carry into the embeddings, so check text quality before blaming the model.

Fine-tune only after a pre-trained model has proven weak on your task and you have enough labeled examples.

## The bottom line

Embeddings let software find and group documents by meaning, which keyword search can't do alone. Chunk carefully, test models on your own queries, and keep keyword search in the mix for exact values.

Docsumo is an [intelligent document processing platform](https://www.docsumo.com/solutions/intelligent-document-processing-platform), not an embeddings tool: it extracts checked fields and tables from business documents such as [invoices](https://www.docsumo.com/solutions/documents/invoices) and [bank statements](https://www.docsumo.com/solutions/documents/bank-statements), and exact values like those can sit alongside the text you embed. See [document classification](https://www.docsumo.com/blog/document-classification) and [agentic document extraction](https://www.docsumo.com/blog/agentic-document-extraction), or [book a demo](https://www.docsumo.com/book-a-demo).

> **Docsumo for OCR & IDP**: [The Docsumo platform](https://www.docsumo.com/platform)
>
> Document AI, Case Management and Document Workflows in one product.

## Frequently asked questions

### What is a document embedding?

A document embedding is a list of numbers (a vector) that a model produces from a document's text. Documents with similar meanings get vectors that are close together, which lets software search, cluster or classify documents by meaning.

### Which embeddings work best for documents?

It depends on your documents and queries. Common choices are API models such as OpenAI's text-embedding-3 and Cohere's Embed v4, and open models such as BGE-M3 and Sentence Transformers models you can run yourself. Run each on a sample of your own documents and real queries, and keep the one that ranks the right passages highest. For specialized terms, fine-tuning or hybrid search with keywords helps.

### How much does it cost to generate embeddings for my document collection?

With an API you pay per token. OpenAI lists text-embedding-3-small at $0.02 and text-embedding-3-large at $0.13 per 1 million tokens (September 2026). A 10,000-document collection averaging 2,000 tokens per document is 20 million tokens: about $0.40 with the small model and $2.60 with the large one, mostly a one-time cost. Self-hosted open models have no per-token fee, but you pay for the servers and the work of running them.

### Can I use embeddings offline, on my own infrastructure?

Yes. Open models such as BGE-M3 and Sentence Transformers models can be downloaded and run on your own servers or private cloud, so no document text goes to a third-party API. That matters for regulated or sensitive data.

### How do I handle very long documents?

Split them into chunks that fit the model's input limit, ideally at section or paragraph breaks, with some overlap between chunks. Search returns the best chunks, and you link each one back to its source document.

### Is semantic search the same as vector search?

Not quite. Vector search is the database operation: store vectors and return the nearest ones to a query vector. Semantic search is the application built on it for text meaning, including the embedding model, ranking and often a keyword search blended in.

**Sources**

1. [OpenAI: API pricing (embeddings)](https://developers.openai.com/api/docs/pricing)
2. [OpenAI: Embeddings guide (dimensions and input limits)](https://developers.openai.com/api/docs/guides/embeddings)
3. [BAAI: BGE-M3 model card (Hugging Face)](https://huggingface.co/BAAI/bge-m3)
4. [Cohere: Embed models documentation](https://docs.cohere.com/docs/cohere-embed)
5. [Sentence Transformers documentation](https://sbert.net/)
6. [Devlin et al.: BERT, arXiv:1810.04805](https://arxiv.org/abs/1810.04805)

First published Apr 27, 2026. Last updated Sep 26, 2026.

**Written by** [Sagnik Chakraborty](https://www.docsumo.com/blog-authors/sagnik-chakraborty) · Product Marketing Manager

An accidental product marketer, Sagnik tries to weave engaging narratives around the most technical jargons, turning features into stories that sell themselves. When he’s not brainstorming Go-to-Market strategies or deep-diving into his latest campaign's performance, he likes diving into the ocean as a certified open-water diver.

[LinkedIn](https://www.linkedin.com/in/sagnik-b2b-saas-product-content-marketer/) · [All posts by Sagnik](https://www.docsumo.com/blog-authors/sagnik-chakraborty)

- [Agentic document extraction: what it is and how it changes document workflows](https://www.docsumo.com/blog/agentic-document-extraction)
- [The best KYC document verification software in 2026: 8 tools compared](https://www.docsumo.com/blog/best-kyc-document-verification-software)
- [The best handwriting recognition software in 2026: 8 tools compared](https://www.docsumo.com/blog/best-handwriting-recognition-software)

## Keep reading

- [The best OCR models in 2026: open-source and vision-language models compared on published benchmarks](https://www.docsumo.com/blog/best-ocr-models): For AI and engineering teams choosing an OCR model: the published scores as of September 2026, what each license allows, and what the scores miss.
- [Text annotation: types, process and tools for training NLP models](https://www.docsumo.com/blog/text-annotation): For data science, ML and operations teams building or buying language models: what text annotation is, the annotation types that matter, how to run a project with consistent labels, and how document AI uses it.
- [Document annotation: what it is, types and best practices](https://www.docsumo.com/blog/document-annotation): For teams training or tuning document AI models: what document annotation is, the kinds of labels you need, how to write guidelines and measure label quality, and how much annotation modern platforms still require.

[More on OCR & IDP](https://www.docsumo.com/blog#posts-idp)

## Next steps

- [Book a demo](https://www.docsumo.com/book-a-demo)
- [Get a free trial (1,000 pages)](https://www.docsumo.com/free-trial)
- [Contact sales or support](https://www.docsumo.com/contact)
- [agents.md](https://www.docsumo.com/agents.md): connect Docsumo to your AI agent with Docsumo MCP
- [llms.txt](https://www.docsumo.com/llms.txt) lists the key pages; add .md to any page's URL for its Markdown copy
