Embedding models and their APIs: an overview

Within modern software architecture for artificial intelligence, embedding models are the invisible but essential link. Where large language models (LLMs) excel at generating human answers, embedding models convert unstructured text into compact numerical vectors. These vectors capture the meaning, contextual nuances and mutual relationships of information in a mathematical way. In this extensive guide we analyze how embedding models work, compare hosted APIs and open-source alternatives, and discuss practical considerations for processing Dutch-language documents.

What are embedding models and how do they fit into the AI ecosystem?

An embedding model is a neural network trained to convert text into a multidimensional sequence of numbers: a vector. Within this vector space, concepts with comparable content or intent lie close together, regardless of exact wording. This principle is the foundation of semantic search, automatic document organization and Retrieval-Augmented Generation (RAG).

If you want to understand how vectors and search relate to the broader infrastructure, you can view the full AI ecosystem mapped out to see exactly where these building blocks sit.

In a standard RAG pipeline, the embedding model processes two kinds of input: the documents indexed into the knowledge base, and the queries users ask. Because both converted texts are placed in the same vector space, a system can locate the most relevant document passages at high speed through distance or angle calculations. This keeps search systems from stumbling over typos, synonyms or divergent sentence structures.

Assessment criteria and measurement methods for embedding quality

Comparing embedding models requires a structured approach. The top-scoring model on a general leaderboard is not automatically the optimal choice for a specific business application. Developers and architects assess embedding models on a number of technical parameters:

• Dimensionality: The length of the generated vector (for example 384, 768, 1536 or 3072 numbers). Higher dimensions offer more room to store subtle nuances, but require more working memory and compute during search.
• Maximum context length: The number of tokens a model can read in one pass. Classic models handle at most 512 tokens, while modern variants can summarize entire chapters of 8,192 to 32,768 tokens into a single vector.
• Matryoshka Representation Learning (MRL): An advanced architecture that lets a long vector (such as 1536 dimensions) be truncated without damage to, say, 256 or 512 dimensions. This enables flexible storage savings without retraining the model.
• Assessment benchmarks: Standardized evaluation sets such as MTEB (Massive Text Embedding Benchmark) and AIRBench measure performance across diverse tasks such as search, clustering, reordering and text classification.

Hosted commercial APIs: OpenAI, Cohere, Google and Voyage AI

For many organizations a commercial API offers the fastest route to production. Managed infrastructures take care of hardware, GPU clusters and scalability entirely. Users pay based on the number of tokens processed.

OpenAI Text Embedding API

OpenAI offers, with text-embedding-3-small and text-embedding-3-large two widely used models. Both support Matryoshka embeddings, meaning developers can specify through an API parameter (dimensions) how large the output vector should be. At very low cost per million tokens, these models provide a solid basis for general search and RAG applications.

Cohere Embed v3

Cohere stands out with its embed-multilingual-v3.0 model, explicitly designed for business applications. An important characteristic is that developers have to indicate on the API call what the text is for, through the parameter input_type (such as search_document or search_query). This asymmetric approach raises search accuracy considerably. Cohere also directly supports compressed output formats (such as int8 and binary) for efficient storage.

Google Vertex AI Embeddings

Through Vertex AI, Google offers various Gecko and multimodal embedding models. These models are integrated seamlessly into the Google Cloud Platform. A particular advantage of Google's multimodal variants is the ability to place text, images and video in one shared vector space, enabling advanced visual and cross-media search.

Voyage AI

Voyage AI positions itself as a specialist provider with tailored models for specific sectors. It offers specialized models for financial news (voyage-finance-2), legal documentation (voyage-law-2) and source code (voyage-code-2). On domain-specific benchmarks these models often show better results than generic enterprise APIs.

Open-source embedding models you can run locally

Alongside the commercial offering, the world of open-source embedding models has grown enormously. Self-hosting offers full control over data, eliminates variable token costs and prevents dependence on external API availability.

Models from the BGE series (BAAI General Embedding), Microsoft's E5 family and the multilingual variants from Qwen perform comparably to or better than many commercial APIs on public benchmarks. The BGE-M3 variant is known for its versatility: it supports multilingualism, long documents (up to 8,192 tokens) and combines dense and sparse search methods within a single model.

For organizations that want to retain full control over their data, we recommend reading about the best-known open-source models at a glance to determine the right balance between licensing and performance.

Running these models locally or in your own cloud environment is usually done with specialized inference engines such as Text Embeddings Inference (TEI) from Hugging Face, ONNX Runtime or vLLM. These engines optimize batch processing and GPU memory usage, allowing thousands of requests per second to be handled locally.

The challenges of Dutch and multilingual text processing

A common bottleneck when implementing embedding models in the Dutch market is that many models are trained primarily on English-language documents. This shows up in two specific technical problems: tokenizer inefficiency and semantic drift.

Tokenizer inefficiency with Dutch compound words

Tokenizers from English-language models often do not know Dutch compound nouns (such as 'arbeidsongeschiktheidsverzekering' or 'klimaatveranderingsbeleid'). The tokenizer breaks such a word into countless separate subword fragments of two or three letters. As a result, Dutch text quickly consumes 40% to 80% more tokens than identical English text. This causes the context window to be exceeded sooner and, with paid APIs, leads to higher costs per page.

Semantic alignment and domain jargon

In multilingual models (such as Cohere Embed Multilingual or BGE-M3), the vector space is built so that a Dutch query can lie close to the English answer. With specifically Dutch domain jargon — think of legal terms from the Dutch Civil Code, municipal ordinances or tax regulations — the distance between related Dutch concepts can be too great in a poorly tuned multilingual model. Testing models on a representative dataset of your own Dutch documents is therefore a mandatory step.

Vector space, indexing and integration with vector databases

Once an embedding model converts text into a vector, that vector has to be made efficiently searchable. Computing exact similarity (such as the cosine angle or dot product) between a query and millions of stored vectors takes too long with traditional methods. Vector databases therefore use approximate nearest neighbor (ANN) search algorithms.

Indexing techniques: HNSW versus IVF

The two most widely used indexing structures in modern vector databases are:

• HNSW (Hierarchical Navigable Small World): Builds a multilayer network of vectors. This delivers extremely fast search times and high accuracy, but requires the full index to sit in the server's working memory (RAM).
• IVF (Inverted File Index): Divides the vector space into clusters and searches only the nearest clusters. This is considerably more memory-efficient, but with the wrong configuration it can cause a slight drop in search accuracy (recall).

To understand how these vector representations are stored and searched efficiently at scale, see the overview of vector databases compared for a detailed analysis of indexing techniques.

Two-stage retrieval: embedding models versus rerankers

In practice, a purely embedding-based query sometimes falls short on highly specific or complex questions. Embedding models work as so-called bi-encoders: the query and the document are converted into vectors separately. As a result, the model misses the direct interaction between specific words in the query and the retrieved passage.

To solve this, professional AI architectures almost always set up a two-stage retrieval system:

1. First stage (bi-encoder): The embedding model performs a fast, broad selection in the vector database and retrieves the top 50 or top 100 most relevant documents from a corpus of millions of passages.
2. Second stage (cross-encoder / reranker): A reranking model analyzes the query and the top 50 passages together in a single pass. This model assesses the exact contextual fit and reorders the results before they are sent to the language model.

If you want to determine when a purely vector-based query suffices and when a second refinement step is necessary, we refer to the article on embedding, reranker or hybrid retrieval in the knowledge center.

Quantization, compression and memory optimization

When storing millions of vectors, the memory requirement of the database can climb quickly. A single vector of 1536 dimensions stored in standard 32-bit floating-point numbers (FP32) requires 6,144 bytes of storage. With 10 million documents this means more than 60 GB of pure working memory, quite apart from the index structures.

Saving memory through quantization

Two forms of compression are applied to push these storage costs down:

• Scalar Quantization (SQ8): Converts the 32-bit numbers in the vector into 8-bit integers (INT8). This reduces memory usage by 75% with negligible loss of search quality (less than a 1% drop in nDCG@10).
• Binary Quantization (BQ): Converts each number in the vector into a single bit (0 or 1). This yields a memory reduction of 97% and enables extremely fast Hamming distance calculations on the CPU. This method works particularly well with models trained specifically for binary output, such as Cohere Embed v3 or OpenAI through MRL.

Governance, GDPR and data residency in vector processing

A persistent misconception within IT organizations is that a vector is an anonymized representation of a text. Research into so-called vector inversion attacks has shown that under certain conditions it is possible, with a specialized decoder model, to reconstruct much of the original text (including names, addresses and medical data) from a vector.

When confidential customer information or personal data is converted into embeddings through an external US cloud API, that data transfer falls directly under the strict rules of the General Data Protection Regulation (GDPR). Organizations in the public sector, legal services and healthcare therefore increasingly choose to run embedding models locally or house them in European data centers with data residency guarantees.

Comparative overview of embedding models

A selection model: which embedding model fits your application?

Choosing the right embedding model is a trade-off between privacy, processing speed, cost and substantive quality. Use the decision path below as a guide for your IT architecture:

1. Getting started quickly at low upfront cost: Choose a managed API such as OpenAI text-embedding-3-small. This offers an excellent price-performance ratio for general business applications, provided the privacy terms are acceptable.
2. Strict GDPR requirements and privacy-sensitive data: Host an open-source model such as BGE-M3 or Multilingual-E5 on your own server or within a European cloud provider using a TEI container.
3. Multilingual queries and asymmetric RAG: Use Cohere embed-multilingual-v3.0 because of its explicit separation between queries and documents.
4. Large-scale archives with a limited RAM budget: Combine a model that supports Matryoshka embeddings with 8-bit or binary vector quantization in the vector database to minimize storage costs.

By decoupling the embedding layer within the software architecture cleanly from the generative language model and the database, it stays possible to switch to newer or better-performing embedding models in future without rebuilding the entire application.