How to use FlagEmbedding: install, embed text and rerank

What is FlagEmbedding and what does it do?

FlagEmbedding is the FlagOpen project's retrieval toolkit, also called BGE (the README describes it as a one-stop retrieval toolkit for search and RAG). Embedding models convert text into vectors so software can find passages with similar meaning, even when the exact words differ.

The toolkit also includes reranker models. The README recommends using them to re-rank the top-k documents returned by an embedding model, and it describes cross-encoder rerankers such as BAAI/bge-reranker-base and BAAI/bge-reranker-large.

• Embedding models: turn text into vectors for similarity search.
• Reranker models: re-order a short list of candidate results.
• Fine-tuning code: adapt the models to your own data, including a script to mine hard negatives.

Check the prerequisites before you start

The only prerequisite listed for this package is Python. The models are run locally or on hosted compute, so you need enough compute to load and run them. The source does not list specific hardware, so check the model card of the model you choose on Hugging Face (the BGE models are published in the BAAI collection there).

• Python installed on your machine.
• Compute capable of running machine-learning models, locally or through hosted inference.
• Basic familiarity with running Python scripts from a terminal.

How to get FlagEmbedding onto your machine

The README's installation step starts by cloning the repository from GitHub. Run the command below in a terminal, then open the cloned folder to find the research projects and example code.

The README's installation section and the BGE documentation at bge-model.com cover the remaining setup steps. Follow those for the exact packaging and installation commands, since this guide only uses the command the source shows.

Choose the right BGE model for your task

The README lists many models, and the right one depends on your job. For multilingual work, the README describes BGE-M3 as supporting 100+ languages and inputs up to 8192 tokens, with dense, lexical and multi-vector retrieval in one model. For English-focused retrieval with in-context examples, it lists bge-en-icl. For a multilingual model based on gemma-2-9b, it lists bge-multilingual-gemma2.

For re-ranking, the source names bge-reranker-base, bge-reranker-large and the lighter bge-reranker-v2.5-gemma2-lightweight, which supports token compression and layerwise operations to save resources.

• Multilingual retrieval: BGE-M3 or bge-multilingual-gemma2.
• Re-ranking top results: bge-reranker-base, bge-reranker-large or the lightweight v2.5 reranker.
• Multimodal search over images and text: BGE-VL, which the README says is released under the MIT license.

Build a semantic search or RAG pipeline

A typical pipeline embeds your documents once, stores the vectors, embeds each incoming query, and returns the nearest documents. For RAG, you pass those documents to an LLM as context. Adding a reranker to the top results is an optional step the README recommends for better ordering.

The README notes that BGE models are integrated into LangChain. Check the repository's tutorials folder and the BGE documentation for example code, since the source here is truncated and does not show full pipeline code.

• Embed and store your documents.
• Embed the user's query with the same model.
• Retrieve the nearest documents.
• Optionally re-rank them with a reranker.
• Pass the best documents to your LLM for RAG.

Fine-tune a model on your own data

The repository includes fine-tuning code under research/baai_general_embedding. According to the README, it includes a script to mine hard negatives and supports adding an instruction during fine-tuning. Fine-tuning is useful when general models miss the vocabulary or relevance rules of your domain.

Fine-tuning needs labelled or constructed query-passage pairs and compute. Read the README in that folder before running any training script.