How to generate multiple queries to retrieve data

Prerequisites

This guide assumes familiarity with the following concepts:

• Vector stores
• Retrievers
• Retrieval-augmented generation (RAG)

Distance-based vector database retrieval embeds (represents) queries in high-dimensional space and finds similar embedded documents based on “distance”. But retrieval may produce different results with subtle changes in query wording or if the embeddings do not capture the semantics of the data well. Prompt engineering / tuning is sometimes done to manually address these problems, but can be tedious.

The MultiQueryRetriever automates the process of prompt tuning by using an LLM to generate multiple queries from different perspectives for a given user input query. For each query, it retrieves a set of relevant documents and takes the unique union across all queries to get a larger set of potentially relevant documents. By generating multiple perspectives on the same question, the MultiQueryRetriever can help overcome some of the limitations of the distance-based retrieval and get a richer set of results.

Get started

tip

See this section for general instructions on installing integration packages.

npm

npm i @langchain/anthropic @langchain/cohere @langchain/core

yarn

yarn add @langchain/anthropic @langchain/cohere @langchain/core

pnpm

pnpm add @langchain/anthropic @langchain/cohere @langchain/core

import { MemoryVectorStore } from "langchain/vectorstores/memory";
import { CohereEmbeddings } from "@langchain/cohere";
import { MultiQueryRetriever } from "langchain/retrievers/multi_query";
import { ChatAnthropic } from "@langchain/anthropic";

const embeddings = new CohereEmbeddings({
  model: "embed-english-v3.0",
});

const vectorstore = await MemoryVectorStore.fromTexts(
  [
    "Buildings are made out of brick",
    "Buildings are made out of wood",
    "Buildings are made out of stone",
    "Cars are made out of metal",
    "Cars are made out of plastic",
    "mitochondria is the powerhouse of the cell",
    "mitochondria is made of lipids",
  ],
  [{ id: 1 }, { id: 2 }, { id: 3 }, { id: 4 }, { id: 5 }],
  embeddings
);

const model = new ChatAnthropic({
  model: "claude-3-sonnet-20240229",
});

const retriever = MultiQueryRetriever.fromLLM({
  llm: model,
  retriever: vectorstore.asRetriever(),
});

const query = "What are mitochondria made of?";
const retrievedDocs = await retriever.invoke(query);

/*
  Generated queries: What are the components of mitochondria?,What substances comprise the mitochondria organelle?  ,What is the molecular composition of mitochondria?
*/

console.log(retrievedDocs);

[
  Document {
    pageContent: 'mitochondria is made of lipids',
    metadata: {},
    id: undefined
  },
  Document {
    pageContent: 'mitochondria is the powerhouse of the cell',
    metadata: {},
    id: undefined
  },
  Document {
    pageContent: 'Buildings are made out of stone',
    metadata: { id: 3 },
    id: undefined
  },
  Document {
    pageContent: 'Buildings are made out of brick',
    metadata: { id: 1 },
    id: undefined
  },
  Document {
    pageContent: 'Cars are made out of metal',
    metadata: { id: 4 },
    id: undefined
  }
]

Customization

You can also supply a custom prompt to tune what types of questions are generated. This prompt must return queries separated by commas.

import { pull } from "langchain/hub";
import { BaseOutputParser } from "@langchain/core/output_parsers";
import { PromptTemplate } from "@langchain/core/prompts";

// Default prompt is available at: https://smith.langchain.com/hub/jacob/multi-vector-retriever-german
const customPrompt: PromptTemplate = await pull(
  "jacob/multi-vector-retriever-german"
);

/*
You are an AI language model assistant. Your task is
to generate {queryCount} different versions of the given user
question to retrieve relevant documents from a vector database.
By generating multiple perspectives on the user question,
your goal is to help the user overcome some of the limitations
of distance-based similarity search.

All questions should be in German.

Provide these alternative questions separated by newlines between XML tags. For example:

<questions>
Question 1
Question 2
Question 3
</questions>

Original question: {question}
*/

const germanVectorstore = await MemoryVectorStore.fromTexts(
  [
    "Gebäude werden aus Ziegelsteinen hergestellt",
    "Gebäude werden aus Holz hergestellt",
    "Gebäude werden aus Stein hergestellt",
    "Autos werden aus Metall hergestellt",
    "Autos werden aus Kunststoff hergestellt",
    "Mitochondrien sind die Energiekraftwerke der Zelle",
    "Mitochondrien bestehen aus Lipiden",
  ],
  [{ id: 1 }, { id: 2 }, { id: 3 }, { id: 4 }, { id: 5 }],
  embeddings
);
const anthropicModel = new ChatAnthropic({
  model: "claude-3-sonnet-20240229",
});

const germanRetriever = MultiQueryRetriever.fromLLM({
  retriever: germanVectorstore.asRetriever(),
  prompt: customPrompt,
  llm: anthropicModel,
});

const newQuery = "What are mitochondria made of?";
const germanRetrievedDocs = await germanRetriever.invoke(newQuery);

/*
  Generated queries: Was besteht ein Mitochondrium?,Aus welchen Komponenten setzt sich ein Mitochondrium zusammen?  ,Welche Moleküle finden sich in einem Mitochondrium?
*/

console.log(germanRetrievedDocs);

[
  Document {
    pageContent: 'Mitochondrien bestehen aus Lipiden',
    metadata: {},
    id: undefined
  },
  Document {
    pageContent: 'Mitochondrien sind die Energiekraftwerke der Zelle',
    metadata: {},
    id: undefined
  },
  Document {
    pageContent: 'Autos werden aus Metall hergestellt',
    metadata: { id: 4 },
    id: undefined
  },
  Document {
    pageContent: 'Gebäude werden aus Stein hergestellt',
    metadata: { id: 3 },
    id: undefined
  },
  Document {
    pageContent: 'Autos werden aus Kunststoff hergestellt',
    metadata: { id: 5 },
    id: undefined
  },
  Document {
    pageContent: 'Gebäude werden aus Ziegelsteinen hergestellt',
    metadata: { id: 1 },
    id: undefined
  }
]

Next steps

You’ve now learned how to use the MultiQueryRetriever to query a vector store with automatically generated queries.

See the individual sections for deeper dives on specific retrievers, the broader tutorial on RAG, or this section to learn how to create your own custom retriever over any data source.