RAG using LangChain : Part 3- Vector Stores and Retrievers

Welcome to the third article in this series of RAG where we explore the different components in the Retrieval process in RAG. So far we have looked into how to load unstructured data, transform it by means of text splitters and embed it using various text embedding models.

In this article, we will understand the functionalities of Vector Stores and Retrievers which are very important components in the Retrieval process.

So, what are Vector Stores?

Vector stores are essentially databases that are designed for a specific kind of data format — vector embeddings. They specialize in storing these embeddings very efficiently. Vector stores enable fast retrieval based on similarity searches.

Essentially the unstructured data is loaded, transformed and embedded into vectors and stored in a vector store. During query time, the unstructured query is embedded and the most similar embedding vectors with respect to the embedded query is retrieved. Vector stores takes care of storing the embedded data and performing vector search.

There are a lot of free, open-source vector stores namely, ChromaDB, FAISS, Qdrant, Lance, etc. Let’s look into the functionality of some of these.

Import all necessary dependencies.

from langchain_community.document_loaders import WikipediaLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import HuggingFaceBgeEmbeddings

Step 1: Document Loading- Let’s use WikipediaLoader as the Document Loader.

loader = WikipediaLoader(query = 'Elon Musk', load_max_docs=5)
documents = loader.load()
documents

#[Document(page_content='Elon Reeve Musk (; EE-lon; born June 28, 1971) is a businessman and investor. He is the founder, chairman, CEO, and CTO of SpaceX; angel investor, CEO, product architect, and former chairman of Tesla, Inc.; owner, executive chairman, and CTO of X Corp.; founder of the Boring Company and xAI; co-founder of Neuralink and OpenAI; and president of the Musk Foundation. He is one of the wealthiest people in the world, with an estimated net worth of US$190 billion as of March 2024, according to the Bloomberg Billionaires Index, and $195 billion according to Forbes, primarily from his ownership stakes in Tesla and SpaceX.A member of the wealthy South African Musk family, Elon was born in Pretoria and briefly attended the University of Pretoria before immigrating to Canada at age 18, acquiring citizenship through his Canadian-born mother. Two years later, he matriculated at Queen\'s University at Kingston in Canada. Musk later transferred to the University of Pennsylvania, and received bachelor\'s degrees in economics and physics. He moved to California in 1995 to attend Stanford University, but dropped out after two days and, with his brother Kimbal, co-founded online city guide software company Zip2. The startup was acquired by Compaq for $307 million in 1999, and that same year Musk co-founded X.com, a direct bank. X.com merged with Confinity in 2000 to form PayPal.\nIn October 2002, eBay acquired PayPal for $1.5 billion, and that same year, with $100 million of the money he made, Musk founded SpaceX, a spaceflight services company. In 2004, he became an early investor in electric vehicle manufacturer Tesla Motors, Inc. (now Tesla, Inc.). He became its chairman and product architect, assuming the position of CEO in 2008. In 2006, Musk helped create SolarCity, a solar-energy company that was acquired by Tesla in 2016 and became Tesla Energy. In 2013, he proposed a hyperloop high-speed vactrain transportation system. In 2015, he co-founded OpenAI, a nonprofit artificial intelligence research company. The following year, Musk co-founded Neuralink—a neurotechnology company developing brain–computer interfaces—and the Boring Company, a tunnel construction company. In 2022, he acquired Twitter for $44 billion. He subsequently merged the company into newly created X Corp. and rebranded the service as X the following year. In March 2023, he founded xAI, an artificial intelligence company.\nMusk has expressed views that have made him a polarizing figure. He has been criticized for making unscientific and misleading statements, including COVID-19 misinformation and antisemitic conspiracy theories. His ownership of Twitter has been similarly controversial, being marked by the laying off of a large number of employees, an increase in hate speech and misinformation and disinformation on the website, as well as changes to Twitter Blue verification. In 2018, the U.S. Securities and Exchange Commission (SEC) sued him, alleging that he had falsely announced that he had secured funding for a private takeover of Tesla. To settle the case, Musk stepped down as the chairman of Tesla and paid a $20 million fine.\n\n\n== Early life and education ==\n\n\n=== Childhood and family ===\n\nElon Reeve Musk was born on June 28, 1971, in Pretoria, South Africa\'s administrative capital. He is of British and Pennsylvania Dutch ancestry. His mother, Maye Musk (née Haldeman), is a model and dietitian born in Saskatchewan, Canada, and raised in South Africa. His father, Errol Musk, is a South African electromechanical engineer, pilot, sailor, consultant, emerald dealer, and property developer, who partly owned a rental lodge at the Timbavati Private Nature Reserve. Elon has a younger brother, Kimbal, and a younger sister, Tosca.The family was wealthy during Elon\'s youth. Despite both Musk and Errol previously stating that Errol was a part owner of a Zambian emerald mine, in 2023, Errol recounted that the deal he made was to receive "a portion of the emeralds produced at three sma', metadata={'title': 'Elon Musk', 'summary': "Elon Reeve Musk (; EE-lon; born June 28, 1971) is a businessman and investor. He is the founder, chairman, CEO, and CTO of SpaceX; angel investor, CEO, product architect, and former chairman of Tesla, Inc.; owner, executive chairman, and CTO of X Corp.; founder of the Boring Company and xAI; co-founder of Neuralink and OpenAI; and president of the Musk Foundation. He is one of the wealthiest people in the world, with an estimated net worth of US$190 billion as of March 2024, according to the Bloomberg Billionaires Index, and $195 billion according to Forbes, primarily from his ownership stakes in Tesla and SpaceX.A member of the wealthy South African Musk family, Elon was born in Pretoria and briefly attended the University of Pretoria before immigrating to Canada at age 18, acquiring citizenship through his Canadian-born mother. Two years later, he matriculated at Queen's University at Kingston in Canada. Musk later transferred to the University of Pennsylvania, and received bachelor's degrees in economics and physics. He moved to California in 1995 to attend Stanford University, but dropped out after two days and, with his brother Kimbal, co-founded online city guide software company Zip2. The startup was acquired by Compaq for $307 million in 1999, and that same year Musk co-founded X.com, a direct bank. X.com merged with Confinity in 2000 to form PayPal.\nIn October 2002, eBay acquired PayPal for $1.5 billion, and that same year, with $100 million of the money he made, Musk founded SpaceX, a spaceflight services company. In 2004, he became an early investor in electric vehicle manufacturer Tesla Motors, Inc. (now Tesla, Inc.). He became its chairman and product architect, assuming the position of CEO in 2008. In 2006, Musk helped create SolarCity, a solar-energy company that was acquired by Tesla in 2016 and became Tesla Energy. In 2013, he proposed a hyperloop high-speed vactrain transportation system. In 2015, he co-founded OpenAI, a nonprofit artificial intelligence research company. The following year, Musk co-founded Neuralink—a neurotechnology company developing brain–computer interfaces—and the Boring Company, a tunnel construction company. In 2022, he acquired Twitter for $44 billion. He subsequently merged the company into newly created X Corp. and rebranded the service as X the following year. In March 2023, he founded xAI, an artificial intelligence company.\nMusk has expressed views that have made him a polarizing figure. He has been criticized for making unscientific and misleading statements, including COVID-19 misinformation and antisemitic conspiracy theories. His ownership of Twitter has been similarly controversial, being marked by the laying off of a large number of employees, an increase in hate speech and misinformation and disinformation on the website, as well as changes to Twitter Blue verification. In 2018, the U.S. Securities and Exchange Commission (SEC) sued him, alleging that he had falsely announced that he had secured funding for a private takeover of Tesla. To settle the case, Musk stepped down as the chairman of Tesla and paid a $20 million fine.", 'source': 'https://en.wikipedia.org/wiki/Elon_Musk'}),
.....]

Step 2: Text Splitting- We will use RecursiveCharacterTextSplitter as the text splitter.

text_splitter = RecursiveCharacterTextSplitter(chunk_size = 400, chunk_overlap = 100)
docs = text_splitter.split_documents(documents=documents)
print(docs)

#[Document(page_content='Elon Reeve Musk (; EE-lon; born June 28, 1971) is a businessman and investor. He is the founder, chairman, CEO, and CTO of SpaceX; angel investor, CEO, product architect, and former chairman of Tesla, Inc.; owner, executive chairman, and CTO of X Corp.; founder of the Boring Company and xAI; co-founder of Neuralink and OpenAI; and president of the Musk Foundation. He is one of the wealthiest', metadata={'title': 'Elon Musk', 'summary': "Elon Reeve Musk (; EE-lon; born June 28, 1971) is a businessman and investor. He is the founder, chairman, CEO, and CTO of SpaceX; angel investor, CEO, product architect, and former chairman of Tesla, Inc.; owner, executive chairman, and CTO of X Corp.; founder of the Boring Company and xAI; co-founder of Neuralink and OpenAI; and president of the Musk Foundation. He is one of the wealthiest people in the world, with an estimated net worth of US$190 billion as of March 2024, according to the Bloomberg Billionaires Index, and $195 billion according to Forbes, primarily from his ownership stakes in Tesla and SpaceX.A member of the wealthy South African Musk family, Elon was born in Pretoria and briefly attended the University of Pretoria before immigrating to Canada at age 18, acquiring citizenship through his Canadian-born mother. Two years later, he matriculated at Queen's University at Kingston in Canada. Musk later transferred to the University of Pennsylvania, and received bachelor's degrees in economics and physics. He moved to California in 1995 to attend Stanford University, but dropped out after two days and, with his brother Kimbal, co-founded online city guide software company Zip2. The startup was acquired by Compaq for $307 million in 1999, and that same year Musk co-founded X.com, a direct bank. X.com merged with Confinity in 2000 to form PayPal.\nIn October 2002, eBay acquired PayPal for $1.5 billion, and that same year, with $100 million of the money he made, Musk founded SpaceX, a spaceflight services company. In 2004, he became an early investor in electric vehicle manufacturer Tesla Motors, Inc. (now Tesla, Inc.). He became its chairman and product architect, assuming the position of CEO in 2008. In 2006, Musk helped create SolarCity, a solar-energy company that was acquired by Tesla in 2016 and became Tesla Energy. In 2013, he proposed a hyperloop high-speed vactrain transportation system. In 2015, he co-founded OpenAI, a nonprofit artificial intelligence research company. The following year, Musk co-founded Neuralink—a neurotechnology company developing brain–computer interfaces—and the Boring Company, a tunnel construction company. In 2022, he acquired Twitter for $44 billion. He subsequently merged the company into newly created X Corp. and rebranded the service as X the following year. In March 2023, he founded xAI, an artificial intelligence company.\nMusk has expressed views that have made him a polarizing figure. He has been criticized for making unscientific and misleading statements, including COVID-19 misinformation and antisemitic conspiracy theories. His ownership of Twitter has been similarly controversial, being marked by the laying off of a large number of employees, an increase in hate speech and misinformation and disinformation on the website, as well as changes to Twitter Blue verification. In 2018, the U.S. Securities and Exchange Commission (SEC) sued him, alleging that he had falsely announced that he had secured funding for a private takeover of Tesla. To settle the case, Musk stepped down as the chairman of Tesla and paid a $20 million fine.", 'source': 'https://en.wikipedia.org/wiki/Elon_Musk'}),
....]

Step 3: Embedding Function- For the embedding function, we will use HuggingFace BGE embedding model and let’s create a query.

model_name = "BAAI/bge-large-en-v1.5"
model_kwargs = {'device':'cpu'}
encode_kwargs = {'normalize_embeddings':True}

embedding_function = HuggingFaceBgeEmbeddings(
    model_name = model_name,
    model_kwargs = model_kwargs,
    encode_kwargs = encode_kwargs
)

query = "Who is elon musk's father?"

FAISS

Let’s look into the functionalities of FAISS vector database, which makes use of the Facebook AI Similarity Search (FAISS) library.

pip install faiss-cpu

from langchain_community.vectorstores import FAISS
db = FAISS.from_documents(
    docs,
    embedding_function
)

Next, we can query the vector database. We can either do similarity search or similarity search with vector.

# Similarity search with query
matched_docs = db.similarity_search(query = query, k = 5)
matched_docs

#[Document(page_content="Elon Musk's paternal great-grandmother was a Dutchwoman descended from the Dutch Free Burghers, while one of his maternal great-grandparents came from Switzerland. His paternal grandmother was English from Liverpool and his paternal grandfather Walter Henry J. Musk was South African. Elon Musk's father, Errol Musk, is a South African former electrical and mechanical engineer consultant and", metadata={'title': 'Musk family', 'summary': 'The Musk family is a wealthy family of South African origin that is largely active in the United States and Canada. The Musks are of English, Anglo-Canadian, Pennsylvania Dutch, and Swiss descent. The family is known for its entrepreneurial endeavours. Elon Musk was formerly the wealthiest person in the world, with an estimated net worth of US$232 billion as of December 2023, according to the Bloomberg Billionaires Index.', 'source': 'https://en.wikipedia.org/wiki/Musk_family'}),
....]

# Similarity search with query vector
embedding_vector = embedding_function.embed_query(query)
matched_docs = db.similarity_search_by_vector(embedding_vector)
matched_docs

#[Document(page_content="Elon Musk's paternal great-grandmother was a Dutchwoman descended from the Dutch Free Burghers, while one of his maternal great-grandparents came from Switzerland. His paternal grandmother was English from Liverpool and his paternal grandfather Walter Henry J. Musk was South African. Elon Musk's father, Errol Musk, is a South African former electrical and mechanical engineer consultant and", metadata={'title': 'Musk family', 'summary': 'The Musk family is a wealthy family of South African origin that is largely active in the United States and Canada. The Musks are of English, Anglo-Canadian, Pennsylvania Dutch, and Swiss descent. The family is known for its entrepreneurial endeavours. Elon Musk was formerly the wealthiest person in the world, with an estimated net worth of US$232 billion as of December 2023, according to the Bloomberg Billionaires Index.', 'source': 'https://en.wikipedia.org/wiki/Musk_family'}),
....]

Since the embedding function is same for both, the result will be same.

Most of the times, we won’t necessarily use an in-memory database. Let’s create another open-source database.

ChromaDB

ChromaDB can be used as both an in-memory database and a backend with persistent storage options.

!pip install chromadb

from langchain.vectorstores import Chroma
db = Chroma.from_documents(docs, embedding_function, persist_directory="output/elon_muskdb")

# the persist_directory parameter saves the database in the specified path

Loading the database

loaded_db = Chroma(persist_directory = "output/elon_muskdb", embedding_function = embedding_function)

Similarly, here also we can do similarity search as well as similarity search with vector.

# Similarity search with query
matched_docs = loaded_db.similarity_search(query = query, k = 5)
matched_docs

#[Document(page_content="Elon Musk's paternal great-grandmother was a Dutchwoman descended from the Dutch Free Burghers, while one of his maternal great-grandparents came from Switzerland. His paternal grandmother was English from Liverpool and his paternal grandfather Walter Henry J. Musk was South African. Elon Musk's father, Errol Musk, is a South African former electrical and mechanical engineer consultant and", metadata={'source': 'https://en.wikipedia.org/wiki/Musk_family', 'summary': 'The Musk family is a wealthy family of South African origin that is largely active in the United States and Canada. The Musks are of English, Anglo-Canadian, Pennsylvania Dutch, and Swiss descent. The family is known for its entrepreneurial endeavours. Elon Musk was formerly the wealthiest person in the world, with an estimated net worth of US$232 billion as of December 2023, according to the Bloomberg Billionaires Index.', 'title': 'Musk family'}),
....]

# Similarity search with query vector
embedding_vector = embedding_function.embed_query(query)
matched_docs = loaded_db.similarity_search_by_vector(embedding_vector)
matched_docs

#[Document(page_content="Elon Musk's paternal great-grandmother was a Dutchwoman descended from the Dutch Free Burghers, while one of his maternal great-grandparents came from Switzerland. His paternal grandmother was English from Liverpool and his paternal grandfather Walter Henry J. Musk was South African. Elon Musk's father, Errol Musk, is a South African former electrical and mechanical engineer consultant and", metadata={'title': 'Musk family', 'summary': 'The Musk family is a wealthy family of South African origin that is largely active in the United States and Canada. The Musks are of English, Anglo-Canadian, Pennsylvania Dutch, and Swiss descent. The family is known for its entrepreneurial endeavours. Elon Musk was formerly the wealthiest person in the world, with an estimated net worth of US$232 billion as of December 2023, according to the Bloomberg Billionaires Index.', 'source': 'https://en.wikipedia.org/wiki/Musk_family'}),
....]

Retrievers

A retriever is an interface that returns documents given an unstructured query. It does not have to store documents like Vector store. Retrievers accept a string query as an input and return a list of Documents as an output. Let look into some basic retrievers in this article. We will cover more of Retrievers in the next one!

Vector Store-backed retriever

It uses a Vector store to retrieve documents. Let’s construct a retriever using the existing ChromaDB Vector store that we have.

retriever = db.as_retriever()
retriever

#VectorStoreRetriever(tags=['Chroma', 'HuggingFaceBgeEmbeddings'], vectorstore=<langchain_community.vectorstores.chroma.Chroma object at 0x000001EF7F17E080>)

Querying the retriever

query = "Who is elon musk's father?"
matched_docs = retriever.get_relevant_documents(query = query)
matched_docs

# [Document(page_content="Elon Musk's paternal great-grandmother was a Dutchwoman descended from the Dutch Free Burghers, while one of his maternal great-grandparents came from Switzerland. His paternal grandmother was English from Liverpool and his paternal grandfather Walter Henry J. Musk was South African. Elon Musk's father, Errol Musk, is a South African former electrical and mechanical engineer consultant and", metadata={'source': 'https://en.wikipedia.org/wiki/Musk_family', 'summary': 'The Musk family is a wealthy family of South African origin that is largely active in the United States and Canada. The Musks are of English, Anglo-Canadian, Pennsylvania Dutch, and Swiss descent. The family is known for its entrepreneurial endeavours. Elon Musk was formerly the wealthiest person in the world, with an estimated net worth of US$232 billion as of December 2023, according to the Bloomberg Billionaires Index.', 'title': 'Musk family'}),
....]

While creating the retriever, we can also mention how the retriever should retrieve the document( like Maximum Marginal Relevance Search or Similarity Search)and how many documents to retrieve.

# Using MMR and limiting the number of retrieved documents to 1

retriever = db.as_retriever(search_type='mmr', search_kwargs={"k": 1})
matched_docs = retriever.get_relevant_documents(query=query)
matched_docs

# [Document(page_content="In 1985, Jobs departed Apple after a long power struggle with the company's board and its then-CEO, John Sculley. That same year, Jobs took some Apple employees with him to found NeXT, a computer platform development company that specialized in computers for higher-education and business markets, serving as its CEO. In 1986, he helped develop the visual effects industry by funding the computer", metadata={'source': 'https://en.wikipedia.org/wiki/Steve_Jobs', 'summary': 'Steven Paul Jobs (February 24, 1955 – October 5, 2011) was an American businessman, inventor, and investor best known for co-founding the technology giant Apple Inc. Jobs was also the founder of NeXT and chairman and majority shareholder of Pixar. He was a pioneer of the personal computer revolution of the 1970s and 1980s, along with his early business partner and fellow Apple co-founder Steve Wozniak.\nJobs was born in San Francisco in 1955 and adopted shortly afterwards. He attended Reed College in 1972 before withdrawing that same year. In 1974, he traveled through India, seeking enlightenment before later studying Zen Buddhism. He and Wozniak co-founded Apple in 1976 to further develop and sell Wozniak\'s Apple I personal computer. Together, the duo gained fame and wealth a year later with production and sale of the Apple II, one of the first highly successful mass-produced microcomputers. Jobs saw the commercial potential of the Xerox Alto in 1979, which was mouse-driven and had a graphical user interface (GUI). This led to the development of the unsuccessful Apple Lisa in 1983, followed by the breakthrough Macintosh in 1984, the first mass-produced computer with a GUI. The Macintosh launched the desktop publishing industry in 1985 with the addition of the Apple LaserWriter, the first laser printer to feature vector graphics and PostScript.\nIn 1985, Jobs departed Apple after a long power struggle with the company\'s board and its then-CEO, John Sculley. That same year, Jobs took some Apple employees with him to found NeXT, a computer platform development company that specialized in computers for higher-education and business markets, serving as its CEO. In 1986, he helped develop the visual effects industry by funding the computer graphics division of Lucasfilm that eventually spun off independently as Pixar, which produced the first 3D computer-animated feature film Toy Story (1995) and became a leading animation studio, producing over 27 films since.\nIn 1997, Jobs returned to Apple as CEO after the company\'s acquisition of NeXT. He was largely responsible for reviving Apple, which was on the verge of bankruptcy. He worked closely with British designer Jony Ive to develop a line of products and services that had larger cultural ramifications, beginning with the "Think different" advertising campaign, and leading to the iMac, iTunes, Mac OS X, Apple Store, iPod, iTunes Store, iPhone, App Store, and iPad. In 2003, Jobs was diagnosed with a pancreatic neuroendocrine tumor. He died of respiratory arrest related to the tumor in 2011, and in 2022, was posthumously awarded the Presidential Medal of Freedom.', 'title': 'Steve Jobs'})]

# Using Similarity Search.
# Also keeping a minimum similarity threshold of 0.5 and retrieved documents = 2
retriever = db.as_retriever(search_type="similarity_score_threshold", search_kwargs={"score_threshold": 0.5, "k" : 2})
matched_docs = retriever.get_relevant_documents(query=query)
matched_docs

#[Document(page_content="Elon Musk's paternal great-grandmother was a Dutchwoman descended from the Dutch Free Burghers, while one of his maternal great-grandparents came from Switzerland. His paternal grandmother was English from Liverpool and his paternal grandfather Walter Henry J. Musk was South African. Elon Musk's father, Errol Musk, is a South African former electrical and mechanical engineer consultant and", metadata={'source': 'https://en.wikipedia.org/wiki/Musk_family', 'summary': 'The Musk family is a wealthy family of South African origin that is largely active in the United States and Canada. The Musks are of English, Anglo-Canadian, Pennsylvania Dutch, and Swiss descent. The family is known for its entrepreneurial endeavours. Elon Musk was formerly the wealthiest person in the world, with an estimated net worth of US$232 billion as of December 2023, according to the Bloomberg Billionaires Index.', 'title': 'Musk family'}),
# Document(page_content="Elon Musk's paternal great-grandmother was a Dutchwoman descended from the Dutch Free Burghers, while one of his maternal great-grandparents came from Switzerland. His paternal grandmother was English from Liverpool and his paternal grandfather Walter Henry J. Musk was South African. Elon Musk's father, Errol Musk, is a South African former electrical and mechanical engineer consultant and", metadata={'source': 'https://en.wikipedia.org/wiki/Musk_family', 'summary': 'The Musk family is a wealthy family of South African origin that is largely active in the United States and Canada. The Musks are of English, Anglo-Canadian, Pennsylvania Dutch, and Swiss descent. The family is known for its entrepreneurial endeavours. Elon Musk was formerly the wealthiest person in the world, with an estimated net worth of US$232 billion as of December 2023, according to the Bloomberg Billionaires Index.', 'title': 'Musk family'})]

BM25 Retriever

It uses the BM25(Best Matching 25) ranking function ranking function to retrieve documents based on a query. It is similar to a bag-of-words approach.

!pip install rank_bm25

from langchain.retrievers import BM25Retriever
bm25_retriever = BM25Retriever.from_documents(docs)

Querying the retriever

matched_docs = bm25_retriever.get_relevant_documents('Musk')
matched_docs

[Document(page_content="October 17. Weeks before the trial was set to begin, Musk reversed course, announcing that he would move forward with the acquisition. The deal was closed on October 27, with Musk immediately becoming Twitter's new owner and CEO. Twitter was taken private and merged into a new parent company named X Corp. Musk promptly fired several top executives, including previous CEO Parag Agrawal. Musk has", metadata={'title': 'Acquisition of Twitter by Elon Musk', 'summary': 'Business magnate Elon Musk initiated an acquisition of American social media company Twitter, Inc. on April 14, 2022, and concluded it on October 27, 2022. Musk had begun buying shares of the company in January 2022, becoming its largest shareholder by April with a 9.1 percent ownership stake. Twitter invited Musk to join its board of directors, an offer he initially accepted before declining. On April 14, Musk made an unsolicited offer to purchase the company, to which Twitter\'s board responded with a "poison pill" strategy to resist a hostile takeover before unanimously accepting Musk\'s buyout offer of $44 billion on April 25. Musk stated that he planned to introduce new features to the platform, make its algorithms open-source, combat spambot accounts, and promote free speech.\nIn July, Musk announced his intention to terminate the agreement, asserting that Twitter had breached their agreement by refusing to crack down on spambot accounts. The company filed a lawsuit against Musk in the Delaware Court of Chancery shortly thereafter, with a trial scheduled for the week of October 17. Weeks before the trial was set to begin, Musk reversed course, announcing that he would move forward with the acquisition. The deal was closed on October 27, with Musk immediately becoming Twitter\'s new owner and CEO. Twitter was taken private and merged into a new parent company named X Corp. Musk promptly fired several top executives, including previous CEO Parag Agrawal. Musk has since proposed several reforms to Twitter and laid off half of the company\'s workforce. Hundreds of employees then resigned from the company after Musk issued an ultimatum demanding they commit to "extremely hardcore" work. Linda Yaccarino was appointed CEO of X Corp. In June 2023, the Twitter service was rebranded as X.\nReactions to the buyout were mixed, with praise for Musk\'s planned reforms and vision for the company, particularly his calls for greater free speech, but criticism over fears of a potential rise in misinformation and disinformation, harassment, and hate speech on the platform. Within the United States, conservatives have largely supported the acquisition, while many liberals and former Twitter employees have voiced concerns about Musk\'s intentions. Since becoming owner, Musk has faced backlash for his handling of the company and account suspensions, including the December 2022 suspensions of ten journalists.', 'source': 'https://en.wikipedia.org/wiki/Acquisition_of_Twitter_by_Elon_Musk'}),
 Document(page_content='just direct transmitters of messages from Vladimir Putin".In an email newsletter, Ian Bremmer, head of Eurasia Group, a political-risk consultancy, said that Musk told him that he had spoken directly with Putin, which Musk denied in a reply on Twitter. Musk claimed he had spoken to Putin only once 18 months prior, on a space-related subject matter. Bremmer later took to Twitter to say Musk had', metadata={'title': 'Views of Elon Musk', 'summary': 'Elon Musk is the CEO or owner of multiple companies including Tesla, SpaceX, and X Corp, and has expressed many views on a wide variety of subjects, ranging from politics to science.\nWithin the context of American politics, Musk has said he supported Barack Obama in 2008 and 2012, Hillary Clinton in 2016, and Joe Biden in 2020. In 2022, Musk said he would start voting for Republican Party candidates, with him showing support for Vivek Ramaswamy in the 2024 Republican presidential primaries. He has stated support for universal basic income, gun rights, freedom of speech, a tax on carbon emissions and opposes government subsidies. He is also a critic of short-selling.\nMusk has expressed concern about issues such as artificial intelligence (AI), climate change and population decline. He has also criticized COVID-19 lockdowns, public transportation, and labor unions. He has promoted conspiracy theories, and made controversial statements that have led to accusations of antisemitism, support of white pride and transphobia. His views on international relations, including on the China-Taiwan and Russia-Ukraine conflicts, have received mixed reactions.', 'source': 'https://en.wikipedia.org/wiki/Views_of_Elon_Musk'}),
 Document(page_content='X Corp. Musk promptly fired several top executives, including previous CEO Parag Agrawal. Musk has since proposed several reforms to Twitter and laid off half of the company\'s workforce. Hundreds of employees then resigned from the company after Musk issued an ultimatum demanding they commit to "extremely hardcore" work. Linda Yaccarino was appointed CEO of X Corp. In June 2023, the Twitter', metadata={'title': 'Acquisition of Twitter by Elon Musk', 'summary': 'Business magnate Elon Musk initiated an acquisition of American social media company Twitter, Inc. on April 14, 2022, and concluded it on October 27, 2022. Musk had begun buying shares of the company in January 2022, becoming its largest shareholder by April with a 9.1 percent ownership stake. Twitter invited Musk to join its board of directors, an offer he initially accepted before declining. On April 14, Musk made an unsolicited offer to purchase the company, to which Twitter\'s board responded with a "poison pill" strategy to resist a hostile takeover before unanimously accepting Musk\'s buyout offer of $44 billion on April 25. Musk stated that he planned to introduce new features to the platform, make its algorithms open-source, combat spambot accounts, and promote free speech.\nIn July, Musk announced his intention to terminate the agreement, asserting that Twitter had breached their agreement by refusing to crack down on spambot accounts. The company filed a lawsuit against Musk in the Delaware Court of Chancery shortly thereafter, with a trial scheduled for the week of October 17. Weeks before the trial was set to begin, Musk reversed course, announcing that he would move forward with the acquisition. The deal was closed on October 27, with Musk immediately becoming Twitter\'s new owner and CEO. Twitter was taken private and merged into a new parent company named X Corp. Musk promptly fired several top executives, including previous CEO Parag Agrawal. Musk has since proposed several reforms to Twitter and laid off half of the company\'s workforce. Hundreds of employees then resigned from the company after Musk issued an ultimatum demanding they commit to "extremely hardcore" work. Linda Yaccarino was appointed CEO of X Corp. In June 2023, the Twitter service was rebranded as X.\nReactions to the buyout were mixed, with praise for Musk\'s planned reforms and vision for the company, particularly his calls for greater free speech, but criticism over fears of a potential rise in misinformation and disinformation, harassment, and hate speech on the platform. Within the United States, conservatives have largely supported the acquisition, while many liberals and former Twitter employees have voiced concerns about Musk\'s intentions. Since becoming owner, Musk has faced backlash for his handling of the company and account suspensions, including the December 2022 suspensions of ten journalists.', 'source': 'https://en.wikipedia.org/wiki/Acquisition_of_Twitter_by_Elon_Musk'}),
 Document(page_content='Business magnate Elon Musk initiated an acquisition of American social media company Twitter, Inc. on April 14, 2022, and concluded it on October 27, 2022. Musk had begun buying shares of the company in January 2022, becoming its largest shareholder by April with a 9.1 percent ownership stake. Twitter invited Musk to join its board of directors, an offer he initially accepted before declining. On', metadata={'title': 'Acquisition of Twitter by Elon Musk', 'summary': 'Business magnate Elon Musk initiated an acquisition of American social media company Twitter, Inc. on April 14, 2022, and concluded it on October 27, 2022. Musk had begun buying shares of the company in January 2022, becoming its largest shareholder by April with a 9.1 percent ownership stake. Twitter invited Musk to join its board of directors, an offer he initially accepted before declining. On April 14, Musk made an unsolicited offer to purchase the company, to which Twitter\'s board responded with a "poison pill" strategy to resist a hostile takeover before unanimously accepting Musk\'s buyout offer of $44 billion on April 25. Musk stated that he planned to introduce new features to the platform, make its algorithms open-source, combat spambot accounts, and promote free speech.\nIn July, Musk announced his intention to terminate the agreement, asserting that Twitter had breached their agreement by refusing to crack down on spambot accounts. The company filed a lawsuit against Musk in the Delaware Court of Chancery shortly thereafter, with a trial scheduled for the week of October 17. Weeks before the trial was set to begin, Musk reversed course, announcing that he would move forward with the acquisition. The deal was closed on October 27, with Musk immediately becoming Twitter\'s new owner and CEO. Twitter was taken private and merged into a new parent company named X Corp. Musk promptly fired several top executives, including previous CEO Parag Agrawal. Musk has since proposed several reforms to Twitter and laid off half of the company\'s workforce. Hundreds of employees then resigned from the company after Musk issued an ultimatum demanding they commit to "extremely hardcore" work. Linda Yaccarino was appointed CEO of X Corp. In June 2023, the Twitter service was rebranded as X.\nReactions to the buyout were mixed, with praise for Musk\'s planned reforms and vision for the company, particularly his calls for greater free speech, but criticism over fears of a potential rise in misinformation and disinformation, harassment, and hate speech on the platform. Within the United States, conservatives have largely supported the acquisition, while many liberals and former Twitter employees have voiced concerns about Musk\'s intentions. Since becoming owner, Musk has faced backlash for his handling of the company and account suspensions, including the December 2022 suspensions of ten journalists.', 'source': 'https://en.wikipedia.org/wiki/Acquisition_of_Twitter_by_Elon_Musk'})]

That’s all from this article. We have looked into some open source Vector stores and how to query them. We also looked into few Retrievers which retrieves the documents based on the user’s query. We will learn more about Retrievers in detail in the upcoming articles.

All the code have been added in the Github link. Let me know if you have any questions.

Connect with me.

References

Vector stores | 🦜️🔗 Langchain