I am trying to build RAG LLM using llamaindex.
I use Ollama models: (1) gemma for chatmodel and (2) nomic-embed-text for the embedding.
I use the exact the models with langchain and it work properly (giving me the result).
Here is my langchain code:
from langchain_ollama import OllamaEmbeddings
from langchain_community.llms import Ollama
from langchain.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import FAISS
from langchain.memory import ConversationBufferMemory
from langchain.chains import ConversationalRetrievalChain
model = Ollama(model = 'gemma', temperature = 0.1)
embedding = OllamaEmbeddings(model = 'nomic-embed-text')
raw_documents = PyPDFLoader(path+file).load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1500, chunk_overlap=200)
documents = text_splitter.split_documents(raw_documents)
db = FAISS.from_documents(documents, embedding)
memory = ConversationBufferMemory(memory_key='chat_history', return_messages = True)
query_engine = ConversationalRetrievalChain.from_llm(model, retriever=db.as_retriever(), memory=memory, verbose=True)
response = query_engine.run({'question':'when in the registration period'})
print(response)
# The registration period is between Jan to Feb.
which give the result as what I expect as "The registration period is between Jan to Feb."
And this is for llamaindex:
from llama_index.llms.ollama import Ollama
from llama_index.embeddings.ollama import OllamaEmbedding
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.core.ingestion import IngestionPipeline
from llama_index.core.node_parser import TokenTextSplitter
Settings.llm = Ollama(model="gemma", request_timeout=360.0)
Settings.embed_model = OllamaEmbedding(model_name="nomic-embed-text")
documents = SimpleDirectoryReader(path).load_data(file)
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("when in the registration period")
print(response)
# The provided text does not contain information regarding..., so I am unable to answer this query from the given context.
Howver, this llamaindex results that it cannot find the information in the provided document. It said "The provided text does not contain information...
I expcet the results to be simialr as I use similar embedding and chat model.