Enterprise Document Intelligence: A Production-Grade RAG Pipeline with Hybrid Retrieval, Reranking, by IRJET Journal

Enterprise Document Intelligence: A Production-Grade RAG Pipeline with Hybrid Retrieval, Reranking, and Conversational Memory

Adeeb Sailani Shaikh1, Sanidhya Sachin Kulkarni2, Ratnamala Kumar Sudhir Paswan3

1Student, Pune Institute of Computer Technology (PICT), Pune, India

2Student, Pune Institute of Computer Technology (PICT), Pune, India

3Assistant Professor, Pune Institute of Computer Technology (PICT), Pune, India

Abstract - Enterprises accumulate large volumes of valuable knowledge in PDFs, reports, manuals, and policy documents, but finding the right information is still difficult when user language does not match document wording. This paper presents an enterprise document intelligence system built on Retrieval-Augmented Generation (RAG) to improve both relevance and factual reliability. Our pipeline combinesdenseretrieval(FAISSwithSentence Transformer embeddings) and sparse retrieval (BM25) through Reciprocal Rank Fusion and then applies cross-encoder rerankingtoimprovetop-resultprecision.Forambiguousor underspecified questions, the system generates alternate query formulations before retrieval to improve recall. A dual-memory design supports multi-turn interaction through short-term conversational context and SQLitebacked long-termpersistence. Theimplementationincludes practical input/output guardrails, a FastAPI backend with streaming responses, and a Streamlit chat interface with session and document management. Evaluation using Recall@K, Precision@K, MRR@K, NDCG@K, and faithfulness-oriented checks shows consistent gains from hybrid retrieval, reranking, and query expansion. The full stackiscontainerizedwithDockerComposeanddesignedas a modular, production-oriented architecture that does not require proprietary infrastructure.

Key Words: Retrieval-Augmented Generation, Hybrid Retrieval, FAISS, BM25, Cross-Encoder Reranking, Query Expansion,ConversationalMemory,Guardrails

1.INTRODUCTION

Enterprises now document almost everything, from compliancenotesandcontractstointernalplaybooks.Yet in day-to-day work, locating one specific answer can still be frustrating. Most enterprise search tools are built aroundkeywordoverlap,whichworks forexactphrasing but struggles when users ask natural questions or use different wording. For example, a sales engineer asking, “what are our SLA commitments for Tier-2 customers?” maymisstheanswerifthecontractinsteadsays“servicelevelobligationsapplicabletoSilver-tieraccounts.”

Large Language Models (LLMs) handle natural-language questionsverywell,buttheyintroduceadifferentrisk: hallucination.Whenrelevantknowledgeismissing,orwhen modelmemoryisoutdated,themodelcanproducefluentbut incorrectresponses.Inenterprisesettings,thatfailuremode isnotminor;afabricatedcompliancenumberoraninvented legalclausecandirectlyaffectdecisions.

Retrieval-Augmented Generation (RAG) addresses this gap by grounding the model’s answer in retrieved evidence. Instead of depending only on parametric memory, the system first fetches relevant passages from the enterprise corpus and then generates a response conditioned on that context. In practice, this improves factual consistency and makessource-backedanswerspossible.

However, a production RAG system requires much more thanwiringavectorstoretoanLLM.Realdeploymentsmust handle:

• Query–document mismatch: Users’ colloquial phrasing may differ substantially from document terminology.

• Precision vs. recall trade-offs: Dense embeddings capturesemanticsimilaritybutmissexactkeywordmatches; sparse methods like BM25 capture lexical overlap but lack semanticunderstanding.

• Multi-turn conversations: Users expect the system torememberearlierpartsoftheconversation.

• Safety and abuse: Public-facing or internally deployed systems must guard against prompt injection, harmfulqueries,andaccidentaldataleakage.

• Latency: Enterprise users expect sub-second retrievalandfastgeneration.

This paper presents Enterprise Document Intelligence; a modular RAG system designed around these practical constraints. The architecture combines dense and sparse retrievalthroughReciprocalRankFusion(RRF),appliescrossencoder reranking for stronger top-result ordering, expands

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 05 | May 2026 www.irjet.net p-ISSN: 2395-0072

ambiguous queries with LLM-generated paraphrases, preserves contextusingdualshorttermandlong-termmemory,andapplies regex-basedinput/outputguardrailsforsafety.

The main contribution is an open-source, productionready implementation that shows how these well-known techniquescanbecombinedintoonecohesivepipelinewith aChatGPT-likeuserexperience.

2.LITERATURE SURVEY

2.1

Retrieval-Augmented Generation

Lewisetal.[1]introducedtheRAGframework,combining a pretrained sequence-to-sequence model with a nonparametricdocumentretriever.Theirworkdemonstrated that grounding generation in retrieved passages significantly reduces hallucination and improves factual consistency. Since then, RAG has become the dominant paradigmforknowledge-intensiveNLPtasksandhasbeen adoptedwidelyinbothresearchandindustry.Lewisetal [1]introducedtheRAGframeworkbypairingasequenceto-sequence generator with a non-parametric retriever. Theirresultsshoweda clearbenefit:when responsesare grounded in retrieved passages, hallucination drops and factual consistency improves. Since that work, RAG has becomeastandardapproachforknowledgeintensiveNLP tasksinbothacademiaandindustry.

2.2 Dense Retrieval and Approximate Nearest Neighbours

Karpukhin et al. [2] proposed Dense Passage Retrieval (DPR) and showed that learned dense embeddings can outperformclassicalTF-IDFandBM25inopen-domainQA settings. FAISS [3] then made large-scale vector search practicalthroughefficientapproximatenearest-neighbour methods. SentenceTransformers [4] lowered adoption barriers further by providing strong sentence-level embeddingsthroughcompactmodelssuchasall-MiniLML6-v2.

2.3 Sparse Retrieval and BM25

BM25 [5] remains a strong and dependable baseline in information retrieval. Its probabilistic term-weighting behaviour is especially useful for exact lexical matches, including rare or domain specific vocabulary that dense models may overlook. Robertson and Zaragoza’s analysis highlights that this strength comes largely from termfrequencysaturationanddocument-lengthnormalisation.

2.4 Hybrid Retrieval and Reciprocal Rank Fusion

Cormacketal.[6]introducedReciprocalRankFusion(RRF), asimpleyeteffectivemethodforcombiningrankedoutputs from different retrievers. RRF uses rank positions rather than raw similarity values, so it avoids unstable score normalisation across heterogeneous systems. More recent workbyMaetal.[7]reportsthatdense–sparsehybridfusion consistentlyoutperformseitherretrievalmodealoneacross multiplebenchmarks.

2.5 Cross-Encoder Reranking

Bi-encoders map queries and documents separately, whereascrossencodersscoreaquery–documentpairjointly andcanmodel finertoken-level interactions.Nogueira and Cho [8] demonstrated that this retrieve-then-rerank setup cansignificantlyimproveprecision.

In practice, ms-marco-MiniLM-L-6-v2 is widely used becauseitoffersagoodaccuracy-latencytrade-off.

2.6 Query Expansion

Classical query expansion methods relied on pseudorelevancefeedbackorcuratedthesauritoimproverecall[9]. Recent LLMbased methods offer a more flexible option by generating semantically equivalent rewrites of the user query and retrieving over all variants. Wang et al. [10] reported Recall@K improvements of 8–15% on enterprise corpora,whereuserpromptsareoftenunderspecified.

2.7 Conversational Memory

Multi-turn question answering depends on preserving context across interactions. Wu et al. [11] proposed conversationalqueryrewritingtechniquesforretrieval,and the LangChain ecosystem [12] helped popularise practical memory designs such as sliding windows and summaries. Long-term persistence, often implemented with relational databases, is essential for enterprise systems that must surviverestartsandmaintainsessioncontinuity.

2.8 Guardrails and Safety

Promptinjectionattacks,inwhichadversarialtextattempts to override model instructions, are now well documented [13]. Rebedea et al. [14] introduced NeMo Guardrails as a programmable framework for safer LLM behaviour. At the same time,lightweight regex-basedfilters remain useful in production as a first defensive layer against common injectionandharmful-contentpatterns.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 05 | May 2026 www.irjet.net p-ISSN: 2395-0072

2.9 RAG Evaluation

Es et al. [15] proposed the RAGAS framework to assess RAG pipelines along three practical axes: faithfulness (grounding in retrievedcontext),answerrelevancy(alignmentwiththequery), and context precision (quality of retrieved passages). These dimensionsarenowwidelyusedwhenbenchmarkingend-to-end RAGsystems.

3.METHODOLOGY

The proposed system Enterprise Document Intelligence is a full-stack RAG application designed for production deployment. Its key design principles are modularity, graceful degradation, and developer accessibility.

3.1 Design Principles

1. Modularity: Every pipeline stage (ingestion, chunking, embedding, retrieval, reranking, generation, evaluation) is encapsulated in its own Python module with a clean interface. This allows any component to be swapped withoutaffectingtherestofthepipeline.

2. GracefulDegradation: Optionalcomponentssuchasthe cross-encoder reranker and BM25 retriever are loaded with try/except guards. If a dependency is missing, the systemcontinueswithreducedfunctionalityratherthan crashing.

3. LLMAgnosticism: ThesystemsupportsbothOllama(for local, privacy-preserving inference with Mistral) and OpenAI (for cloud-based generation with GPT-4o-mini), switchableviaasingleenvironmentvariable.

4. Streaming First: Responses are streamed token-bytokenviaServer-SentEvents(SSE),providingimmediate feedbacktotheuserwhiletheLLMgenerates.

3.2 Core Methodology

The query processing pipeline proceeds through nine sequentialstages:

1. Input Guardrails: The user’s question is checked for length violations, harmful content patterns, and prompt injectionattempts.

2. Cache Lookup: ATTL-basedin-memorycache(max200 entries, 10-minute expiry) is checked for a previously computedanswertothesamequestionwithinthesame session.

3. Query Expansion: The LLM generates two alternative phrasingsofthequestiontobroadenretrievalcoverage.

4. Hybrid Retrieval: Each expanded query is sent to both theFAISSdenseretrieverandtheBM25sparseretriever. ResultsarefusedviaReciprocalRankFusion.

5. Cross-Encoder Reranking: Thefusedcandidate set is reranked using a cross-encoder model for precision.

6. Context Assembly: Thetop-Kretrievedchunks areconcatenatedintoacontextstring.

7. Prompt Construction: A structured prompt is builtfromthecontext,conversationhistory,andthe currentquestion.

8. LLM Generation: The prompt is sent to the configuredLLMbackend(OllamaorOpenAI),which generatestheanswer.

9. Output Guardrails: The generated answer is sanitised,checkedforembeddedinjectionattempts, andtruncatedifnecessary.

After generation, the question–answer pair is persisted to both short-term (in-memory sliding window) and long-term (SQLite) memory stores, andtheresponseiscachedforfuturereuse.

3.3 Hybrid Retrieval with Reciprocal Rank Fusion

The hybrid retriever is a central innovation. For a givenquery��,thesystemobtainstworankedlists:

• Ddense: Top-3�� results from FAISS cosine similaritysearch.

• Dsparse:Top-3��resultsfromBM25Okapiscoring. These are merged via RRF. For each document �� appearingatrank �� indenseresultsandrank ��in sparseresults,thefusedscoreis:

where ��= 60 is the RRF constant, ��= 0.6 is the dense weight, and �� = 04 is the sparse weight. DocumentsarethensortedbydescendingRRFscore andthetop-��arereturned.

Thisapproachofferstwoadvantages:(1)itavoidsthe need to normalise raw scores from heterogeneous retrieval systems, and (2) it naturally boosts documents that appear in both lists while still surfacingdocumentsfoundbyonlyoneretriever.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 05 | May 2026 www.irjet.net p-ISSN: 2395-0072

Fig. -1: High-levelarchitectureoftheEnterprise DocumentIntelligencesystem.Thebrowser communicateswiththeFastAPIbackendvia REST/SSE.Thebackendorchestratesthepipeline componentsforretrieval,reranking,andgeneration.

3.4 Query Expansion Strategy

Forunder-specifiedorambiguousqueries,asingleretrieval passmaymissrelevantdocumentsphraseddifferently.The query expansion module addresses this by prompting the LLM to generate ��= 2 alternative phrasings that preserve theoriginalmeaningbutusedifferentvocabulary.All ��+1 queries (original + expansions) are independently sent throughtheretrieve-rerankpipeline,andresultsaremerged byretainingthehighestscoreforeachuniquechunk.

3.5 System Architecture

The system follows a three-tier architecture: a Streamlitbased frontend, a FastAPI-based backend, and a pipeline layercontainingallMLandretrievalcomponents.

3.5.1 High-Level Architecture

3.5.2 Frontend

Layer

The frontend is a Streamlit web application (frontend/app.py) that provides a ChatGPT-like user interface.Keycapabilitiesinclude:

• Chat Interface: Message bubbles with markdown rendering, code highlighting, and real-time display of assistantresponses.

• Session Management: A sidebar listing previous chat sessions with load and delete buttons. Each session is

identified by a UUID and auto-titled after the first exchange.

• Document Management: A dedicated page for uploading PDF and DOCX files (up to 10MB each), viewingindexeddocuments,anddeletingthem.

Table -1: APIendpointsummary

Meth od Path Description

POST /api/login JWTtokenissuance

GET /api/health Healthcheck

POST /api/upload UploadPDF/DOCX

GET /api/documents Listindexeddocuments

DELE

TE /api/documents/{fn} Deleteadocument

POST /api/query Query(supports streaming)

GET /api/sessions Listchatsessions

POST /api/sessions/new Createnewsession

GET /api/sessions/{id}/mes sages Sessionhistory

DELE

TE /api/sessions/{id} Deletesession

POST /api/evaluate/retrieval Retrievalevaluation

GET /api/metrics Systemmetrics

Table -2: Pipelinemoduleoverview

Module Responsibility ingestion/ PDFandDOCXtextextraction chunking/ Recursiveoverlappingtextchunking embeddings/ SentenceTransformerembedding vector_store/ FAISSindexmanagement retriever/ Dense,sparse,andhybridretrieval reranker/ Cross-encoderreranking query_expansion / LLM-basedqueryparaphrasing

llm/ OllamaandOpenAIgenerators memory/ Short-termandlong-termmemory guardrails/ Input/outputsafetychecks evaluation/ Retrievalandgenerationmetrics

• Metrics Dashboard: Real-time display of system metrics (documents indexed, total chunks, active sessions, cache size) and an evaluation runner that computes faithfulness,answerrelevancy,andcontextprecisionacross atestset.

• Performance Overlay: Aftereachquery,anexpandable panel shows total latency, retrieval time, generation time, andfaithfulnessscore.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 05 | May 2026 www.irjet.net p-ISSN: 2395-0072

The frontend communicates with the backend exclusively through its REST API, using JWT bearer tokensforauthentication.

3.5.3

Backend Layer

The backend is a FastAPI application (app/main.py) exposing twelve RESTful endpoints under the /api prefix:

Authentication uses JWT tokens signed with HS256. The/api/queryendpointsupportsastream:trueflag, inwhichcaseitreturnsatext/event-streamresponse withtoken-bytokenSSEevents.

CORSmiddlewareisenabledwithpermissive settingstosupporttheStreamlitfrontend.An upload-sizelimitermiddlewarerejectspayloads exceeding10MBattheHTTPlevel.

3.5.4 Pipeline Layer

Thepipelinelayer(pipeline/)isorganisedintoeleven independentmodules:

3.5.5 Data Flow

Theend-to-enddataflowcanbesummarisedintwo phases: Ingestion Phase:

1. UseruploadsaPDForDOCXfileviatheUI.

2. Theingestionpipelineextractsrawtextusingpdf plumberorpython-docx.

(FAISS) only

(BM25) only

3. Therecursivechunkersplitsthetextinto400-character chunkswith60-characteroverlap.

4. Each chunk is embedded using all-MiniLM-L6-v2 (384 dimensions).

5. NormalisedembeddingsareaddedtoaFAISSIndexFlatIP index,andtheBM25indexisrebuilt.

6. The FAISS index and chunk metadata are persisted to disk. Query Phase:

1. Usersubmitsanatural-languagequestion.

2. Inputguardrailsvalidatethequery.

3. Cacheischecked;onmiss,queryexpansiongenerates alternativephrasings.

4. Eachphrasingissentthroughhybridretrieval(FAISS +BM25+RRF).

5. Candidatesarererankedbythecross-encoder.

6. Contextandconversationhistoryareassembledintoa prompt.

7. TheLLMgeneratesananswer,whichisstreamedback totheuser.

8. Theexchangeissavedtobothmemorystoresandthe responseiscached.

4.RESULTS AND DISCUSSION 4.1 Evaluation Framework

Toevaluatethesystemcomprehensively,weusetwo complementarytiers:

1. Retrieval Metrics: Recall@K, Precision@K, Mean Reciprocal Rank (MRR@K), and Normalised Discounted Cumulative Gain (NDCG@K), measured againstalabelledevaluationdataset(eval_set.json).

2. Generation Metrics: A heuristic faithfulness score (fraction of answer sentences grounded in context), answer relevancy (keyword overlap between questionandanswer),andcontextprecision(overlap between retrieved and reference context). Optional RAGASintegrationaddsframework-levelchecks.

4.2 Retrieval Quality

We comparedhybrid retrieval againstdense-onlyand sparse-onlybaselinesonthe evaluationdataset.Table 3reportsrepresentativeresultsat��=5. Threepracticaltakeawaysstandout:

• Complementary strengths of dense and sparse retrieval: Denseretrievalshowsbetterrecall(0.72vs. 0.65), which aligns with its semantic matching advantage. Sparse retrieval shows better precision (0.52 vs. 0.45), reflecting BM25’s strength on exact terms.

Table -4: Impactofqueryexpansiononretrieval(K=5)

Table -3: RetrievalperformancecomparisonatK=5

International Research Journal of Engineering and

Volume: 13 Issue: 05 | May 2026 www.irjet.net

Table -5: Generationqualitymetrics(batchevaluation)

typically connective or framing statements (for example, “Based on the available information. . . ”) that may not contain explicit context keywords but are not necessarily hallucinatedclaims.

4.5 Latency Analysis

Table 6 reports typical latency for a single query on consumer-gradehardware(noGPU,16GBRAM).

Table -6: Latencybreakdown(singlequery,noGPU)

• Clear gain from RRF fusion: Hybrid retrieval improves recall by 12 percentage points over denseonly, indicating that dense and sparse retrievers recoverdifferentrelevantchunks.

• Rerankingimprovesorderingquality: The cross-encoder does not affect recall (it reorders candidates),butitimproves precisionby8pointsand MRR by 4 points, which confirms better top-rank quality.

4.3 Effect of Query Expansion

Table 4 shows the effect of query expansion on the hybrid+rerankerpipeline.

Query expansion improves recall by 7 points, supporting the hypothesis that alternative phrasings retrieve relevant content the original wording misses. The 4-point MRR gain suggests that these additional candidatesareoftennotjustrelevant,buthighlyrelevant afterreranking.

4.4 Generation Quality

Thebatchevaluationpipelinecomputesfaithfulnessand answerrelevancyacrossthetestset.Table5 summarizestheresultingaverages.

A faithfulness score of 0.82 means that roughly 82% of answersentencesaredirectlygroundedinretrievedevidence, whichisstrongforaheuristicmetric.The remaining18%are

LLMgeneration is the dominant latencycomponent, which is expected under CPU-only Ollama inference. Caching reduces repeated-query latency to under 10 milliseconds. With GPU acceleration, end-to-end latency for non-cached queriestypicallydropstoabout1–3seconds.

-2: Chatinterfaceshowingauserquery,grounded assistantresponse,andsidebarnavigationforsessions andmodules.

-3: Latencyanalyticspanelshowingrecentqueryresponsetimetrendsandper-querytimingstatistics.

-4: Documentmanagementpageshowingfileupload controlsandthelistofindexeddocumentsavailablefor retrieval.

Fig.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 05 | May 2026 www.irjet.net p-ISSN: 2395-0072

4.6 Guardrails Effectiveness

The guardrails module was tested on 50 adversarial inputs covering prompt injection, harmful content, and length violations. It correctly blocked 48 cases (96% detection rate). The two misses involved injection variantsoutsidethecurrentregexpatterns,highlighting a known limitation of rule-based filtering and a clear directionforstrongerclassifier-baseddefences

5.CONCLUSION

ThispaperpresentedEnterpriseDocumentIntelligence, aproduction-gradeRAGsystemthatcombinesretrieval and generation components into one deployable pipeline.Themaincontributionsandfindingsare:

1. Hybrid retrieval outperforms single-mode retrieval. Combining dense (FAISS) and sparse (BM25) retrieval withReciprocalRankFusiondelivereda12-pointrecall improvement over dense-only retrieval, confirming strongcomplementarity.

2. Cross-encoder reranking improves precision without reducing recall. Thems-marco-MiniLM-L-6-v2 rerankerimprovedPrecision@5by8pointsandMRR@5 by4pointsoverthehybridbaselinewithoutreranking.

3. LLM-based query expansion improves coverage on ambiguous prompts. Generating two alternate phrasings increased Recall@5 from 0.84 to 0.91, a meaningful gain for enterprise queries that are often underspecified.

4. Dual-memory design supports natural conversation flow. In-memory short-term buffers and SQLite-backed long-termstorage together provide low-latencycontext injectionwithcross-sessionpersistence.

5. Practical guardrails provide useful baseline safety. Regex based input/output filtering blocked 96% of adversarialinputsintesting,offeringalightweightlayer thatcanbeextendedwithML-basedclassifiers.

6. A modular design keeps the system adaptable. Clean component boundaries allow the LLM backend (Ollama vs. OpenAI), embedding model, or vector store to be replacedwithoutrefactoringthefullstack. Overall,theresultsshowthatacarefullyengineeredopensource RAG pipeline can deliver a user experience close to commercial assistants while retaining deployment control. Containerization with Docker Compose reduces setup to a singlecommand,makingadoptionpracticalevenforteams withoutdedicatedMLOpssupport.

5.1 Future Directions

Althoughthecurrentsystemcoverscoreenterprise requirements, several high-impact extensions remain:

1. Multi-Modal Document Support: Extend ingestion to handle images, tables, and charts through OCR (Tesseract) and vision language models(GPT-4V,LLaVA).

2. Role-Based Access Control (RBAC): Replace the single admin model with per-user authentication, document-level authorization, and auditloggingsuitableforproduction.

3. ML-Based Guardrails: Augment regex filters withtrainedclassifiers(e.g.,fine-tunedDistilBERT) forstrongertoxicityandprompt-injectiondetection.

4. Agentic RAG: Let the model decide when and how to retrieve instead of always following a fixed path, including tool-use choices across vector search,keywordsearch,anddirectgeneration.

5. Scalable Vector Storage: ReplacetheflatFAISS index with IVF/hierarchical variants or external vector databases (e.g., Qdrant, Milvus) for millionscalecorpora.

6. Fine-Tuned Embeddings: Train or fine-tune embeddingsondomaincorporatoimproveretrieval for specialised terminology (legal, medical, financial).

7. Conversation Summarisation: Add progressive summarisation for long sessions so contextispreservedbeyondfixedwindowmemory limits.

8. Multi-Tenant Architecture: Support multiple organizations with isolated document stores, user identities,andmodelconfigurationsinonedeployment.

9. Streaming Evaluation: Extend evaluation to score streamingresponsesinrealtime,includingincremental faithfulnessandrelevancychecks.

10. GPU-Accelerated Inference: Integrate CUDAenabled Ollama, vLLM, or TensorRT-LLM to push generationlatencyfromsecondstowardmilliseconds.

REFERENCES

[1]P.Lewis,E.Perez,A.Piktus,F.Petroni,V.Karpukhin, N.Goyal,H.Küttler,M.Lewis,W.Yih,T.Rocktäschel,S. Riedel,andD.Kiela,“Retrieval-AugmentedGeneration forKnowledge-IntensiveNLPTasks,”in Proc. NeurIPS, 2020.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 05 | May 2026 www.irjet.net p-ISSN: 2395-0072

[2]V.Karpukhin,B.Oguz,S.Min,P.Lewis,L.Wu,S.Edunov,ˇ D. Chen, and W. Yih, “Dense Passage Retrieval for Open Domain QuestionAnswering,”in Proc. EMNLP,2020.

[3]J. Johnson, M. Douze, and H. Jégou, “Billion-Scale Similarity Search with GPUs,” IEEE Trans. Big Data, vol. 7, no. 3, pp. 535–547,2021.

[4]N. Reimers and I. Gurevych, “Sentence-BERT:Sentence Embeddings using Siamese BERT-Networks,” in Proc. EMNLP, 2019.

[5]S. Robertson and H. Zaragoza, “The Probabilistic Relevance Framework: BM25 and Beyond,” Foundations and Trends in Information Retrieval,vol.3,no.4,pp.333–389,2009.

[6]G. Cormack, C. Clarke, and S. Buettcher, “Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods,”in Proc. SIGIR,2009.

[7]X. Ma, K. Gao, L. Zhao, and J. Lin, “Hybrid Dense-Sparse Retrieval:WhenShouldWeUseEach?”in Proc. ECIR,2023.

[8]R. Nogueira and K. Cho, “Passage Re-ranking with BERT,” arXiv:1901.04085,2019.

[9]J. Xu and W. B. Croft, “Query Expansion Using Local andGlobalDocumentAnalysis,”in Proc. SIGIR,pp.4–11,1996.

[10] L. Wang, N. Yang, and F. Wei, “Query2Doc: Query Expansion with Large Language Models,” in Proc. EMNLP, 2023.

[11] Z. Wu, G. Koncel-Kedziorski, M. Ostendorf, and H. Hajishirzi, “CONQRR: Conversational Query Rewriting for Retrieval,”in Proc. EMNLP,2022.

[12] H. Chase, “LangChain: Building Applications with LLMs throughComposability,”2022.[Online].Available:https: //langchain.com

[13] F. Perez and I. Ribas, “Ignore This Title and HackAPrompt: Exposing Systemic Weaknesses of Language Models,”in Proc. EMNLP,2023.

[14] T.Rebedea,R.Dinu,M.Sreedhar,C.Parisien,andJ.Cohen, “NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications,”in Proc. EMNLP Demo Track,2023.

[15] S. Es, J. James, L. Espinosa-Anke, and S. Schockaert, “RAGAS: Automated Evaluation of Retrieval Augmented Generation,” arXiv:2309.15217, 2023.