Retrieval-Augmented Generation for Large Language Models: A Critical Review of Architectures, Evaluation Practices, and Open Challenges
Keywords:
retrieval-augmented generation; large language models; information retrieval; natural language generation; hallucination mitigation; critical literature reviewAbstract
Large language models (LLMs) generate fluent text but remain constrained by static, parametric knowledge: they hallucinate, cannot readily access information created after training, and are difficult to audit for the provenance of a given claim. Retrieval-Augmented Generation (RAG) addresses these constraints by conditioning generation on evidence retrieved from an external corpus at inference time, and has become one of the dominant design patterns for deploying LLMs in knowledge-intensive settings. This review synthesizes the RAG literature through mid-2026 across four dimensions that are typically treated separately: architecture, retrieval mechanics, evaluation, and failure modes. We organize the diversifying architecture space into a taxonomy spanning naive, advanced, and modular/agentic designs, tracing specific instantiations (Self-RAG, Corrective RAG, GraphRAG, agentic RAG) to their original sources. We then synthesize comparative retrieval results across four independent studies spanning e-commerce, financial, exam-style, and research-corpus retrieval, showing that no single retrieval strategy (sparse, dense, or hybrid) dominates across domains, which complicates common design defaults. We review the evaluation ecosystem (RAGAS, ARES, RAGChecker, RGB, CRAG-bench) and identify a structural blind spot: reference-free faithfulness metrics can score a confidently wrong answer as faithful when the retriever, rather than the generator, is the source of the error. We further synthesize the RAG-versus-long-context debate using a large controlled study and show the two approaches are complementary rather than substitutable, with retrieval remaining necessary for a non-trivial share of queries even against long-context models. Finally, we summarize adoption patterns in a safety-critical domain (medicine and nursing) using a 2025 scoping review, and outline open problems in evaluation standardization, retrieval robustness, and provenance.
Downloads
References
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459–9474.
[2] Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W. (2020). Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 6769–6781).
[3] Robertson, S. E., & Zaragoza, H. (2009). The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4), 333–389.
[4] Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., van den Driessche, G. B., Lespiau, J.-B., Damoc, B., Clark, A., et al. (2022). Improving language models by retrieving from trillions of tokens. In Proceedings of the 39th International Conference on Machine Learning (ICML) (pp. 2206–2240). PMLR.
[5] Asai, A., Wu, Z., Wang, Y., Sil, A., & Hajishirzi, H. (2024). Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations (ICLR 2024).
[6] Yan, S.-Q., Gu, J.-C., Zhu, Y., & Ling, Z.-H. (2024). Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15884.
[7] Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R. O., & Larson, J. (2024). From local to global: A graph RAG approach to query-focused summarization. arXiv preprint arXiv:2404.16130.
[8] Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Guo, Q., Wang, M., & Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997.
[9] Singh, A., Ehtesham, A., Kumar, S., Khoei, T. T., & Vasilakos, A. V. (2025). Agentic retrieval-augmented generation: A survey on agentic RAG. arXiv preprint arXiv:2501.09136.
[10] Sharma, C. (2025). Retrieval-augmented generation: A comprehensive survey of architectures, enhancements, and robustness frontiers. arXiv preprint arXiv:2506.00054.
[11] Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2026). Retrieval-augmented generation for AI-generated content: A survey. Data Science and Engineering, 11, 1–29.
[12] Es, S., James, J., Espinosa-Anke, L., & Schockaert, S. (2024). RAGAS: Automated evaluation of retrieval augmented generation. arXiv preprint arXiv:2309.15217.
[13] Saad-Falcon, J., Khattab, O., Potts, C., & Zaharia, M. (2024). ARES: An automated evaluation framework for retrieval-augmented generation systems. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) (pp. 338–354).
[14] Ru, D., Qiu, L., Hu, X., Zhang, T., Shi, P., Chang, S., Wang, C., Sun, S., Liu, H., Zhu, S., et al. (2024). RAGChecker: A fine-grained framework for diagnosing retrieval-augmented generation. In Advances in Neural Information Processing Systems 38 (NeurIPS).
[15] Yu, H., Gan, A., Zhang, K., Tong, S., Liu, Q., & Liu, Z. (2024). Evaluation of retrieval-augmented generation: A survey. arXiv preprint arXiv:2405.07437.
[16] Li, X., Cao, Y., Ma, Y., & Sun, A. (2024). Long context vs. RAG for LLMs: An evaluation and revisits. arXiv preprint arXiv:2501.01880.
[17] Li, Z., Li, C., Zhang, M., Mei, Q., & Bendersky, M. (2024). Retrieval augmented generation or long-context LLMs? A comprehensive study and hybrid approach. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track (pp. 881–893).
[18] Xu, P., Ping, W., Wu, X., Liu, Z., Shoeybi, M., & Catanzaro, B. (2024). ChatQA 2: Bridging the gap to proprietary LLMs in long context and RAG capabilities. arXiv preprint arXiv:2407.14482.
[19] Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., & Manning, C. D. (2024). RAPTOR: Recursive abstractive processing for tree-organized retrieval. In The Twelfth International Conference on Learning Representations (ICLR 2024).
[20] Bai, Y., Lv, X., Zhang, J., Lyu, H., Tang, J., Huang, Z., Du, Z., Liu, X., Zeng, A., Hou, L., Dong, Y., Tang, J., & Li, J. (2024). LongBench: A bilingual, multitask benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 3119–3137).
[21] Creating a taxonomy for retrieval augmented generation applications. (2024). arXiv preprint arXiv:2408.02854.
[22] Towards knowledge checking in retrieval-augmented generation: A representation perspective. (2024). arXiv preprint arXiv:2411.14572.
[23] When retrieval succeeds and fails: Rethinking retrieval-augmented generation for LLMs. (2025). arXiv preprint arXiv:2510.09106.
[24] Causal-counterfactual RAG: The integration of causal-counterfactual reasoning into RAG. (2025). arXiv preprint arXiv:2509.14435.
[25] Miao, Y., Zhao, Y., Luo, Y., Wang, H., & Wu, Y. (2025). Improving large language model applications in the medical and nursing domains with retrieval-augmented generation: Scoping review. Journal of Medical Internet Research, 27, e80557. https://doi.org/10.2196/80557
[26] Sivakumar, A., Sugumaran, V., & Qiang, Y. (2026). RAG-X: Systematic diagnosis of retrieval-augmented generation for medical question answering. arXiv preprint arXiv:2603.03541.
[27] Medical hallucinations in foundation models and their impact on healthcare. (2025). arXiv preprint arXiv:2503.05777.
[28] Tao, Y., Li, Y., Qin, Y., & Liu, Y. (2025). Retrieval-augmented code generation: A survey with focus on repository-level approaches. arXiv preprint arXiv:2510.04905.
[29] İlter, H. K. (2026). The 17% gap: Quantifying epistemic decay in AI-assisted survey papers. arXiv preprint arXiv:2601.17431.
[30] Toward an engineering of science: Rebalancing generation and verification in the age of AI. (2026). arXiv preprint arXiv:2605.10425.
[31] Yu, T., Xu, A., & Akkiraju, R. (2024). In defense of RAG in the era of long-context language models. arXiv preprint arXiv:2409.01666.
[32] CogRAG: Tackling heterogeneous cognitive demands in RAG via stratified retrieval and reasoning. (2026). arXiv preprint arXiv:2604.25928.
[33] From BM25 to corrective RAG: Benchmarking retrieval strategies for text-and-table documents. (2026). arXiv preprint arXiv:2604.01733.
[34] Novelty-aware agentic retrieval: Comparing research contributions through structured multi-step reasoning. (2026). arXiv preprint arXiv:2606.22151.
[35] Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901.
[36] Hybrid search: BM25, vector & reranking reference 2026 [Industry technical reference, not peer-reviewed; cited for the WANDS e-commerce benchmark figures in Section 5 only]. Digital Applied, 2026. https://www.digitalapplied.com/blog/hybrid-search-bm25-vector-reranking-reference-2026
Published
Issue
Section
License
Copyright (c) 2026 Wei li (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
