Skip to content
Get packs

arXiv Paper Curator

A hands-on course project that builds a research assistant for arXiv papers using production RAG methods.jamwithai/production-agentic-rag-course

GitHub Download week7.0
Stars
10k
Forks
2.2k
Licence
MIT
Language
Python
Last update
5 months ago
Type
Learning resource

What it is

This is a learner-focused course project that walks you through building a research assistant. The system fetches academic papers from arXiv, parses and indexes them, and answers questions about them with retrieval-augmented generation (RAG). It is built in weekly stages, from Docker and search infrastructure up to an agentic version with LangGraph and a Telegram bot.

  • Fetch and parse academic papers from arXiv into a local database
  • Run keyword (BM25) and hybrid search over indexed papers in OpenSearch
  • Ask questions about papers through a Gradio chat interface
  • Trace RAG pipeline runs with Langfuse and cache results with Redis
  • Query papers through a Telegram bot in the Week 7 agentic version

Great for

  • Engineers learning how to build RAG systems step by step
  • Students who want a complete project tied to weekly blog posts
  • Developers who already know Python and Docker and want a hands-on search-plus-LLM example
  • Anyone who wants to follow the course path week by week using tagged releases

Not ideal for

  • People who want a ready-made research tool without setup, since it requires Docker, Python 3.12+ and a sizeable machine (8GB+ RAM, 20GB+ disk)
  • Readers new to Python or containers, because the README assumes familiarity with Docker Compose and uv
  • Teams looking for a general production library, since this is structured as a course rather than a reusable package
Step-by-step guide: how to use arXiv Paper CuratorInstall, first run, examples, tips and FAQs
LangChain vs arXiv Paper CuratorLlamaIndex vs arXiv Paper Curator

How to get it

  1. 1

    Clone and build from source

    You'll need: Docker Desktop with Docker Compose · Python 3.12+ · UV Package Manager · 8GB+ RAM and 20GB+ free disk space

    Clone the repository

    git clone <repository-url>

    Copy the environment template

    cp .env.example .env

    Install Python dependencies

    uv sync

    Start all services

    docker compose up --build -d
  2. 2

    Run it

    • Add your Jina embeddings API key and Langfuse keys to the .env file
    • Start all services with docker compose up --build -d
    • Check the service is healthy with curl http://localhost:8000/api/v1/health
    • Open the API docs at http://localhost:8000/docs or the Gradio interface at http://localhost:7861
  3. 3

    Get the most out of it

    • Work through the weekly tagged releases (week1.0 through week7.0) to see how the system grows
    • Use the Langfuse dashboard at localhost:3000 to trace RAG pipeline runs
    • Explore the OpenSearch Dashboards at localhost:5601 to inspect hybrid search results
    • Follow the Week 1 notebook at notebooks/week1/week1_setup.ipynb for guided verification

Downloads

Source code (.zip)main branch · straight from GitHubRelease week7.0Release notes and source

Strengths

  • Organized as a clear weekly progression with blog posts and tagged releases for each stage
  • Starts from search fundamentals before adding vector retrieval, which the README presents as a deliberate teaching choice
  • Covers a full stack of components (FastAPI, PostgreSQL, OpenSearch, Airflow, Ollama) in one runnable project

Trade-offs

  • Setup is heavy: Docker Desktop, Python 3.12+, uv, and a Jina API key plus Langfuse keys are needed before the system runs fully
  • Several parts rely on external blog posts for explanation, so the README alone gives limited detail
  • The README is truncated here, so later weeks and some configuration specifics could not be verified

How it compares

ProjectStarsTypeLevelUpdatedBest for
production-agentic-rag-course (this)10kLearning resourceintermediate5 months agoEngineers learning how to build RAG systems step by step
langchain149kFrameworkintermediate4 days agoPython developers building LLM-powered applications
llama_index52.8kLibraryintermediate10 days agoPython developers building retrieval-augmented generation (RAG) apps over private documents

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-11-02 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.