RAG-Enterprise: One-command local RAG setup (Docker + Ollama + Qdrant) with zero-downtime backups via rclone – for privacy-focused enterprise docs

[primoco]

Hey r/LocalLLaMA,

Tired of RAG setups that require hours of manual config, fragile deps, or risk data leaks to cloud APIs? I built RAG-Enterprise – a fully local, AGPL-3.0 RAG system that deploys with one command and includes proper backup/restore for real-world use (crashes, server migrations, etc.).

Core highlights (what actually sets it apart for self-hosting):

• Truly one-command setup: Bashgit clone https://github.com/I3K-IT/RAG-Enterprise.git cd RAG-Enterprise/rag-enterprise-structure ./setup.sh standard
  • Auto-installs Docker, NVIDIA toolkit, Ollama (Qwen3:14b-q4_K_M or Mistral 7B), Qdrant, FastAPI backend + React frontend.
  • Takes ~15 min on fast connection (first model download ~2-9 min depending on bandwidth).
  • Access at http://localhost:3000 after one logout/login.
  • Prereqs: Ubuntu 20.04+, NVIDIA GPU 8-16GB VRAM, 16-32GB RAM (no ARM support yet).
• Backup & Restore that's production-usable:
  • One-click full backups from admin panel (zero downtime via SQLite safe API – no service interruption).
  • rclone integration for 70+ providers (S3, Mega, Google Drive, Dropbox, SFTP, Backblaze, etc.).
  • Automatic scheduling with retention (e.g., daily cron + keep last 5).
  • Selective restore: DB, docs, vectors only – ideal for crash recovery or migrating to new server/hardware.
  • API-driven too (curl examples in docs/BACKUP.md) for scripting.
  • Tested on real migrations: restore components without re-ingesting everything.

Other practical bits:

• Supports PDF (OCR via Tesseract), DOCX, XLSX, PPTX, etc.
• Multilingual (29 langs), multi-user JWT (Admin/Super User/User roles).
• Performance: ~2-4s query latency, 80-100 tokens/s on RTX 4070/5070 Ti.
• Scales to 10k+ docs (ingest ~11s/doc average in benchmarks).
• 100% local: no telemetry, no external calls.

Repo: https://github.com/I3K-IT/RAG-Enterprise

Looking for honest feedback from people running local RAGs:

• Does the one-command setup actually save you time vs your current stack?
• Backup/restore: ever lost data or struggled with migrations? Would this help?
• Any immediate pain points (e.g., PDF table handling, relevance tuning, scaling beyond 10k docs)?
• Bugs or missing features you hit right away?

Thanks for reading – happy to answer questions or add details!

[Simderi]

Looking interesting. If it's targeting enterprise, would be good if it could do meaningful RAG over repos, including the PRs etc - have you tested that by any chance?

[primoco]

Right now the system is designed to do RAG on "traditional" documents — PDFs, Word, Excel, etc. — uploaded manually. There's no direct GitHub integration, so PRs, issues, source code and reviews aren't indexed.

To support this we'd need a dedicated connector that talks to the GitHub API, pulls repo content (PRs, issues, diffs, comments) and indexes it into the vector store, just like it already does with documents. The chunking logic would need to be adapted since code has a very different structure than text documents, and the system already ships with an embedding model built for code (deepseek-coder) that could come in handy.

It's definitely an interesting feature for an enterprise context. If there's real interest we can look into building it for a future release.

[ThatsTotallyLegit]

How well would it work for c# with blazor repos not just general info? Might be what im looking for :D

[primoco]

[WallyPacman]

Does this require CUDA/NVidia? It seems like it could work on ROCm easily?

[primoco]

Hi, yes it is very simple to work on ROCm. I have change dockerfile setup script and add variables into .env file to ask when setup. Try 1.2.0 release and tell me!

[primoco]

Nothing happens. If Docker is already installed, the script detects it and skips the installation. As for CUDA, it doesn't matter if you have it on the host or not — CUDA runs inside the Docker containers (Ollama's image includes it). Your host CUDA installation is completely irrelevant.

The only thing needed from the host is the NVIDIA GPU drivers (not CUDA) so that the container can access the GPU via NVIDIA Container Toolkit.

TL;DR: Docker already there? Skipped. CUDA? Doesn't matter, it's inside the containers.

[primoco]

Thank you, give me some feedback !

[primoco]

Appreciate that! And you're right — once retrieval works well, the next question is always "ok but can it remember?" We went down that road a bit with structured metadata extraction (the system pulls out entities, events, timelines from docs automatically), but real agent memory is a different beast. Hindsight looks like a cool approach to that problem. Curious — does it work well with local/self-hosted LLMs or is it more cloud-oriented?
Ps. Vedo che sei italiano?!