How to Build a Private RAG Pipeline with Qdrant, SearXNG, and Ollama

Out of the box, a Large Language Model suffers from two fundamental flaws: absolute amnesia regarding your personal or corporate data, and an inherent inability to reference current, real-time events published after its final training cutoff date. If you ask a pristine Llama3.3 model what your company's refund policy is, or what the stock market closed at today, it will confidently hallucinate an answer.

The industry-standard solution to this problem is RAG: Retrieval-Augmented Generation. RAG intercepts an arbitrary user prompt, converts it into a mathematical vector, queries a database for highly-similar factual documentation, and then forcibly injects those retrieved facts into the LLM's prompt window before it begins typing. The model is effectively given an open-book test.

However, running RAG implementations via third-party cloud services fundamentally requires transmitting your sensitive PDFs, API specs, and proprietary research directly to external remote servers. This tutorial outlines how to construct a 100% air-gapped, entirely private RAG infrastructure utilizing Qdrant for vector storage, SearXNG for real-time web awareness, and Ollama for isolated semantic inference.

The Anatomy of the Private Stack

We are going to orchestrate four fundamentally distinct open-source software packages to work symmetrically.

1. The Inference Engine (Ollama): Will host both the heavy conversational generative model (e.g., Llama 3) and a small, highly efficient 'Embedding Model' designed exclusively to translate text strings into dense arrays of numbers (e.g., Nomic-Embed-Text).
2. The Vector Database (Qdrant): Written entirely in Rust, Qdrant is staggeringly fast. It will permanently store these vast embedding arrays and rapidly calculate the geometric 'cosine-distance' between your question and thousands of pages of internal documents in milliseconds.
3. The Web Crawler (SearXNG + Browserless): For real-time data lacking in your internal database, SearXNG will quietly scour the internet anonymously, while Browserless utilizes a headless Chromium instance to bypass Cloudflare scripts and read raw target paragraphs dynamically.
4. The Orchestrator (n8n): The visual logic glue that controls the physical data pipeline pathways.

Infrastructure Generation via better-openclaw

Wiring these internal Docker bridges manually is hazardous due to latency bottlenecks and potential security misconfigurations. Using better-openclaw significantly accelerates the deployment by utilizing a heavily-tested preset flag:

npx create-better-openclaw --preset researcher --yes

This explicit preset scaffolds the exact requested services. It links the Redis cache layer into SearXNG dynamically so sequential identical search queries resolve in 0ms without exhausting API limits, and it automatically connects Qdrant to an isolated persistive volume so your vector arrays survive total server reboots permanently.

Phase 1: The Ingestion Pipeline (Loading Data)

Before the LLM can pull data, the data must be vectorized.

Using n8n, create an entirely new automated ingestion workflow:

• Trigger: Watch a specific local server directory via a local File Trigger, or listen to a webhook endpoint that receives uploaded PDFs dynamically.
• Chunking: Large entire documents must be fractured. A 100-page PDF will overload a context window. Use the 'Document Chunking' node to slice the text into overlapping segments (e.g., 512 tokens long, with a 50-token overlap to ensure paragraphs aren't abruptly cut off mid-sentence).
• Embedding Translation: Pass each tiny 512-token chunk to the Ollama API, explicitly demanding it uses the nomic-embed-text model. Ollama responds with a massive array of floats like [0.0123, -0.0456...].
• Database Injection: Send these vast numerical arrays into Qdrant alongside critical metadata tags representing the origin of the chunk (author: "Alice", department: "Legal", date: "2026-01-14").

Phase 2: The Retrieval Pipeline (Answering Questions)

When the user subsequently queries the chatbot interface (like Open WebUI or Librechat), the flow inverses:

The user asks: "What is our updated remote work policy regarding out-of-state travel?"

The orchestrator algorithm intercepts this sentence explicitly. It immediately bounces the text string to the exact same nomic-embed-text model on Ollama, converting the question itself into a vector array. It then queries Qdrant: "Fetch the top 5 most mathematically similar arrays to this question, but strictly filter metadata so the department is equal to 'HR'."

Qdrant retrieves the five exact correct paragraphs spanning millions of documents instantly. Finally, the orchestrator compiles the final super-prompt dynamically:

System rules: Answer the user's question using ONLY the provided explicit context blocks below. 
If the answer is not present, explicitly declare ignorance.

Context Block 1: [Inserted Qdrant text]
Context Block 2: [Inserted Qdrant text]

User Question: What is our updated remote work policy regarding out-of-state travel?
		

This super-prompt is passed to the massive conversational Llama 3 model. Because the context is forcefully anchored natively within the prompt, the Llama model flawlessly synthesizes the exact truthful answer without a single hallucinated artifact, and crucially, without a single byte of plaintext data ever traversing the public internet.