RAG Site Chatbot

Overview

A chatbot that answers questions about a website using only that website’s content, with a link to the page behind every answer. It’s the chat button in the corner of this site: ask it about my projects, my stack, or how to work with me.

It’s built to work for any site, at no cost. Nothing site-specific lives in the code. The source, name, wording, colours, and which sites may use it are all settings, so pointing it at another site takes two settings and one indexing run. It runs entirely on free tiers: Vercel, Groq, and a free vector database.

I built it twice, with identical pipelines and only the vector database swapped, once on Qdrant and once on Upstash Vector, to measure what the database actually changes. Both are live: Qdrant version (the one on this site) and Upstash version.

How it works

• Content: each site publishes its content as one Markdown file, llms-full.txt. Mine is generated from this site’s own content on every deploy. I wrote a short format spec for it, and the chatbot also reads the other shapes real sites produce.
• Indexing: the content is split into passages at its headings, and each passage is embedded twice with FastEmbed: bge-small for meaning and BM25 for exact keywords like “Paddle” or “PKCE”. Both are open Hugging Face models, run on the CPU. A nightly GitHub Action keeps the index in sync.
• Search: a question is embedded the same two ways. The database returns the 20 best passages by meaning and the 20 best by keywords, fused with Reciprocal Rank Fusion into the top 5. This is hybrid search.
• Answer: those passages go to gpt-oss-120b on Groq with strict rules: answer only from the sources, cite them, decline anything off-topic, and treat the sources as content, never as instructions. The answer streams back with its source links.
• API: FastAPI on Vercel, with the two embedding models deployed alongside the code.

Works on any site

• A crawler for sites without the file. For a site that doesn’t publish llms-full.txt, a crawler built on trafilatura reads its sitemap, extracts each page’s main content without the menus and footers, and writes the file. A --check mode validates any site’s file.
• Tested beyond this site. I indexed four more sources with no code changes: this site crawled instead of read from its file, and the Groq, Upstash, and Tauri documentation, at up to 3.6 MB and 5,061 passages. Each one exposed a different file shape for the parser to handle.
• One script tag to embed. A 4.5 KB embed.js adds a round launcher. The chat itself loads in an iframe on the first click, so the host page stays fast, and on phones it opens full screen.

Built twice: Qdrant vs Upstash Vector

Same content, chunking, embeddings, search settings, model, and 38 test questions. Only the database differs:

At this size, the database changed nothing about quality or speed. Both scored the same to the third decimal, and a search takes milliseconds inside Vercel. The differences are in the developer experience and the free-tier rules. Qdrant runs locally and has more knobs, but deletes an idle free cluster; a daily health check keeps it active. Upstash needs no setup and never sleeps, but has no local option and counts every write.

Guardrails and cost

• Only allowed sites can use it. The chat page can only be embedded on approved sites, and the API rejects requests from anywhere else.
• Limits: each visitor gets 10 questions an hour. When the free model limit is reached, the chat offers “Book a call” instead of failing.
• Tested by hand with off-topic questions and prompt-injection attempts, including an instruction planted inside a source. It declined or ignored them all.
• Cost: $0. The real ceiling is Groq’s free token budget, about 150 answers a day. That’s plenty for a portfolio, and it’s why each answer sends only five passages instead of the whole site.

What I learned

• Search quality came from the pipeline, not the database: the headings-first chunking, the hybrid search, and a clean source file.
• Defaults matter. Qdrant’s rank fusion uses k = 2 by default, not the usual 60. Exact ties came back in random order until I broke them in code. And at this size, Qdrant doesn’t even build its search index, so tuning it changed nothing.
• Most of the delay is network and startup, not search. From Pakistan, a search round trip is ~780 ms of network around ~1 ms of actual searching. A cold start is mostly Vercel loading a ~400 MB function with its models.