RRAG System

System architecture

Evidence moves forward.Trust is checked twice.

RAG System separates storage, retrieval, authorization, generation, and evaluation into inspectable stages. The model never chooses what a user is allowed to see; application code and PostgreSQL do.

Read the product overviewTry document search

Production evidence

Measured on the deployed pipeline

A frozen 15-case benchmark keeps the live system accountable while workspace evaluation runs provide newer, tenant-specific evidence.

0 execution errors

66.7%

Retrieval recall

17.3%

Retrieval precision

74.6%

Citation precision

93.3%

Citation recall

80%

Answer correctness

3.30s

Average latency

Verified Aurora baseline · top-5 hybrid retrieval · Mistral 7B · 49.6 seconds total runtime.

System map

Two pipelines, one authorization boundary

Ingestion prepares evidence. Query-time retrieval selects it. The final permission checkpoint sits between ranking and model context.

The only arrows into Mistral originate after the independent permission checkpoint. Retrieval scores never grant access.

Production stack

Open-source inference, managed delivery

The browser stays on one Vercel origin. Application routes enforce sessions and workspace scope, Neon stores searchable evidence, and a private gateway reaches Ollama without exposing it directly.

Application

Next.js App Router

Server-rendered routes, authenticated API handlers, and responsive React interfaces deployed on Vercel.

Database

Neon PostgreSQL

Workspace-scoped relational data, PostgreSQL full-text search, migration tracking, and durable retrieval traces.

Vector index

pgvector

768-dimensional nomic embeddings and indexed distance search run beside document metadata and permissions.

Inference runtime

Ollama

Open-source embedding and generation models run outside Vercel and remain reachable through an authenticated gateway.

Generation

Mistral 7B

Grounded answer generation receives authorized chunks only and emits source labels for deterministic validation.

Private network

Tailscale gateway

A shared-secret HTTPS bridge connects serverless API routes to Ollama without publishing the local Ollama port.

Pipeline 01

Document ingestion

The write path validates authorization and content before it spends inference time or creates searchable state.

  1. 01

    Authorize

    Require live admin or editor membership in the target workspace.

  2. 02

    Validate

    Accept non-empty text or Markdown up to 50 MiB locally; cap synchronous Vercel uploads at 100 KB (roughly 50 chunks).

  3. 03

    Hash

    A SHA-256 content digest prevents duplicates inside one workspace.

  4. 04

    Chunk

    Use ~500-token windows, natural boundaries, and 100-character overlap.

  5. 05

    Embed and store

    Persist text, offsets, and optional 768-dimensional vectors.

Recoverable provenance

Each chunk stores its zero-based index and exact start/end character offsets, so the original source slice can be reconstructed.

Bounded concurrency

Embedding work runs in batches with configurable concurrency capped at eight, protecting a local Ollama process from an unbounded fan-out.

Visible partial failure

Failed embeddings are logged and stored as null. A document becomes error only when every chunk embedding fails; status and error details remain inspectable.

Pipeline 02

Hybrid retrieval

Lexical and semantic signals run concurrently, stay workspace-scoped in SQL, and meet through rank fusion rather than incomparable raw scores.

Keyword signal

PostgreSQL full-text search

A stored English tsvector and GIN index serve plainto_tsquery matches. ts_rank normalization 32 bounds each lexical relevance score to 0–1.

Deadline: 5 seconds

Semantic signal

pgvector L2 distance

nomic-embed-text embeds the question, then the HNSW vector index orders completed workspace chunks by embedding distance.

Combined embedding + SQL deadline: 30 seconds

Reciprocal Rank Fusion
// One-based rank, with k = 60
score(chunk) =
  1 / (60 + keywordRank)
  + 1 / (60 + vectorRank)

Each list contributes by position. A chunk present in both rankings receives both terms; duplicates are collapsed, ties are deterministic, and the highest fused scores become candidates.

Security boundary

Permission filtering happens before context

Workspace filters inside retrieval reduce exposure, but the orchestration layer does not trust that first check alone.

What the checkpoint proves

  • The requesting user is still a member of the session workspace.
  • Every candidate chunk belongs to a document in that same workspace.
  • Only authorized chunk IDs survive into prompt construction.
  • Filtered IDs and counts are logged without sending their text to the model.

The implemented boundary is workspace membership. Fine-grained per-document sharing rules would require an additional grants table and predicate.

Orchestration invariant
const candidates = await hybridSearch(query, workspaceId, topK);

// Security checkpoint: candidate text cannot enter the prompt before this.
const authorized = await filterByPermissions(
  candidates,
  workspaceId,
  userId,
);

return authorized.slice(0, topK);

The comment is enforced by data flow: generateAnswer receives only the array returned after filterByPermissions.

Pipeline 03

Grounded generation and citation validation

Mistral receives the question and authorized chunks as untrusted source material, with exact source labels it is allowed to copy.

  • 01

    Build context

    Format each chunk as [source: chunk_id] followed by its text.

  • 02

    Constrain

    Require document-only claims and an explicit refusal when evidence is missing.

  • 03

    Generate

    Call local Mistral without streaming under a 60-second timeout.

  • 04

    Validate

    Keep source occurrences only when the ID was in authorized context.

    • Citation extraction
    const pattern = /\[source:\s*([^\]\s]+)\s*\]/gi;

for (const match of answer.matchAll(pattern)) {
  const chunk = authorizedChunksById.get(match[1]);
  if (chunk) citations.push({ chunk_id: chunk.chunk_id, position: match.index });
  else console.warn("Fabricated citation filtered", { chunk_id: match[1] });
}

    Validation is deterministic. Unknown IDs are logged and discarded, while every valid occurrence retains its answer position for inline rendering.

    Failure contract

    “I cannot answer this question based on the provided documents.”

    That response is an intended safety result when retrieval returns no authorized evidence—not a generation failure.

    Closed feedback loop

    Evaluation and observability

    Search traces make production behavior inspectable; the benchmark turns that same path into repeatable quality signals.

    Recall

    Expected chunks retrieved ÷ expected chunks

    Precision

    Expected chunks retrieved ÷ retrieved chunks

    Citation quality

    Cited expected evidence in both directions

    Answer correctness

    Deterministic expected key-phrase coverage

    No-answer accuracy

    Correct refusal when no evidence is expected

    What is persisted

    Each run stores status, case count, aggregate summary, timestamps, and errors. Case results retain retrieved/cited chunk IDs, answer, metrics, latency, and categorized failure reason.

    What the score does not claim

    Answer correctness is a transparent token-coverage heuristic, not a semantic LLM judge. It is reproducible and free, but human review remains the authority for nuanced paraphrases.

    Defense in depth

    Trust boundaries at a glance

    No single model response, route parameter, or retrieval score decides authorization or provenance.

    Session

    Input

    Opaque cookie token

    Control

    HMAC-hashed server record, expiry, live workspace membership

    Output

    User and workspace identity

    Database

    Input

    Validated IDs and text

    Control

    Parameterized SQL, foreign keys, workspace predicates

    Output

    Scoped documents and chunks

    Retrieval

    Input

    Ranked candidates

    Control

    Independent membership + document ownership join

    Output

    Authorized chunk array

    Model

    Input

    Authorized chunks only

    Control

    Grounding prompt, source allowlist, timeout

    Output

    Untrusted answer text

    Response

    Input

    Untrusted source labels

    Control

    Exact chunk-ID validation and fabricated-citation logging

    Output

    Answer with validated citations

    Inspect the evidence

    Architecture is only useful when it can be measured.

    Run the seeded cases, inspect individual failures, and compare retrieval, citation, answer, refusal, and latency metrics.

    Open evaluations