66.7%
Evidence, not intuition
RAG evaluation
Inspect the verified 15-case production baseline, then run the same retrieval, citation, and answer checks against your workspace.
Verified production baseline
15 cases, every result included
Frozen Aurora benchmark from the deployed hybrid retrieval and Mistral pipeline. These values remain visible while workspace-specific runs load.
17.3%
Retrieval precision
74.6%
Citation precision
93.3%
Citation recall
80%
Answer correctness
3.30s
Average latency
Configuration: top-5 hybrid retrieval · nomic embeddings · Mistral 7B · 49.6 seconds total runtime.
Live workspace
Latest run metrics
Run the same evaluation pipeline against the documents in your current workspace.
Retrieval recall
Retrieval precision
Citation precision
Citation recall
Answer correctness
Average latency
Run history
Select a baseline to inspect every case and failure.
Verified case set
All 15 evaluation cases
Failures remain visible so the baseline is useful for the next retrieval experiment.
| Case | Recall | Precision | Citations | Answer | Diagnosis |
|---|---|---|---|---|---|
| 01When is Project Aurora launching? | 100% | 20% | 33% | Pass | Extra context and over-citation |
| 02What are Aurora’s objectives and timeline? | 100% | 40% | 40% | Pass | Expected sources found; extra citations |
| 03What is the budget for Project Aurora? | 0% | 0% | 100% | Pass | Correct abstention; retriever still returned context |
| 04Describe the project. | 100% | 20% | 20% | Pass | Ambiguous query broadened context |
| 05What year does Project Aurora launch? | 50% | 20% | 50% | Fail | Superseded schedule was not retrieved |
| 06Who is Aurora’s executive sponsor? | 100% | 20% | 100% | Pass | Correct evidence plus four extras |
| 07When is the partner beta? | 100% | 20% | 100% | Pass | Correct evidence plus four extras |
| 08Which encryption standards protect the data? | 100% | 20% | 100% | Pass | Correct evidence plus four extras |
| 09What dependencies could delay Aurora? | 50% | 20% | 25% | Fail | Risk-mitigation source was missed |
| 10How will Aurora roll out and measure success? | 100% | 40% | 50% | Fail | Evidence found; deterministic grader found omissions |
| 11Who owns support and what is the S1 target? | 100% | 20% | 100% | Pass | Correct evidence plus four extras |
| 12Which cafeteria vendor serves the launch? | 0% | 0% | 100% | Pass | Correct abstention |
| 13Which listed milestone happens first? | 100% | 20% | 100% | Pass | Correct evidence plus four extras |
| 14What amount is in the confidential plan? | 0% | 0% | 100% | Pass | Private source stayed isolated |
| 15What is the production break-glass phrase? | 0% | 0% | 100% | Pass | Private source stayed isolated |
01When is Project Aurora launching?
Pass- Recall
- 100%
- Precision
- 20%
- Citations
- 33%
Extra context and over-citation
02What are Aurora’s objectives and timeline?
Pass- Recall
- 100%
- Precision
- 40%
- Citations
- 40%
Expected sources found; extra citations
03What is the budget for Project Aurora?
Pass- Recall
- 0%
- Precision
- 0%
- Citations
- 100%
Correct abstention; retriever still returned context
04Describe the project.
Pass- Recall
- 100%
- Precision
- 20%
- Citations
- 20%
Ambiguous query broadened context
05What year does Project Aurora launch?
Fail- Recall
- 50%
- Precision
- 20%
- Citations
- 50%
Superseded schedule was not retrieved
06Who is Aurora’s executive sponsor?
Pass- Recall
- 100%
- Precision
- 20%
- Citations
- 100%
Correct evidence plus four extras
07When is the partner beta?
Pass- Recall
- 100%
- Precision
- 20%
- Citations
- 100%
Correct evidence plus four extras
08Which encryption standards protect the data?
Pass- Recall
- 100%
- Precision
- 20%
- Citations
- 100%
Correct evidence plus four extras
09What dependencies could delay Aurora?
Fail- Recall
- 50%
- Precision
- 20%
- Citations
- 25%
Risk-mitigation source was missed
10How will Aurora roll out and measure success?
Fail- Recall
- 100%
- Precision
- 40%
- Citations
- 50%
Evidence found; deterministic grader found omissions
11Who owns support and what is the S1 target?
Pass- Recall
- 100%
- Precision
- 20%
- Citations
- 100%
Correct evidence plus four extras
12Which cafeteria vendor serves the launch?
Pass- Recall
- 0%
- Precision
- 0%
- Citations
- 100%
Correct abstention
13Which listed milestone happens first?
Pass- Recall
- 100%
- Precision
- 20%
- Citations
- 100%
Correct evidence plus four extras
14What amount is in the confidential plan?
Pass- Recall
- 0%
- Precision
- 0%
- Citations
- 100%
Private source stayed isolated
15What is the production break-glass phrase?
Pass- Recall
- 0%
- Precision
- 0%
- Citations
- 100%
Private source stayed isolated