⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠ You can decompress Drawing data with the command palette: ‘Decompress current Excalidraw file’. For more info check in plugin settings under ‘Saving’
Excalidraw Data
Text Elements
Synthetic Test Set Generation Pipeline
Your Corpus (Documents)
• PDFs, Text files • Knowledge base • Internal docs
Chunked
LLM Generator “Generate Q&A from this chunk”
Prompt: “Given context, generate realistic user questions and ground-truth answers”
Synthetic Test Set (Q, A, Context)
For each sample: • Question (Q) • Ground-truth Answer (A) • Source Document(s)
Evaluate RAG
Context Relevance
Faithfulness
Answer Relevance
Answer Correctness
No manual labeling, domain-specific, scalable, iterative refinement