> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.eyelevel.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.eyelevel.ai/_mcp/server.

## Introduction

Test RAG and agentic systems against real documents, application-specific questions, and reviewed answers. We use human evaluation to assess whether a system meets an application's needs.

## Our Core Testing Philosophy

Use human evaluation as the primary measure of answer quality:

* Select documents that represent the application.
* Define realistic questions and reviewed expected answers, also called ground truth Q/A pairs.
* Run the questions against the RAG system.
* Have people compare generated answers with the expected answers.

Automated question generation and LLM judges can speed up initial checks. They also have limits: generated questions can bias the evaluation set, and LLM judges can misjudge difficult answers. People should define the questions, review the expected answers, and score the results.

Human evaluation takes time. The datasets below provide a starting point for testing GroundX.

## Datasets

> This list is under development. More datasets are planned.

### Deloitte 1K

[github link](https://github.com/eyelevelai/code-samples/tree/main/rag-battles/rag-battle-2)

This dataset contains 1,000 pages of public Deloitte PDFs with dense information in varied formats. It tests how well a RAG system parses and represents complex multimodal content.

The dataset includes 92 questions with ground truth answers. Compare generated answers with these expected answers to measure performance.

See the [GroundX comparison with other RAG approaches](https://www.eyelevel.ai/post/most-accurate-rag) on this dataset.

### Deloitte 100K

[github link](https://github.com/eyelevelai/code-samples/tree/main/rag-battles/rag-battle-3)

This extends Deloitte 1K into four partitions. Each uses the same 92 Q/A pairs:

* **partition 0**: the original 1,000 pages.
* **partition 1**: the original pages plus 9,000 pages of erroneous documents, for 10,000 pages total.
* **partition 2**: the original pages plus erroneous documents, for 50,000 pages total.
* **partition 3**: the original pages plus erroneous documents, for 100,000 pages total.

The added erroneous and irrelevant documents test how well retrieval finds relevant information as the dataset grows.

See the [GroundX comparison with other RAG approaches](https://www.eyelevel.ai/post/do-vector-databases-lose-accuracy-at-scale) on this dataset.

## Defining your Own Dataset

Tooling to simplify dataset construction and testing is planned. For guidance, read:

[RAG Evaluation: Almost Everything You Need to Know](https://www.eyelevel.ai/post/how-to-test-rag-and-agents-in-the-real-world)

1. Start with 100 pages and 20 Q/A pairs curated by subject matter experts.
2. Use automation to save time, with human review. In our experience, LLM evaluation can be off by roughly 15% to 20%, producing false positives and false negatives. Short, simple answers reduce this problem.
3. Include the content formats and question types your application needs. Cover text, tables, and graphics, along with tasks such as entity extraction, summarization, and aggregation.
4. Scale up once the initial test performs well. We typically move to 100 questions and 1,000 pages, with more pages for enterprise evaluations. We have seen popular vector approaches lose accuracy with datasets of 10,000 pages.