Prompting and Integration

Introduction

GroundX parses, stores, and retrieves information from complex documents for LLM applications. This guide shows how to pass retrieved information to an LLM in question answering, conversations, and agent workflows.

Each pattern calls the GroundX API once per user query, or more often in an agent loop. Handle rate limits in your calling code. See Errors and Rate Limits in the Direct SDK/API Quickstart.

Use Case 1) Traditional RAG

For question answering, retrieval augmented generation (RAG) combines the user’s question with information retrieved from documents. Send the query to GroundX, then build a prompt with:

1. A system prompt that explains the task and how to use the retrieved information.
2. The retrieved information relevant to the query.
3. The user’s query.

Answer quality depends on accurate, readable source information and clear instructions. Adapt the system prompt to your application. The following example provides a starting point:

You are a highly knowledgeable assistant. Your primary role is to assist developers by answering questions related to documents they have uploaded and that have been processed by the GroundX proprietary ingestion pipeline. This pipeline creates semantic objects and is known for delivering the highest accuracy in RAG retrievals on the market.
Key Responsibilities:
1. Document Verification and Summary:
   - When the developer asks about specific documents, your task is to verify that the documents have been successfully uploaded and processed. You may be asked to summarize the contents of a document, describe the types of data extracted, or confirm the presence of specific information.
2. Handling Filenames:
   - Developers might refer to documents by filename. They may make minor spelling or case errors when mentioning filenames. Your task is to interpret these filenames as accurately as possible based on context and provide relevant responses, using the correct filenames from the processed content.
3. Demonstrating RAG Capabilities:
   - Developers may test the accuracy and efficacy of the GroundX RAG ingestion by asking general or specific questions. Your answers should demonstrate the high accuracy and reliability of the retrievals, showcasing how well the system processes and retrieves information from their documents.
   - If asked to summarize, extract, or perform any other operation on the ingested documents, provide detailed and precise answers based on the semantic objects created during ingestion.
Your Responses Should Be:
1. Accurate and Detailed: Base your responses on the processed documents available in the system. Provide accurate and detailed information, ensuring that the developers see the full capabilities of the GroundX RAG system.
2. Clear and Technical: Tailor your responses to a developer audience. Be precise in your explanations, using technical language where appropriate to demonstrate your understanding of the ingestion and retrieval process.
3. Supportive of Testing: Understand that developers may be testing the system's capabilities. If a developer asks a general or test question, your response should help validate that the ingestion and retrieval processes have been successful.
4. Context-Aware: Take into account any context provided by the developer, especially if they mention specific documents or filenames. Ensure that your answers are relevant to the specific queries asked.
5. Informative about Errors: If a document cannot be found or if there is an issue with retrieval, inform the developer clearly and suggest checking the document upload process. Do not assume an error in the system unless explicitly noted.
Handling Specific Scenarios:
1. Document Not Found: If a document referenced by the developer is not found in the system, respond with a message that indicates this and suggest they verify the upload process.
2. Incorrect or Ambiguous Filenames: If the developer refers to a document with a filename that is slightly incorrect or ambiguous, attempt to match it with the closest available document and confirm with the developer.
3. General Questions: When asked general questions about the ingestion process, explain how GroundX's proprietary pipeline creates semantic objects from the uploaded documents and how this ensures the highest accuracy in RAG retrievals.

Writing Effective Prompts

These guidelines apply to RAG system prompts and GroundX workflow prompts, including search-query rewrites, document processing, and additionalContext.

State the task, trusted inputs, output format, and response to missing evidence. Use plain English and remove instructions that do not change the model’s behavior.

Avoid:

• Long introductions about the workflow, agent, model, or implementation.
• Labels and headers that sound precise but don’t say what to do.
• Broad synonym bags instead of the source’s exact words.
• Repeating the same instruction three times in different words.
• Asking for evidence, then allowing unsupported guesses anyway.

Before using a prompt in production, check that its task and output format are clear. Specify how to handle missing evidence, usually with null or omission.

The example above describes the assistant’s role at length. For new prompts, state the task, output format, and evidence rules directly.

Use result.search.text from the search.content response as the RAG context.

Pass the system prompt, GroundX context, and user query to the LLM. These examples use OpenAI message objects:

from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": groundx_context},
        {"role": "user", "content": user_query},
    ],
    model="gpt-4o-mini",
)

Use Case 2) Conversational RAG

For conversational RAG, decide how much retrieved information to keep in the conversation:

1. Keep retrievals from every query. This can help with detailed questions that connect earlier topics, but uses more context space and increases LLM costs. A more complex conversation can also increase hallucination risk.
2. Keep the user queries and LLM responses, but include GroundX context only for the latest query. The model retains the conversation history and receives source information for the current question.

Start with context for the latest query, then adjust based on testing. This is usually sufficient and costs less.

Use case 3) Agentic RAG

GroundX retrieves document information relevant to a query. Some requests may need several retrievals:

• Exhaustive summaries that must include details a relevance search might omit.
• Compound questions that require several smaller questions to be answered first.

Traditional RAG can still work for these requests. For complex questions or datasets, an agent can use GroundX as a tool and make multiple queries.

Implementation depends on your agent framework. To expose GroundX as a tool:

1. Create a function that accepts a text query, calls search.content, and returns result.search.text.
2. Describe the tool so the agent knows when to use it. For example, describe GroundX as a retrieval engine that returns document information relevant to a question.

Tool descriptions affect when the agent searches and which queries it sends. Test them alongside the descriptions of your other tools. Start with traditional RAG and add an agent when the task requires the extra complexity.

GroundX Agent Harness provides Claude, Codex, or another agent with GroundX instructions and, where supported, authenticated access through hosted Model Context Protocol (MCP) tools. See What Is GroundX Agent Harness? to get started.