Corrective RAG (CRAG) and Self-RAG | RAG Techniques | Nahean Fardous

Corrective RAG (CRAG) and Self-RAG represent a new frontier in "Refined Inference." Instead of blindly trusting a vector store, these architectures introduce an Evaluator to grade every retrieved document. If the document is irrelevant, CRAG triggers a fallback search (e.g., a web search or a larger corpus lookup). Self-RAG takes this further by generating "Reflection Tokens" that judge whether the final answer is grounded in the provided facts, allowing for real-time self-correction during the token generation phase.

Engineering Trade-offs

Feature	Technical Impact
Pros	Eliminates hallucination by discarding irrelevant data; provides dynamic fail-safes for out-of-distribution queries.
Cons	Significant latency impact (2 extra LLM calls); complexity in orchestrating fallback search providers.

Architecture Overview

The Corrective RAG data flow introduces a "Gatekeeper" to the RAG pipeline:

       [USER Query]
             │
     ┌───────▼───────┐
     │ Initial Search│ (Find Top 5 Candidates)
     └───────┬───────┘
             │
    ┌────────▼────────┐
    │ Relevance Grader│ (LLM: "Is this chunk helpful?")
    └──────┬──┬───────┘
           │  │
    ┌──────▼──▼──────┐
    │ [YES]: Use Chunks │ [NO]: TRIGGER FALLBACK (e.g. WebSearch)
    └──────┬─────────┘ ┌─────┴────────┐
           │           │ [New Data]   │
    ┌──────▼───────────▼─────┐
    │ Grounded Generation    │ (Final Answer based on validated data)
    └────────────────────────┘

Implementation Walkthrough

The following steps trace the process of grading a retrieved document and triggering a corrective action using the Vercel AI SDK.

Neural Relevance Grading

We create a grading function using a distilled, fast model (GPT-4o-mini). This function provides a binary yes/no score for every retrieved snippet.

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
 
const query = "What is the consensus mechanism for Phoenix?";
const chunk = "The document mainly discusses marketing strategy for the project.";
 
const { text: grade } = await generateText({
  model: openai('gpt-4o-mini'),
  prompt: `Grade if this chunk answers the query: "${query}"? Chunk: "${chunk}". List ONLY "yes", "no", or "partial".`,
});
 
console.log(`[SYS] Relevance Grading Result: ${grade.toLowerCase()}`);

Corrective Fallback Execution

If the grader emits a no, we trigger an alternate search tool. This ensures the agent does not answer based on the irrelevant "marketing" chunk.

// SIMULATED Corrective Fallback (Using a different index or web search)
const fallbackDocs = ["Corrective Search SUCCESS: Source document retrieved."];
console.log(`[SEARCH] Corrective data retrieved. Proceeding to Generation.`);

Complete Production Script

This script implements a baseline Corrective RAG pipeline to illustrate gated retrieval behavior.

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import 'dotenv/config';
 
/**
 * Intelligent Corrective RAG Pipeline.
 */
async function correctiveRag() {
  const logger = {
    info: (msg) => console.log(`[INFO]  ${new Date().toISOString()} | ${msg}`),
    timer: (label) => console.time(`[TIME]  ${label}`),
  };
 
  const query = "Security protocol for project Phoenix?";
  const initialDocs = ["Phoenix marketing is great for users."]; // IRRELEVANT DOC
 
  try {
    logger.info(`Source Query: "${query}"`);
    
    // 1. Gated Evaluation
    logger.timer("grading");
    const { text: grade } = await generateText({
      model: openai('gpt-4o-mini'),
      prompt: `Scale 1-10: Relevance of "${initialDocs[0]}" to "${query}". Output only the number.`,
    });
    console.timeEnd("grading");
 
    const score = parseInt(grade.trim());
    logger.info(`Initial Chunk Score: ${score}/10`);
 
    let finalContext = initialDocs;
 
    // 2. Corrective Fallback (If score is below threshold)
    if (score < 5) {
      logger.info(`CRITICAL Failure: Relevance score ${score} is below threshold. Corrective Search required.`);
      finalContext = ["REFINED SOURCE: Protocol Phoenix uses RSA-2048 with Ed25519 signatures."];
      logger.info(`Fallback data retrieved: "${finalContext[0].slice(0, 30)}..."`);
    }
 
    // 3. Final Answer Generation
    const { text: answer } = await generateText({
      model: openai('gpt-4o-mini'),
      system: "Answer using only provided context.",
      prompt: `Context: ${finalContext[0]}\n\nQuestion: ${query}`,
    });
 
    console.log(`\n--- CORRECTIVE RAG RESPONSE ---\n${answer}\n-------------------------------\n`);
 
  } catch (error) {
    console.error(`[FATAL] Pipeline failed: ${error.message}`);
  }
}
 
correctiveRag();

Summary of Impact

Metric	Naive RAG	Corrective RAG (CRAG)
Trust Factor	Low (May use bad data)	High (Discards invalid context)
Precision (P@1)	Moderate (60-75%)	Very High (90-95%)
System Latency	Baseline (~2s)	Baseline + ~1.5s (Grading Step)

When to Use This

For high-stakes document search where accuracy is more important than speed.
When your vector store search is returning irrelevant results ("Near Misses").
When you already have multiple search providers (e.g., Vector DB + Web Search).

Feature

Technical Impact

Pros

Eliminates hallucination by discarding irrelevant data; provides dynamic fail-safes for out-of-distribution queries.

Cons

Significant latency impact (2 extra LLM calls); complexity in orchestrating fallback search providers.

Architecture Overview

The Corrective RAG data flow introduces a "Gatekeeper" to the RAG pipeline:

       [USER Query]
             │
     ┌───────▼───────┐
     │ Initial Search│ (Find Top 5 Candidates)
     └───────┬───────┘
             │
    ┌────────▼────────┐
    │ Relevance Grader│ (LLM: "Is this chunk helpful?")
    └──────┬──┬───────┘
           │  │
    ┌──────▼──▼──────┐
    │ [YES]: Use Chunks │ [NO]: TRIGGER FALLBACK (e.g. WebSearch)
    └──────┬─────────┘ ┌─────┴────────┐
           │           │ [New Data]   │
    ┌──────▼───────────▼─────┐
    │ Grounded Generation    │ (Final Answer based on validated data)
    └────────────────────────┘

Implementation Walkthrough

The following steps trace the process of grading a retrieved document and triggering a corrective action using the Vercel AI SDK.

Neural Relevance Grading

We create a grading function using a distilled, fast model (GPT-4o-mini). This function provides a binary yes/no score for every retrieved snippet.

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
 
const query = "What is the consensus mechanism for Phoenix?";
const chunk = "The document mainly discusses marketing strategy for the project.";
 
const { text: grade } = await generateText({
  model: openai('gpt-4o-mini'),
  prompt: `Grade if this chunk answers the query: "${query}"? Chunk: "${chunk}". List ONLY "yes", "no", or "partial".`,
});
 
console.log(`[SYS] Relevance Grading Result: ${grade.toLowerCase()}`);

Corrective Fallback Execution

If the grader emits a no, we trigger an alternate search tool. This ensures the agent does not answer based on the irrelevant "marketing" chunk.

// SIMULATED Corrective Fallback (Using a different index or web search)
const fallbackDocs = ["Corrective Search SUCCESS: Source document retrieved."];
console.log(`[SEARCH] Corrective data retrieved. Proceeding to Generation.`);

Complete Production Script

This script implements a baseline Corrective RAG pipeline to illustrate gated retrieval behavior.

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import 'dotenv/config';
 
/**
 * Intelligent Corrective RAG Pipeline.
 */
async function correctiveRag() {
  const logger = {
    info: (msg) => console.log(`[INFO]  ${new Date().toISOString()} | ${msg}`),
    timer: (label) => console.time(`[TIME]  ${label}`),
  };
 
  const query = "Security protocol for project Phoenix?";
  const initialDocs = ["Phoenix marketing is great for users."]; // IRRELEVANT DOC
 
  try {
    logger.info(`Source Query: "${query}"`);
    
    // 1. Gated Evaluation
    logger.timer("grading");
    const { text: grade } = await generateText({
      model: openai('gpt-4o-mini'),
      prompt: `Scale 1-10: Relevance of "${initialDocs[0]}" to "${query}". Output only the number.`,
    });
    console.timeEnd("grading");
 
    const score = parseInt(grade.trim());
    logger.info(`Initial Chunk Score: ${score}/10`);
 
    let finalContext = initialDocs;
 
    // 2. Corrective Fallback (If score is below threshold)
    if (score < 5) {
      logger.info(`CRITICAL Failure: Relevance score ${score} is below threshold. Corrective Search required.`);
      finalContext = ["REFINED SOURCE: Protocol Phoenix uses RSA-2048 with Ed25519 signatures."];
      logger.info(`Fallback data retrieved: "${finalContext[0].slice(0, 30)}..."`);
    }
 
    // 3. Final Answer Generation
    const { text: answer } = await generateText({
      model: openai('gpt-4o-mini'),
      system: "Answer using only provided context.",
      prompt: `Context: ${finalContext[0]}\n\nQuestion: ${query}`,
    });
 
    console.log(`\n--- CORRECTIVE RAG RESPONSE ---\n${answer}\n-------------------------------\n`);
 
  } catch (error) {
    console.error(`[FATAL] Pipeline failed: ${error.message}`);
  }
}
 
correctiveRag();

Metric

Naive RAG

Corrective RAG (CRAG)

Trust Factor

Low (May use bad data)

High (Discards invalid context)

Precision (P@1)

Moderate (60-75%)

Very High (90-95%)

System Latency

Baseline (~2s)

Baseline + ~1.5s (Grading Step)