> ## Documentation Index
> Fetch the complete documentation index at: https://docs.runflow.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge (RAG)

> Semantic search in vector knowledge bases

The **Knowledge** module (also called RAG) manages semantic search in vector knowledge bases.

## Standalone Knowledge Manager

```typescript theme={null}
import { Knowledge } from '@runflow-ai/sdk';

const knowledge = new Knowledge({
  vectorStore: 'support-docs',
  k: 5,
  threshold: 0.7,
});

// Basic search
const results = await knowledge.search('How to reset password?');

results.forEach(result => {
  console.log(result.content);
  console.log('Score:', result.score);
});

// Get formatted context for LLM
const context = await knowledge.getContext('password reset', { k: 3 });
console.log(context);
```

## Agentic RAG in Agent

When RAG is configured in an agent, the SDK automatically creates a `searchKnowledge` tool that the LLM can decide when to use. This is more efficient than always searching, as the LLM only searches when necessary.

```typescript theme={null}
const agent = new Agent({
  name: 'Support Agent',
  instructions: 'You are a helpful support agent.',
  model: openai('gpt-4o'),
  rag: {
    vectorStore: 'support-docs',
    k: 5,
    threshold: 0.7,

    // Custom search prompt - guides when to search
    searchPrompt: `Use searchKnowledge tool when user asks about:
- Technical problems
- Process questions
- Specific information

Don't use for greetings or casual chat.`,

    toolDescription: 'Search in support documentation for solutions',
  },
});

// Agent automatically has 'searchKnowledge' tool
// LLM decides when to search (not always - more efficient!)
const result = await agent.process({
  message: 'How do I reset my password?',
});
```

## Multiple Vector Stores

```typescript theme={null}
const agent = new Agent({
  name: 'Advanced Support Agent',
  instructions: 'Help users with multiple knowledge bases.',
  model: openai('gpt-4o'),
  rag: {
    vectorStores: [
      {
        id: 'support-docs',
        name: 'Support Documentation',
        description: 'General support articles',
        threshold: 0.7,
        k: 5,
        searchPrompt: 'Use search_support-docs when user has technical problems or questions',
      },
      {
        id: 'api-docs',
        name: 'API Documentation',
        description: 'Technical API reference',
        threshold: 0.8,
        k: 3,
        searchPrompt: 'Use search_api-docs when user asks about API endpoints or integration',
      },
    ],
  },
});
```

## Managing Documents

Add text documents:

```typescript theme={null}
import { Knowledge } from '@runflow-ai/sdk';

const knowledge = new Knowledge({
  vectorStore: 'support-docs',
});

// Add a text document
const result = await knowledge.addDocument(
  'How to reset password: Go to settings > security > reset password',
  {
    title: 'Password Reset Guide',
    category: 'authentication',
    version: '1.0'
  }
);

console.log('Document added:', result.documentId);
```

Upload files:

```typescript theme={null}
import * as fs from 'fs';

// Node.js - Upload from file system
const fileBuffer = fs.readFileSync('./manual.pdf');

const result = await knowledge.addFile(
  fileBuffer,
  'manual.pdf',
  {
    title: 'User Manual',
    mimeType: 'application/pdf',
    metadata: {
      department: 'Support',
      version: '2.0'
    }
  }
);
```

## Async Ingestion for Large Files

<Info>
  Available since SDK `1.3.2` (platform July 2026). For small files `addFile` still works — async ingestion is the recommended path for anything big.
</Info>

`addFile` processes the file synchronously — fine for a manual or an FAQ, but a 50k-row catalog would hold the HTTP request open for minutes. `ingestFile` uploads the file, returns in seconds with a job id, and the platform embeds everything in the background with batched embeddings and checkpoint resume (if a worker restarts mid-job, ingestion continues from where it stopped instead of starting over).

```typescript theme={null}
import * as fs from 'fs';
import { Knowledge } from '@runflow-ai/sdk';

const knowledge = new Knowledge({ vectorStore: 'product-catalog' });

// Fire-and-forget: returns as soon as the job is accepted (HTTP 202)
const { jobId } = await knowledge.ingestFile(
  fs.readFileSync('./catalog.csv'),
  'catalog.csv',
  { mimeType: 'text/csv' }
);

// Poll whenever you want
const job = await knowledge.getIngestionJob(jobId);
console.log(job.status, `${job.processedChunks}/${job.totalChunks}`);
```

Or block until it finishes:

```typescript theme={null}
const result = await knowledge.ingestFile(
  fs.readFileSync('./catalog.csv'),
  'catalog.csv',
  {
    mimeType: 'text/csv',
    waitForCompletion: true,
    onProgress: (job) => console.log(`${Math.round(job.progress * 100)}%`),
  }
);

console.log('Indexed', result.job?.processedChunks, 'chunks');
```

### CSV: one document per row

CSV files are ingested **one document per row** — ideal for product catalogs and structured data. Each row becomes a searchable `Column: value` document. You can control which columns are embedded and which go to metadata:

```typescript theme={null}
await knowledge.ingestFile(fs.readFileSync('./catalog.csv'), 'catalog.csv', {
  mimeType: 'text/csv',
  csv: {
    delimiter: ';',                       // sniffed automatically when omitted
    contentColumns: ['name', 'brand'],    // embedded (default: all columns)
    metadataColumns: ['sku', 'price'],    // copied to each document's metadata
  },
});
```

### Data hygiene

Optional cleanup applied server-side before embedding — useful when the source data carries HTML, URLs, or placeholder values that hurt search quality:

```typescript theme={null}
await knowledge.ingestFile(file, 'export.csv', {
  hygiene: {
    stripHtml: true,            // strip tags, decode entities
    removeUrls: true,           // drop URLs from content
    dropEmptyValues: true,      // remove "-", "n/a", "..." placeholder fields
    normalizeWhitespace: true,  // collapse repeated spaces/newlines
    dedupeUnits: true,          // drop duplicate rows/chunks
  },
});
```

### Job status

`getIngestionJob(jobId)` (and the `onProgress` callback) return:

| Field                             | Description                                                    |
| --------------------------------- | -------------------------------------------------------------- |
| `status`                          | `queued` → `processing` → `completed` \| `failed`              |
| `progress`                        | Completion ratio `0..1` (0 while the worker is still planning) |
| `processedChunks` / `totalChunks` | Embedded units vs. total planned                               |
| `documentId`                      | Set when `completed` — the parent document id                  |
| `error`                           | Set when `failed`                                              |

`waitForCompletion` throws if the job fails or the timeout (default 30 min) elapses — on timeout the job keeps running server-side and you can keep polling.

## Metadata Filters

Filter search results by document metadata using the `filters` option. Each key maps to a metadata field.

**Simple equality filter:**

```typescript theme={null}
const results = await knowledge.search('reset password', {
  k: 5,
  filters: {
    category: 'authentication',
    language: 'en',
  },
});
```

**Custom operators (JSONB):**

Pass an object with `value` and `operator` for non-equality comparisons:

```typescript theme={null}
const results = await knowledge.search('pricing plans', {
  k: 10,
  filters: {
    version: { value: "2.0", operator: '>=' },
    status: 'published',
  },
});
```

Supported operators: `=` (default), `!=`, `>`, `>=`, `<`, `<=`, `@>` (contains), `<@` (contained by).

Filters also work in agent RAG config:

```typescript theme={null}
const agent = new Agent({
  name: 'Support Agent',
  model: openai('gpt-4o'),
  rag: {
    vectorStore: 'support-docs',
    k: 5,
    threshold: 0.7,
    filters: {
      department: 'support',
      status: 'published',
    },
  },
});
```

## RAG Interceptor & Rerank

**Interceptor - Filter & Transform Results:**

```typescript theme={null}
const agent = new Agent({
  name: 'Smart Agent',
  model: openai('gpt-4o'),
  rag: {
    vectorStore: 'docs',
    k: 10,
    
    // Interceptor: Customize results before LLM
    onResultsFound: async (results, query) => {
      // Filter sensitive data
      const filtered = results.filter(r => !r.metadata?.internal);
      
      // Enrich with external data
      const enriched = await Promise.all(
        filtered.map(async r => ({
          ...r,
          content: `${r.content}\n\nSource: ${r.metadata?.url}`,
        }))
      );
      
      return enriched;
    },
  },
});
```

**Rerank Strategies:**

* `reciprocal-rank-fusion` - Standard RRF algorithm
* `score-boost` - Boost results containing keywords
* `metadata-weight` - Weight by metadata field value
* `custom` - Custom scoring function

## Next Steps

<CardGroup cols={2}>
  <Card title="Agents" icon="robot" href="/core-concepts/agents">
    Learn about agents
  </Card>

  <Card title="Use Cases" icon="code" href="/use-cases/customer-support-rag">
    See RAG examples
  </Card>
</CardGroup>