Building RAG with Claude | Choosing Between Built-in Projects and Custom Pipelines

What Is RAG — Giving Claude Long-Term Memory

LLMs can only work with information that fits inside the context window. Trying to feed Claude a million characters worth of internal policies, product manuals, or past customer support logs would either cost a fortune in tokens or simply not fit in the window at all.

RAG solves this problem. With a "search first, then generate" architecture, only the fragments most relevant to the question are retrieved in real time and added to the context before being passed to Claude. The biggest advantage is that you don't have to stuff unnecessary text into the window — you can supply exactly the information needed, precisely when it's needed.

The official Claude Help Center defines RAG as "a technique where an AI model searches for relevant information from documents before generating a response" (source: Retrieval augmented generation (RAG) for projects | Claude Help Center). This reduces hallucinations (inaccurate answers) and enables Claude to deliver accurate responses grounded in real business data.

Built-in RAG in Claude Projects

The quickest option available is Claude Projects' built-in RAG. It is available on all plans — Free, Pro, Max, Team, and Enterprise.

The mechanism is straightforward. Simply upload files to a project, and Claude automatically searches them during the conversation using a "project knowledge search tool." No code, vector database, or embedding model is required.

There are three key benefits:

1. Expanded capacity: Store up to 10 times more content than a standard context window allows
2. Maintained accuracy: Retrieved fragments are passed directly into the context, preserving response quality equivalent to full-text processing
3. Fast responses: Only the relevant portions are retrieved, so even large knowledge bases respond quickly

How and When RAG Activates Automatically

RAG in Projects is not something you turn on manually — it activates automatically. When the amount of knowledge in a project approaches the context window limit, Claude automatically switches to search mode (source: Retrieval augmented generation (RAG) for projects | Claude Help Center).

From the user's perspective, this looks no different from a normal conversation. Claude tracks which documents it references behind the scenes, so no additional configuration is needed. Just add files and ask questions — you'll benefit from RAG automatically.

Supported file formats include PDF, Word, text, Markdown, and other major formats. Source code and CSV files can also be loaded. Storage limits per project vary by plan, with Max and Enterprise plans offering the highest capacity.

Building a Custom RAG Pipeline

When you need finer control, you can build a custom pipeline by combining the Claude API with a vector database. The typical build process follows four steps:

1. Index creation: Split documents into chunks, vectorize them using an embedding model, and store them in a vector database
2. Query transformation: Vectorize the user's question using the same embedding model
3. Similarity search: Retrieve the most similar chunks based on vector space similarity (e.g., cosine similarity)
4. Generation: Send the retrieved chunks along with the original question to the Claude API to generate a response

With the Claude API, the simplest pattern is to inject context into the system prompt. Here is a minimal Python implementation:

import anthropic

client = anthropic.Anthropic()

def rag_query(user_question: str, context_chunks: list[str]) -> str:
    context = "\n\n".join(context_chunks)
    message = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        system=f"以下のコンテキストを参考に回答してください。\n\n{context}",
        messages=[{"role": "user", "content": user_question}]
    )
    return message.content[0].text

Pass the similar fragments retrieved from the vector database as context_chunks. For implementation details, refer to the official Anthropic documentation.

Vector Database Options

A vector database is essential for a custom pipeline, and several options are available. Choosing by scale, cost, and operational overhead makes the decision straightforward.

For production environments, cloud services like Pinecone or Weaviate tend to offer the most stability. A common pattern is to run Chroma locally during the prototype stage and migrate when scaling up. It is also worth noting that when Claude integrates with Supabase via MCP, pgvector can be used directly.

Best Practices for Chunking Strategy

The most critical factor affecting RAG accuracy is the granularity of chunking (document splitting). Chunks that are too large introduce irrelevant information; chunks that are too small lose context.

Here are four representative splitting strategies:

1. Fixed-size splitting: Split uniformly at around 500–1,000 tokens. Easy to implement with a good balance — the best place to start
2. Semantic boundary splitting: Split at paragraphs, sections, or H2 headings. Improves accuracy when document structure is clear
3. Overlap: Add 50–100 token overlaps between chunks to prevent context from being cut off
4. Parent retrieval (small → large): Search using small chunks, then pass the surrounding larger chunk to Claude. Excellent for balancing accuracy and context

Passing 3–10 chunks to Claude is a realistic target. Too many pollutes the context; too few produces thin responses.

Managed RAG Services: A Third Option

Sitting between Projects and fully custom solutions are managed RAG services. These automatically sync with your company documents and external data sources, enabling production-grade RAG without writing code.

Estimated setup times are: 5 minutes for Claude Projects, 10–15 minutes for managed services, and several weeks to months for fully custom builds (source: RAG for Claude: 3 Ways to Add Your Business Data | Context Link). This option suits teams like content or customer support where data updates frequently but code management is not feasible.

Choosing by Use Case

Organizing the three options by use case makes the decision easier.

Choose Claude Projects when:

• You want non-technical staff to use a knowledge base or manual without any developer involvement
• You need to launch quickly as a PoC or internal tool
• Data updates infrequently and manual uploads are sufficient

Choose a Managed RAG Service when:

• You have multiple data sources that require automatic syncing
• You cannot write code but need more flexibility than Projects offers
• You want to integrate with external services like websites, blogs, or CRMs

Choose a Custom Pipeline when:

• Data changes in real time (inventory, news, CRM integration)
• You are embedding it into a publicly accessible service (with security requirements)
• You need custom ranking logic or hybrid search

Summary

Claude RAG options come in three tiers: "built-in Projects," "managed services," and "fully custom." If you need something up and running immediately without writing code, Projects is the fastest path. If you need to integrate into a production system with real-time data connectivity, build a custom pipeline.

By giving Claude your organization's knowledge through RAG, you can dramatically reduce hallucinations (generation of incorrect information) and deliver highly reliable responses. A practical approach is to start with Projects for internal knowledge, then migrate to a custom implementation if the requirements demand it.