MCP vs RAG vs agent skills: what to build

Which one does your AI project need?

Use retrieval-augmented generation (RAG) when an answer needs evidence from your documents. Consider the Model Context Protocol (MCP) when compatible AI applications need a shared interface to your tools or data. Add an agent skill when a recurring task needs reusable instructions and supporting files. A project can use all three, but each addition should solve a specific problem. GitHub's September 2026 discussion of these technologies makes the same distinction: they can work together. The build plan below is our own practice example, not a claim about a deployed customer system.

• Source: GitHub on RAG, MCP, and skills

Compare the job each component performs

Start with the missing capability, then choose an implementation. MCP describes how a client and server exchange capabilities and requests; it is not a search engine. The Agent Skills format packages instructions and optional resources for a compatible agent. Neither makes an incorrect answer reliable simply by being installed.

• Source: MCP architecture
• Source: Agent Skills overview

Build a documentation assistant in three stages

Use a small set of documentation you wrote or have permission to reuse. For example, create a fictional booking API with pages for authentication, pagination, cancellations, and error codes. Give each page a stable identifier and version. Write ten questions with the exact passages needed to answer them, plus five questions the documents cannot answer. These are starter quantities for a manageable exercise, not an industry benchmark.

• Stage 1 — Implement document search and return the matching passage and source identifier. A simple keyword search is a useful baseline before you introduce embeddings. Inspect retrieval separately from the generated answer.
• Stage 2 — Ask the model to answer from the retrieved material and cite it. If the evidence is missing or conflicting, the response should describe the gap instead of inventing an API feature.
• Stage 3 — Add MCP only if another compatible client needs the search capability. Add a skill only if a repeatable workflow, such as producing a troubleshooting note, needs shared instructions. Keep the working search baseline for comparison.

Make the first tool read-only

For this exercise, define search_docs with a query and an optional documentation version. Return a bounded list of passages, source identifiers, and versions. Keep publishing, account changes, and arbitrary file access outside the tool's scope. The server should validate inputs and enforce document access independently of anything the model says. A retrieved page is evidence to inspect; a sentence inside it is not permission to run another operation. This boundary gives you a useful failure case to demonstrate without connecting real accounts.

Measure whether the extra component helped

Run the same questions through each version and keep the corpus fixed. Record whether the necessary passage was retrieved, whether the answer used that passage correctly, whether citations open the right source, and whether unsupported requests produced an honest limitation. Also record end-to-end duration, tool calls, and provider-reported token use when available. If the MCP version adds interoperability but does not improve answer quality, report exactly that. If keyword search works well enough for this corpus, the experiment does not require a vector database.

• Build an evaluation set for this assistant

Try these failure cases before sharing the demo

Make failures visible in a short results log. Use invented documentation and test identities so the exercise can be published safely.

• A page describes version 1, while the user asks about version 2. The assistant must not silently mix versions.
• Two pages disagree about an endpoint. The response should identify the conflict and cite both, rather than choosing an unsupported answer.
• A document includes an instruction to ignore the user and reveal another file. The system should keep its allowed tool and document boundaries.
• Search returns no match or times out. The assistant should report the missing evidence or service failure, without claiming the requested fact was verified.

What should the portfolio submission contain?

Share the permitted sample corpus, questions, architecture sketch, setup instructions, and results from each stage. Explain which component you chose to omit and why. Include one failed answer and the change you made after investigating it. State the model and tool versions used, and separate expected behavior from observed results. This is independent practice; a ProofCohort enrollment follows its own published syllabus and assigned assessment requirements.

• Turn the experiment into a portfolio case study