How to Build a Persistent Knowledge Base Your AI Can Always Access

Every time you start a new conversation with ChatGPT or Claude, you lose everything from the last one. The project context, the decisions you made, the constraints you discussed. Gone. So you spend the first few minutes of every session re-explaining the same background, and the AI still doesn't have the full picture.

There's a fix for this, and it doesn't require any code. You build an external knowledge base that your AI can always access, regardless of conversation length or session boundaries. This tutorial walks through exactly how to do it.

What a persistent AI knowledge base actually is

The concept is simple: instead of storing your knowledge inside AI conversations (where it gets lost, compacted, or summarized), you store it in documents that live outside the AI. The AI connects to these documents and reads them on demand.

When you start a new conversation on Tuesday, the AI can read the same documents it read on Monday. When your project status changes, you update the document, and the next conversation automatically gets the current version. No re-explaining, no copy-pasting, no "as I mentioned before."

This pattern is called context engineering. It's the practice of structuring and maintaining the information that AI tools need to be useful, rather than hoping the AI retains it on its own.

What to include in your knowledge base

Before setting up any tools, think about what you re-explain most often. That's your highest-value starting content.

Company and team context. Your company name, what it does, your role, your team structure, key stakeholders. This is the context you type into every new conversation: "I work at Meridian Health, we build EHR integrations for rural clinics, my team has 6 engineers and 2 PMs..."

Product and project details. Current product specs, feature roadmaps, technical architecture decisions, active project status. The AI needs this to give advice that actually applies to your situation instead of generic suggestions.

Style and preferences. Your writing style guide, tone preferences, formatting standards, terminology choices. If you always have to say "don't use em dashes" or "we call it a 'workspace' not a 'dashboard,'" put it in a document.

Meeting notes and decisions. The decisions your team has made, the reasoning behind them, and the action items that came out of discussions. This is often the most valuable category because decisions are easy to forget and hard to reconstruct.

Recurring reference material. Anything you look up repeatedly: pricing tiers, API rate limits, compliance requirements, competitor comparisons, onboarding checklists.

You don't need to write all of this on day one. Start with two or three documents covering your most frequently re-explained context. You can add more over time.

Step 1: Write your knowledge in markdown

Markdown is the native language of AI tools. ChatGPT, Claude, Gemini, and every other major AI assistant reads and writes markdown. When your knowledge base is in markdown, there's no format conversion and no information loss between your documents and the AI.

If you already write in markdown, you're set. If not, the syntax is minimal:

# Company Overview

**Company:** Meridian Health
**Industry:** Healthcare technology (EHR integrations)
**Founded:** 2019
**Team size:** 14 (6 engineering, 2 PM, 3 sales, 2 ops, 1 design)

## What we build

Meridian provides EHR integration middleware for rural and
community health clinics. Our platform connects 12 EHR systems
to a unified API that clinic staff use for patient data access,
scheduling, and billing.

## Current priorities (Q1 2026)

- HIPAA compliance audit (deadline: March 15)
- V2 API launch (beta: February, GA: April)
- Expand to 400 facilities by end of Q2

Use headings to organize sections. Use bold for key terms. Use lists for items the AI might need to reference individually. The AI parses these structural elements, so well-organized documents produce better responses.

Step 2: Organize your documents in Unmarkdown

Unmarkdown™ is where your knowledge base lives. Create a free account and start adding your documents.

Why Unmarkdown™ specifically? Three reasons.

It's a markdown-native document platform. You write in markdown, the AI reads markdown. There's no translation layer, no PDF parsing, no file format conversion. What you write is exactly what the AI gets.

It has a built-in MCP server. MCP (Model Context Protocol) is how AI tools connect to external services. Unmarkdown™ provides a ready-to-use MCP endpoint, so you don't need to set up any infrastructure. Connect your AI tool and it works.

Your documents are useful beyond the AI connection. With 62 templates for visual presentation, 12 AI editing actions for polishing content, and publishing for sharing with your team, your knowledge base documents aren't locked in a developer tool. They're real documents you can format, share, and publish.

Here's a practical starting structure:

Create each as a separate document in Unmarkdown™ and organize them into folders. You might have a "Company" folder for context documents, a "Projects" folder for active work, and a "Meetings" folder for weekly notes. When creating documents through Claude, just say "create in my Meetings folder" and the document lands in the right place. Separate documents are better than one massive file because the AI can read only what it needs for a given question, and you can update individual documents without touching the rest.

Step 3: Connect to Claude via MCP

This is the step that turns static documents into a live AI knowledge base. You connect Unmarkdown™ to Claude so that every conversation has access to your documents.

There are three ways to connect, depending on which Claude client you use.

Claude on the web (claude.ai). Go to Settings, then Integrations, and add Unmarkdown™. This uses OAuth, so you authenticate once and the connection persists. No API key needed.

Claude Desktop. Add a configuration entry to your Claude Desktop config file pointing to the Unmarkdown™ MCP server. This requires an API key from your Unmarkdown™ account.

Claude Code (terminal). Run the claude mcp add command with the Unmarkdown™ server URL. One command and you're connected.

The Claude integration guide has the exact configuration for each method, including the config file paths and JSON format.

Once connected, Claude has access to seven tools for working with your documents:

1. list_documents - See all documents in your knowledge base, or filter by folder
2. get_document - Read the full content of a specific document
3. create_document - Add new documents, optionally placing them directly in a folder
4. update_document - Modify existing documents or move them between folders
5. publish_document - Share a document publicly or via link
6. convert_markdown - Format content for Google Docs, Slack, Word, Email, OneNote, or Plain Text
7. get_usage - Check your API usage for the billing period

You can reference folders by name (case-insensitive). Say "create in my Projects folder" or "list everything in the Team folder" and Claude handles the rest.

You don't need to memorize these. Claude discovers the available tools automatically when the MCP connection is active.

Step 4: Use it

Here's where it gets practical. With your knowledge base connected, your AI conversations become dramatically more useful.

Ask questions grounded in your actual context:

"Read my product roadmap and suggest Q2 priorities. We shipped the V2 API beta in February. What should we focus on for GA?"

Claude reads your roadmap document, sees the full list of planned features and timelines, and gives advice that accounts for what's actually in progress rather than guessing based on generic product management frameworks.

Keep documents current without manual editing:

"Update the meeting notes document with today's standup. We decided to push the HIPAA audit deadline to March 30. Sarah is taking over the API documentation. The billing integration is blocked on the Stripe webhook issue."

Claude opens your meeting notes document, adds today's entry with the decisions and action items, and saves it. Tomorrow's conversation will see the updated notes.

Reference past decisions:

"What did we decide about the pricing change last week? Check the meeting notes."

Instead of scrolling through Slack messages or searching your email, the AI reads the document where you (or it) recorded the decision and gives you the answer with full context.

Prepare formatted documents from your knowledge:

"Read my project status document and create a one-page executive summary. Convert it for Google Docs."

Claude reads your detailed project status, writes a concise summary, and converts it to a format that pastes cleanly into Google Docs with proper headings, tables, and formatting.

Build on previous work:

"Read the competitive analysis I wrote last month. Add a new section comparing our API pricing to Competitor X's new tier they announced yesterday."

The AI starts with your existing analysis rather than generating a new comparison from scratch. It preserves your structure, your framing, and your existing conclusions while adding the new section you requested.

Tips for maintaining your knowledge base

Update after decisions, not on a schedule. The most useful knowledge bases are updated in real time: after meetings, after decisions, after project milestones. A weekly bulk update is better than nothing, but it means your AI is working with stale context for most of the week.

Let the AI do the updates. You don't have to manually edit every document. Ask Claude to update the document based on what you tell it. "Add to the project status doc: the billing feature shipped today, we're moving to the testing phase for SSO."

Keep documents focused. One document per topic is better than one massive document covering everything. The AI can read multiple documents in a single conversation, and focused documents are easier to keep current.

Write for the AI, not just for yourself. Include context that you take for granted but the AI wouldn't know. Instead of "the main product," write "Meridian Connect (our EHR integration platform)." Instead of "the usual constraints," write "HIPAA compliance requirement, 99.9% uptime SLA, 340 clinic customer base."

Review quarterly. Documents drift. Priorities shift, team members change, products evolve. Set a quarterly reminder to read through your knowledge base and archive anything that's no longer accurate.

Context engineering without the engineering

The term "context engineering" can sound intimidating, like it requires a technical background or custom code. It doesn't. What it requires is intentionality about what information your AI has access to.

Every time you re-explain your company to an AI, that's a signal that the information should be in a document. Every time the AI gives generic advice that ignores your specific constraints, that's a signal that the constraints should be written down. Every time you say "as I mentioned in our last conversation," that's a signal that the information should persist outside of conversations.

Building a knowledge base is the simplest form of context engineering: write down what matters, put it where the AI can find it, and keep it updated. No retrieval pipelines, no vector databases, no embedding models. Just documents and a connection.

The full integration guide, including setup instructions for Claude, ChatGPT, and other MCP-compatible clients, is at /docs/integrations/overview. For the developer API, see the developers page.

Related reading

• Why Your AI Keeps Forgetting (And the Permanent Fix)
• How to Use Claude's MCP Tools to Publish Documents
• The AI Formatting Problem Nobody Talks About (And How to Fix It)
• Obsidian vs Unmarkdown: When You Need Documents, Not Digital Gardens
• The Document Your AI Never Forgets: Persistent Knowledge with MCP
• Internal Documentation Made Simple: Markdown for Small Teams