Why AI Needs Memory

LLMs have a fundamental limitation: context windows are finite. Even the largest models can only see a limited amount of text at once. But real software projects span months or years, accumulating decisions, conventions, architecture changes, and lessons learned.

The memory system solves this by maintaining structured, persistent files that the AI loads selectively at the start of each conversation.

• 🧠 Persistence — Knowledge survives across conversations and sessions
• 📦 Structured — Not raw chat logs, but curated, organized knowledge
• 🎯 Selective — Only load what's relevant to the current task
• ⚡ Efficient — Respects context window limits through progressive loading

Two Types of Memory

The memory system distinguishes between two fundamentally different kinds of knowledge:

Memory Architecture

.memory/MEMORY_INDEX.md: The Always-Loaded File

This is the only memory file loaded at the start of every conversation. It is a compact summary (~30 lines) that tells the AI what the project is about and where to find more detail.

# MEMORY_INDEX
last_verified: 2026-03-28

## Project
- Name: MyApp
- Stack: Node.js 20, TypeScript, NestJS, PostgreSQL, React
- Architecture: Clean Architecture (domain → application → infrastructure)

## Key Files
- Architecture: .memory/semantic/architecture.md
- Conventions: .memory/semantic/conventions.md
- Codebase: .memory/semantic/codebase.md

## Current Sprint
- Sprint 4: "User Management"
- Goal: Complete user CRUD, roles, and permissions
- Stories: US-041 through US-048

## Recent Decisions
- ADR-012: Chose JWT over sessions for auth (2026-03-15)
- ADR-013: PostgreSQL over MongoDB for relational data (2026-03-20)

Progressive Loading Protocol

Memory is loaded in levels, not all at once. This respects context window limits and keeps the AI focused on what matters for the current task.

Level 1 Routing Table

Task Type	Load These Files
Any task	.memory/semantic/project.md, .memory/semantic/conventions.md (always)
Backend work	.memory/semantic/architecture.md, .memory/semantic/codebase.md, .memory/semantic/testing.md
Frontend work	.memory/semantic/architecture.md, .memory/semantic/codebase.md, .memory/semantic/domain/design.md
Bug fix	.memory/semantic/codebase.md, .memory/semantic/testing.md
DevOps	.memory/semantic/deployment.md, .memory/semantic/architecture.md
Planning	.memory/semantic/architecture.md, .memory/semantic/codebase.md

The Save Protocol

Not every conversation should update memory. The AI follows a strict protocol for when to save and when not to save.

Lazy Loading & Peek-then-Load

Lazy Loading

Don't load memory files until you actually need them. If a task only involves writing a unit test, you don't need the deployment docs or the design system specs.

Peek-then-Load

Before loading a full memory file, read just its frontmatter summary to decide if it's relevant:

# Step 1: Peek at frontmatter
Read first 10 lines of .memory/semantic/architecture.md

→ ---
  last_verified: 2026-03-20
  summary: "Clean Architecture. 3 layers. NestJS. PostgreSQL."
  ---

# Step 2: Decide
Is .memory/semantic/architecture.md relevant to this task?
  Yes → load full file
  No  → skip, save context window for something else

Staleness Detection

Memory files can become stale — the code changes, but the memory doesn't. Every memory file has a last_verified date in its frontmatter.

Memory vs Conversation Context

Not everything belongs in memory. Here's how to decide:

Belongs in Memory	Belongs in Conversation
Architecture decisions (permanent)	Current debugging session (temporary)
Project conventions (shared)	Specific implementation details (in-progress)
Sprint outcomes (historical record)	Draft code being iterated on
Lessons learned (reusable)	One-off questions and answers

Example: How Memory Flows Through a Sprint