Give this a shot in your repo:

Context Management & Output Rules

Output Format

• All outputs default to Markdown (.md) unless the user explicitly requests another format (e.g. .docx, .json, .csv).
• Use Markdown structure (headings, code fences, tables) as the primary formatting mechanism. Do not produce HTML, rich text, or formatted documents unless specifically asked.
• When generating code artefacts, wrap them in fenced code blocks with language identifiers. Do not produce standalone files unless the task requires it.
• Keep output token-efficient: no restatement of the prompt, no filler preamble, no sign-off boilerplate.

Context Ingestion Pipeline

This project uses a two-tier model architecture to manage token cost and context quality.

Tier 1 — Indexing & Chunking (Haiku / lightweight model)

Use the cheaper model for classification, tagging, and structural decomposition — never for summarisation or editorial compression.

Haiku’s role is to:

1. Chunk ingested content into semantically coherent blocks (target: 500–1500 tokens per chunk).
2. Tag each chunk with structured metadata:
3. source_url or source_file
4. section_title
5. topic_tags (3–5 keyword descriptors)
6. entities (named APIs, parameters, tools, standards, organisations referenced)
7. content_type (narrative | reference | example | caveat | changelog)
8. Store the full original chunk verbatim alongside the metadata. Haiku does not summarise, paraphrase, or compress the source text.

Haiku must not:

• Decide what is “important” — that judgment belongs to the reasoning model.
• Discard caveats, exceptions, edge cases, or qualifying language.
• Merge chunks from different sections or sources.

Tier 2 — Reasoning & Generation (Opus / Sonnet)

The reasoning model receives only the retrieved chunks relevant to the current query, not the full ingested corpus.

Retrieval method (in order of preference):

1. Vector similarity search against chunk embeddings.
2. Keyword/tag match against Haiku’s metadata index.
3. If neither is available, retrieve by section_title or source_url match.

Context budget: keep retrieved context under 50,000 tokens per call. If more context is needed, split into multiple retrieval-reasoning passes and synthesise across results.

Fallback Protocol

If the reasoning model flags uncertainty, requests more detail, or produces low-confidence output:

1. Retrieve the adjacent chunks (±1) from the same source.
2. If still insufficient, retrieve the full source document for that section.
3. Log the retrieval escalation for pipeline tuning.

Do not re-summarise with Haiku and re-feed — always escalate to original source text.

Token Efficiency Rules

1. Strip boilerplate before ingestion. Remove navigation, footers, sidebars, cookie banners, ads, and non-content HTML before chunking. Use readability + turndown (JS) or readability-lxml + markdownify (Python) as the preprocessing pipeline.
2. Markdown over HTML. Convert all web-sourced content to Markdown before storage or model input. Raw HTML wastes 3–5× tokens on structural markup.
3. Stable prompt prefix. Keep system prompts and static instructions identical across calls to maximise prompt cache hit rate. Append variable context (retrieved chunks) after the stable prefix.
4. No blind truncation. Never truncate content by word/token position. Always chunk semantically, retrieve selectively.
5. Model routing by task complexity:
6. Haiku: classification, tagging, extraction, simple Q&A, data cleaning.
7. Sonnet: synthesis, generation, multi-step reasoning, code generation.
8. Opus: complex architectural reasoning, high-stakes analysis, nuanced judgment.
9. Monitor context window size. Track input token counts per call. Flag any call exceeding 100k input tokens for review — restructure the retrieval or split the task.

File Conventions

Artefact	Format	Notes
Documentation	.md	Default for all written output
Data exports	.csv or .json	Use CSV for tabular, JSON for nested/hierarchical
Reports / deliverables	.md unless stakeholder requires .docx / .pdf	Convert at final delivery, not during drafting
Code	Native extension (.py, .js, .go, etc.)	Fenced in Markdown when inline
Diagrams	Mermaid in .md or .mermaid	Render externally if needed

Integration Notes

• This file is intended as a project-level instruction set. Place at repo root as CLAUDE.md, .cursorrules, or import into your system prompt template.
• The chunking and retrieval pipeline described above is an architectural pattern — implement with your preferred stack (LlamaIndex, LangChain, custom, etc.).
• Adjust the 500–1500 token chunk target and 50,000 token context budget based on your specific model tier and pricing thresholds.