RAG-01 · Chunking: choose, benchmark and re-chunk safely¶
Teams pick a chunk size by guesswork and only find out it is wrong when users complain. Changing it later means re-embedding a live corpus with no tested procedure, so the change is either avoided or done with downtime.
- Who reads it: GenAI engineers designing RAG pipelines and DevOps engineers running re-chunk and re-embed jobs
- Outcome: Chunking chosen by benchmark, changed without downtime
- Pack owner: Yogeshwaran (catalog owner role: RAG Lead)
Master catalog entry¶
- Catalog name: Chunking Strategy Framework (P1)
- Why it matters: Retrieval quality is the top driver of user trust in a knowledge assistant
- What it removes: Ends the trial-and-error chunking loop that eats early sprints
- Stack: Python, LangChain splitters, Document Intelligence layout
- Maturity target: Level 3 — decision tree plus benchmarked implementations by document type
Playbooks¶
| ID | Story | Title | Kind | Complexity | Urgency | Status | Owner | Planned publish |
|---|---|---|---|---|---|---|---|---|
| PB-RAG-01-01 | RAG-01-S1 | Choose a chunking strategy by document type | Decision guide | Medium | 3 | draft | Yogeshwaran | 2026-12-07 |
| PB-RAG-01-02 | RAG-01-S2 | Build layout-aware chunking with Document Intelligence and LangChain | How-to | Medium | 3 | draft | Kandiah Chockalingam | 2026-12-10 |
| PB-RAG-01-03 | RAG-01-S3 | Re-chunk and re-embed a live corpus without downtime | Runbook | High | 5 | draft | Kranthi | 2026-11-18 |
| PB-RAG-01-04 | RAG-01-S4 | Chunking change review and troubleshooting checklist | Checklist | Low | 4 | draft | Kandiah Chockalingam | 2026-11-25 |