Skip to content
⚠️ Before You Start
Risk: High
Duration: 4-8 hours hands-on, plus full indexing and re-embedding time (measure in the sandbox) and the watch window
Last Validated: Not yet validated (draft)
Kind: Runbook
Risk Note: A failed cut-over can serve empty, mixed or badly chunked results to every user, and deleting the old index cannot be undone

Re-chunk and re-embed a live corpus without downtime

Draft: not validated

Outline drafted with AI on 16 Oct 2026 and checked against the sources on 26 Oct 2026 (RAG-01-T09). Steps and snippets written with AI on 29 Oct 2026 (RAG-01-T10). Do not use on a live system until it is dry-run by a non-author and signed off (planned 2026-11-18). Author: Kranthi · dry-run: Yogeshwaran · sign-off: Sivaraj. Work items: RAG-01-T09, RAG-01-T10, RAG-01-T11, RAG-01-T12.

1. Summary

This runbook moves a live Azure AI Search index to a new chunking profile with no downtime. Users keep querying the current index (v1) while you build and test a new one (v2). When v2 passes, one alias update switches all traffic to it.

  • How: build v2 with the new chunking, compare it with v1 on counts and the golden set, then repoint the alias.
  • Rollback: repoint the alias to v1. It takes minutes, as long as v1 still exists.
  • Scope: only what is specific to chunking. For general alias and capacity steps, see PB-RAG-09-01.

Facts checked on 26 October 2026. Aliases, the Text Split skill and index projections are generally available in 2026-04-01. Token-based chunking is preview in 2026-08-01-preview.

2. When to use and when not to

Use this runbook when:

  • The chunk size, overlap, split mode or unit changes on a production index.
  • The splitter or layout extractor changes, so chunk boundaries move.
  • Chunk metadata changes and every chunk must be rebuilt.
  • A benchmark shows the current chunking causes poor retrieval.

Do not use it when:

  • The new profile is not yet chosen with benchmark evidence. Use PB-RAG-01-01.
  • Only the embedding model changes. Use PB-RAG-09-02.
  • The change does not touch chunking. Use PB-RAG-09-01.
  • You are reviewing a change before merge. Use PB-RAG-01-04.
  • The index has no live users. Drop and rebuild it.

3. What you must know first

  1. Never re-chunk in place. Chunk keys include a hash that changes on reprocessing. A reset mixes old and new chunks while it runs, cannot be undone, and does not remove orphaned documents.
  2. Overlap limit. pageOverlapLength must be less than half of maximumPageLength. Length limits: check current limits in the docs.
  3. Characters are not tokens. Token chunking (unit: azureOpenAITokens) is preview and does not support every encoder.
  4. Embedding input limit. A chunk over the model's input token limit is truncated or rejected. Confirm the limit on the model card.
  5. Alias rules. One index per alias. Updates take up to 10 seconds to propagate. You cannot delete an index while an alias maps to it.
  6. No alias as a target. Indexers and knowledge sources must name a concrete index.
  7. Counts change by design. With skipIndexingParentDocuments, the index holds only chunks. Compare distinct parents, not document counts.
  8. Chunk keys are not stable. Citation caches, feedback records and evaluation labels that store chunk keys break after the swap.

4. Prerequisites and access

Item Detail
Change approval Approved request with the new profile, benchmark evidence, window, rollback owner and sign-off owner.
Roles Search Service Contributor and Search Index Data Contributor, through Microsoft Entra ID. The search service managed identity can call the embedding deployment.
Tools Azure CLI (az), curl, jq, envsubst (GNU gettext), Python 3.10+ with azure-search-documents 11.4 or later and azure-identity.
Alias in use The application queries an alias. If not, add it as a separate earlier change (PB-RAG-09-01 step 3).
Capacity Storage and vector quota for a second copy. v2 can be larger than v1 (step 2).
Embedding quota Enough tokens-per-minute quota on the embedding deployment for a full re-embed. Estimate v2 chunks as sandbox v2 chunks ÷ sandbox parents × production parents. Check the deployment's current quota before approval; check current limits in the docs.
Golden set eval/queries.json and eval/qrels.json in the ranx format, labelled with parent keys, not chunk keys (PB-RAG-02-01).
Definitions in Git v1 exported (step 1). v2 index and skillset reviewed in a pull request with PB-RAG-01-04.
Sandbox evidence Dry-run timings and chunk counts recorded (section 13).

Set up the session

Run every command in one bash shell (Azure Cloud Shell, Linux, macOS or Git Bash). Work in your change repository, not in this playbook repository. The steps create two folders there: defs/ for search definitions and eval/ for reports. Put the golden set in eval/queries.json and eval/qrels.json before step 3.

S.1. Sign in, then set the names and paths. Replace every ${...} value and the example names with your own.

az login
export SEARCH_SERVICE="${SEARCH_SERVICE}" RG="${RG}" API_VERSION="2026-04-01"
export SEARCH_ENDPOINT="https://${SEARCH_SERVICE}.search.windows.net"
export ALIAS="kb" V1_INDEX="kb-v1" V2_INDEX="kb-v2" DATA_SOURCE="kb-blob-ds"
export V1_INDEXER="kb-v1-indexer" V1_SKILLSET="kb-v1-skillset"
export V2_INDEXER="kb-v2-indexer" V2_SKILLSET="kb-v2-skillset"
export AOAI_ENDPOINT="https://${AOAI_RESOURCE}.openai.azure.com"
export EMBEDDING_DEPLOYMENT="${EMBEDDING_DEPLOYMENT}" EMBEDDING_MODEL="${EMBEDDING_MODEL}"
export SNIPPETS="${PLAYBOOKS_CLONE}/playbooks/rag/rag-01-chunking-choose-benchmark-and-re-chunk-s/snippets"
mkdir -p defs/before defs/after defs/final eval

EMBEDDING_DEPLOYMENT and EMBEDDING_MODEL must match the vectorizer on the v2 index. PLAYBOOKS_CLONE is the folder where you cloned this playbook repository.

S.2. Get a data-plane token and the service resource ID, then test them.

export TOKEN="$(az account get-access-token --scope https://search.azure.com/.default --query accessToken --output tsv)"
export curl -sS "${SEARCH_ENDPOINT}/aliases/${ALIAS}?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" | jq '.indexes'

Verify: the last command prints ["kb-v1"]. If any later call returns HTTP 401, the token has expired: run the TOKEN line again.

5. Decide your approach

Change Path
New size, overlap, split mode or unit This runbook
New extractor or layout model This runbook (extraction is billed again)
New chunking and new embedding model This runbook, plus the model checks in PB-RAG-09-02
New embedding model only PB-RAG-09-02
No live users Drop and rebuild

6. Steps

Placeholders: ${ALIAS} (kb), ${V1_INDEX} (kb-v1), ${V2_INDEX} (kb-v2), ${V1_INDEXER}, ${V2_INDEXER}, ${V1_SKILLSET}, ${V2_SKILLSET}, ${DATA_SOURCE}, ${KNOWLEDGE_SOURCE}, ${PARENT_KEY}. Never paste keys into tickets or chat.

Snippets used (in the pack's snippets/ folder):

  • skillset-v2.json: v2 skillset. Text Split, Azure OpenAI embedding and an index projection to ${V2_INDEX}.
  • compare_indexes.py: v1 and v2 distinct parents, chunk count, largest chunk, recall@10 and nDCG@10. Exit 0 pass, 1 do not swap, 2 bad input.
  • swap-alias.sh: points the alias at one index, waits, and confirms. The same script rolls back.
Stage Steps When Users affected
1. Prepare 1-4 Days before the window No
2. Build v2 5-9 Before the window; indexing can take hours No (watch throttling)
3. Verify before the swap 10-12 Before the window No
4. Cut over 13-14 In the change window Yes
5. Watch and decommission 15-16 After the watch window and sign-off Only if you roll back

Stage 1: Prepare

Step 1. Export v1.

1.1. Save every search definition.

for kind in indexes indexers datasources skillsets aliases; do
  curl -sS "${SEARCH_ENDPOINT}/${kind}?api-version=${API_VERSION}" \
    -H "Authorization: Bearer ${TOKEN}" | jq '.' > "defs/before/${kind}.json"
done

1.2. Show the current Text Split settings.

jq '.value[].skills[] | select(."@odata.type" == "#Microsoft.Skills.Text.SplitSkill")
    | {name, maximumPageLength, pageOverlapLength, unit}' defs/before/skillsets.json

1.3. Commit defs/before to the change repository.

Verify: each file in defs/before has a value array. Step 1.2 prints the v1 values, and they match profile A in the decision record.

Step 2. Check capacity.

2.1. Read the service usage and quota, and the v1 index size.

curl -sS "${SEARCH_ENDPOINT}/servicestats?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" | jq '.counters | {storageSize, vectorIndexSize}'
curl -sS "${SEARCH_ENDPOINT}/indexes('${V1_INDEX}')/search.stats?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" | jq '{documentCount, storageSize, vectorIndexSize}'

2.2. Estimate the v2 size: v1 storageSize × (sandbox v2 chunks ÷ sandbox v1 chunks). Do the same for vectorIndexSize. More overlap or smaller chunks mean a larger v2.

2.3. For storage and for vectors, work out (service usage + v2 estimate) ÷ service quota.

Verify: both results are below 0.8 (80%, team default, as in PB-RAG-09-01). If not, stop. Add a partition and agree the cost before you continue.

Step 3. Record the baseline.

3.1. Run the golden set against v1 in the production query shape. Keep --vector-field for hybrid search. Add --semantic-config <name> if production uses semantic ranking. Remove --vector-field if production is keyword-only.

python "${SNIPPETS}/compare_indexes.py" --v1 "${V1_INDEX}" \
  --queries eval/queries.json --qrels eval/qrels.json \
  --vector-field chunk_vector --out eval/baseline-v1.json

The output looks like this (numbers are illustrative):

                                 kb-v1
documents                        48210
distinct_parents                  1000
chunks_scanned                   48210
largest_chunk_chars               2000
recall@10                        0.874
ndcg@10                          0.712

3.2. Record latency and throttling for the last 7 days.

az monitor metrics list --resource "${SEARCH_ID}" --offset 7d --interval PT1H \
  --metrics SearchLatency ThrottledSearchQueriesPercentage --aggregation Average Maximum --output table

3.3. Attach eval/baseline-v1.json and the metrics to the ticket.

Verify: the report has no WARNING lines, and distinct_parents matches the number of source documents you expect.

Step 4. Find chunk-key and index-name users.

4.1. In each application and pipeline repository, search for the v1 index name.

git grep -n "${V1_INDEX}"

4.2. List the knowledge sources on the service.

curl -sS "${SEARCH_ENDPOINT}/knowledgesources?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" | jq -r '.value[] | .name'

4.3. List the stores that hold chunk keys: citation caches, feedback records and evaluation labels.

4.4. In the ticket, name an owner and a plan for each item. Each store owner exports their store before step 14.

Verify: every git grep hit, knowledge source and store has an owner and a plan in the ticket. A hit that hard-codes ${V1_INDEX} instead of ${ALIAS} is fixed before step 14.

Stage 2: Build v2

Step 5. Create the v2 index.

5.1. Copy the v1 index definition and give it the v2 name.

jq --arg v1 "${V1_INDEX}" --arg v2 "${V2_INDEX}" \
  '.value[] | select(.name == $v1) | .name = $v2 | del(."@odata.etag")' \
  defs/before/indexes.json > "defs/after/index-${V2_INDEX}.json"

5.2. Edit defs/after/index-${V2_INDEX}.json only where the chunking change needs it. Check these rules:

  • The key field is Edm.String with the keyword analyser.
  • parent_id is Edm.String and filterable.
  • The field names match the mappings in skillset-v2.json (chunk, chunk_vector, title, plus your ACL fields).
  • The vectorizer uses the same model as ${EMBEDDING_MODEL}.

5.3. Create the index.

curl -sS -X PUT "${SEARCH_ENDPOINT}/indexes('${V2_INDEX}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" \
  --data @"defs/after/index-${V2_INDEX}.json" -o /dev/null -w "HTTP %{http_code}\n"   # expect 201

Verify: search.stats on ${V2_INDEX} (command in step 2.1) shows documentCount 0. GET /aliases/${ALIAS} (setup S.2) still returns ["kb-v1"].

Step 6. Create the v2 skillset.

6.1. Copy the snippet into the change repository. Do not edit the snippet in the playbook repository.

cp "${SNIPPETS}/skillset-v2.json" defs/after/skillset-v2.template.json

6.2. Show the v1 projection mappings, so you can copy every child field into v2, including ACL fields.

jq --arg s "${V1_SKILLSET}" '.value[] | select(.name == $s) | .indexProjections.selectors[].mappings' \
  defs/before/skillsets.json

6.3. Edit defs/after/skillset-v2.template.json:

  • Set maximumPageLength and pageOverlapLength to profile B from the decision record (PB-RAG-01-01).
  • Add each v1 child-field mapping that is missing. Do not map the parent key.

6.4. Fill in the names, then check the result.

F="defs/after/skillset-${V2_INDEX}.json"
envsubst < defs/after/skillset-v2.template.json > "${F}"
jq '{name, target: .indexProjections.selectors[0].targetIndexName, model: .skills[1].modelName,
     overlap_ok: (.skills[0].pageOverlapLength * 2 < .skills[0].maximumPageLength)}' "${F}"

Verify: name, target and model show your real values, not empty strings. envsubst turns an unset variable into an empty string. target is ${V2_INDEX}, not the alias. overlap_ok is true.

6.5. Create the skillset.

curl -sS -X PUT "${SEARCH_ENDPOINT}/skillsets('${V2_SKILLSET}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" \
  --data @"${F}" -o /dev/null -w "HTTP %{http_code}\n"   # expect 201

Verify: HTTP 201. On HTTP 400, read the error with the same command without -o /dev/null, and see section 9.

Step 7. Create the v2 indexer, disabled.

7.1. Write the indexer definition. Then copy any fieldMappings and other parameters from the v1 indexer in defs/before/indexers.json into this file.

cat > "defs/after/indexer-${V2_INDEX}.json" <<EOF
{
  "name": "${V2_INDEXER}", "dataSourceName": "${DATA_SOURCE}",
  "targetIndexName": "${V2_INDEX}", "skillsetName": "${V2_SKILLSET}",
  "disabled": true, "schedule": { "interval": "PT5M" },
  "parameters": { "configuration": { "dataToExtract": "contentAndMetadata" } }
}
EOF

7.2. Create the indexer. It is disabled, so nothing runs yet.

curl -sS -X PUT "${SEARCH_ENDPOINT}/indexers('${V2_INDEXER}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" \
  --data @"defs/after/indexer-${V2_INDEX}.json" -o /dev/null -w "HTTP %{http_code}\n"   # expect 201

7.3. Commit defs/after to the change repository.

Verify: HTTP 201, and GET /indexers('${V2_INDEXER}') shows "disabled": true.

Costs money

The next step re-runs extraction and embeds every v2 chunk. Confirm the cost estimate and quota are approved. To roll back, disable the v2 indexer (step 8.3) and delete the v2 objects. Users are not affected.

Step 8. Run the v2 indexer.

8.1. Enable the indexer. The 5-minute schedule starts the first run.

jq '.disabled = false' "defs/after/indexer-${V2_INDEX}.json" > defs/after/indexer-on.json
curl -sS -X PUT "${SEARCH_ENDPOINT}/indexers('${V2_INDEXER}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" \
  --data @defs/after/indexer-on.json -o /dev/null -w "HTTP %{http_code}\n"   # expect 200

8.2. Check the status every few minutes until the indexer has caught up.

while true; do
  curl -sS "${SEARCH_ENDPOINT}/indexers('${V2_INDEXER}')/search.status?api-version=${API_VERSION}" \
    -H "Authorization: Bearer ${TOKEN}" \
    | jq -c '.lastResult | {status, itemsProcessed, itemsFailed, endTime}'
  sleep 300
done

Runs with a skillset are time-limited, so one run may not finish the corpus. The schedule starts the next run where the last one stopped (see Run or reset indexers). Stop the loop with Ctrl+C.

8.3. If production queries throttle (section 7), pause the indexer. Resume off-peak with step 8.1.

curl -sS -X PUT "${SEARCH_ENDPOINT}/indexers('${V2_INDEXER}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" \
  --data @"defs/after/indexer-${V2_INDEX}.json" -o /dev/null -w "HTTP %{http_code}\n"   # disabled again

Verify: the indexer has caught up: the last run is success, and the next two scheduled runs show itemsProcessed 0. itemsFailed is 0, or each failure is explained in the ticket.

Step 9. Sync push writers. Skip this step if nothing pushes documents to the index.

9.1. Make the application write to both ${V1_INDEX} and ${V2_INDEX} until step 16. If it cannot, replay every write made since step 8 into ${V2_INDEX}.

Verify: push one test document after step 8 started, and look it up in ${V2_INDEX}.

Stage 3: Verify before the swap

Step 10. Compare v1 and v2.

10.1. Run the comparison on the two indexes directly, not on the alias. Use the same flags as step 3.

python "${SNIPPETS}/compare_indexes.py" --v1 "${V1_INDEX}" --v2 "${V2_INDEX}" \
  --queries eval/queries.json --qrels eval/qrels.json \
  --vector-field chunk_vector --out eval/compare-v2.json
echo "exit code $?"   # 0 pass, 1 do not swap, 2 bad input

10.2. Check the counts in the table.

Verify:

  • distinct_parents is the same for v1 and v2. If not, the report says FAIL distinct parents differ: read the indexer errors (section 9).
  • v2 documents (chunks) is close to the step 2 estimate.
  • v2 largest_chunk_chars is near your maximumPageLength. Convert it to tokens and confirm it is under the embedding model's input limit (model card).
  • There are no WARNING lines.

Step 11. Read the golden-set result. It is in the same report.

Verify: the last line is RESULT: pass. v2 recall@10 and nDCG@10 are no more than 0.02 below v1 (team default, as in PB-RAG-02-01; set --tolerance to the value in the ticket). If the last line is RESULT: do not swap, stop here and return to PB-RAG-01-01.

Step 12. Spot-check chunks.

12.1. Take the parent key of the top result for the first golden query.

export PARENT_KEY="$(jq -r '.indexes.v2.per_query | to_entries[0].value.top[0]' eval/compare-v2.json)"
echo "${PARENT_KEY}"

12.2. Read the chunks of that parent in v2.

curl -sS -X POST "${SEARCH_ENDPOINT}/indexes('${V2_INDEX}')/docs/search.post.search?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" -H "Content-Type: application/json" \
  -d "{\"search\": \"*\", \"filter\": \"parent_id eq '${PARENT_KEY}'\", \"select\": \"chunk,title\", \"top\": 10}" \
  | jq -r '.value[] | "--- \(.title)\n\(.chunk)"'

12.3. Repeat 12.1 and 12.2 for one parent of each document type in the corpus.

Verify: headings, tables and numbered steps are not cut in the middle. Source, page and heading path fields are present where the index has them.

Stage 4: Cut over

Step 13. Start the probe. In a second terminal, start the probe that queries the alias every second (script in PB-RAG-09-01 section 13). Leave it running until step 14 is verified.

Verify: the probe prints HTTP 200 lines for ${ALIAS}.

Affects every user

The next step moves all traffic to v2. Run it in the change window with the rollback owner present. Rollback is bash "${SNIPPETS}/swap-alias.sh" "${V1_INDEX}" (section 8).

Step 14. Repoint the alias to v2.

14.1. Run the swap script. It checks that v2 exists, records the current target, updates the alias, waits 15 seconds (Microsoft documents up to 10) and confirms.

bash "${SNIPPETS}/swap-alias.sh" "${V2_INDEX}"

A good run ends like this:

2026-11-12T10:15:31Z alias kb after: [kb-v2]
2026-11-12T10:15:31Z OK: alias kb points to kb-v2. Previous index: kb-v1

If it ends with FAIL: index ... not found, FAIL: alias ... not found or FAIL: alias update returned HTTP ..., the alias did not change: fix the cause and run it again. If it ends with FAIL: alias does not point to ..., run the rollback command it prints, then check the probe.

14.2. Repoint each knowledge source from step 4 to ${V2_INDEX}. Use the knowledge source definition in defs/before with the index name changed.

14.3. Run the golden set through the alias.

python "${SNIPPETS}/compare_indexes.py" --v1 "${ALIAS}" \
  --queries eval/queries.json --qrels eval/qrels.json \
  --vector-field chunk_vector --out eval/after-swap.json

14.4. Stop the probe, and record the swap time in the ticket.

Verify: step 14.1 exited 0. The probe showed no non-2xx responses. The scores in eval/after-swap.json match v2 in eval/compare-v2.json.

Stage 5: Watch and decommission

Step 15. Watch. Keep v1 and its indexer running for 24-72 hours (team default, as in PB-RAG-09-01), so a rollback serves fresh data. Roll back (section 8) if any rollback trigger fires.

Verify: the section 7 alerts stay quiet, and sign-off is recorded in the ticket.

Irreversible: deletes the v1 index

After this, rollback means a full rebuild from defs/before (hours to days). Confirm sign-off and that the watch window has ended.

Step 16. Decommission v1.

16.1. Confirm that no alias points to v1.

curl -sS "${SEARCH_ENDPOINT}/aliases?api-version=${API_VERSION}" -H "Authorization: Bearer ${TOKEN}" \
  | jq --arg i "${V1_INDEX}" '[.value[] | select(.indexes | index($i))] | length'   # expect 0

16.2. Export the definitions one last time: run the step 1.1 loop with defs/final in place of defs/before, and commit it.

16.3. Delete the v1 indexer first (this stops it), then the skillset, then the index.

for obj in "indexers('${V1_INDEXER}')" "skillsets('${V1_SKILLSET}')" "indexes('${V1_INDEX}')"; do
  curl -sS -X DELETE "${SEARCH_ENDPOINT}/${obj}?api-version=${API_VERSION}" \
    -H "Authorization: Bearer ${TOKEN}" -o /dev/null -w "${obj} HTTP %{http_code}\n"   # expect 204
done

16.4. Remove any partition added in step 2, and delete the two alerts from section 7.

Verify: step 16.1 printed 0, and each delete returned 204. servicestats (step 2.1) shows storage back to plan.

7. Monitoring and alerts

Thresholds are team defaults. Tune them against the step 3 baseline.

Signal Alert when Action
DocumentsProcessedCount, Failed, v2 indexer Any failure during the v2 run Pause. Read indexer errors, fix, rerun.
SkillExecutionCount, Failed, v2 skillset Any failed skill run Check embedding quota. Lower batchSize or raise quota.
ThrottledSearchQueriesPercentage Above 0 for 5 minutes (team default) Disable the v2 indexer and resume off-peak, or add replicas.
IndexStorageUsage, IndexVectorUsage v1 + v2 above 80% of quota (team default) Stop indexing. Add a partition or reduce overlap.
compare_indexes.py golden set on v2 Exit code 1 (do not swap) Do not swap. Return to PB-RAG-01-01.
Non-2xx through the alias Any rise after the swap Roll back (section 8).

Create the two metric alerts before step 8 (they use SEARCH_ID from setup S.2). Delete them in step 16.4.

SEARCH_ID="$(az search service show --name "${SEARCH_SERVICE}" --resource-group "${RG}" --query id --output tsv)"
az monitor metrics alert create --name "rechunk-v2-failed-docs" --resource-group "${RG}" \
  --scopes "${SEARCH_ID}" --severity 2 --action "${ACTION_GROUP_ID}" \
  --condition "total DocumentsProcessedCount > 0 where IndexerName includes ${V2_INDEXER} and Failed includes true" \
  --window-size 15m --evaluation-frequency 5m
az monitor metrics alert create --name "rechunk-throttling" --resource-group "${RG}" \
  --scopes "${SEARCH_ID}" --severity 2 --action "${ACTION_GROUP_ID}" \
  --condition "avg ThrottledSearchQueriesPercentage > 0" --window-size 5m --evaluation-frequency 1m

For the latency, throttling and status-code KQL queries, use PB-RAG-09-01 section 7.

8. Rollback

Situation Rollback Time
Before the swap Disable the v2 indexer. Delete the v2 indexer, skillset and index. No user impact. Minutes
After the swap, v1 kept Run bash "${SNIPPETS}/swap-alias.sh" "${V1_INDEX}". Repoint knowledge sources to v1. Check the probe. Under 5 minutes
After decommission Rebuild v1 from defs/before with this runbook. Hours to days
Chunk-key stores migrated Restore each store from its step 4 export, by the method its owner named. Depends on the store

Rollback trigger (any one): more non-2xx than baseline, a rejected golden-set regression, a sustained p95 latency rise over the step 3 baseline, or confirmed missing content.

9. Known issues and fixes

Symptom Likely cause Fix
Fewer parents in v2 Skill errors or file-size limits Read indexer errors, fix, rerun failed documents.
Chunk count far from estimate Wrong values, or parent documents indexed Check the skillset against the decision record. Set skipIndexingParentDocuments.
Skill failures or throttling during the run Embedding quota too low Raise quota, or lower batchSize and use the schedule.
Skillset rejected Overlap not under half the length, or length out of range Fix the values. Check the Text Split docs.
Skillset has empty names after envsubst A variable was not exported Export it and render again (step 6).
Indexer or knowledge source creation fails Target set to the alias Name the concrete v2 index.
compare_indexes.py shows recall@10 near 0 for both indexes qrels use chunk keys or file names, not parent keys Relabel at parent level (PB-RAG-02-01).
compare_indexes.py warns "counts are partial" The scan hit the paging limit, or writes happened during the scan Pause push writers and rerun. On a large index, count parents with a facet on parent_id instead. Check current limits in the docs.
swap-alias.sh fails with HTTP 401 or 403 Token expired, or role missing Run the TOKEN line again. Check Search Service Contributor.
Alias swap seems not to apply Propagation delay, or hard-coded index name Wait 10 seconds and retest. Remove hard-coded names.
Citations or feedback links break Stored chunk keys not in v2 Apply the step 4 plan.
Wrong users see or lose chunks ACL fields not mapped Map them, rebuild v2, re-verify.
Poor vector results on v2 Vectorizer and embedding skill use different models Align them. Rerun the golden set.
Deleting v1 returns 400 An alias still maps to it Repoint the alias, wait 10 seconds, delete.

10. Cost and capacity notes

  • Second copy. v1 and v2 coexist until decommission. Extra partitions cost money. Check the current price before approval.
  • Full re-embed. Every v2 chunk is embedded. Smaller chunks or more overlap mean more calls. Check the current price before approval.
  • Re-extraction. Layout or extraction skills bill the whole corpus again. Check the current price before approval.
  • Text Split itself is not billed.

11. AWS and open-source equivalents

Platform Equivalent Key differences
Amazon Bedrock Knowledge Bases New data source or knowledge base, sync, switch the app. chunkingConfiguration cannot change after the data source is created. No alias.
Amazon OpenSearch Service and OpenSearch New index, then POST _aliases with remove and add in one request. Alias changes are atomic. Re-ingest from source; _reindex copies old chunks.
LangChain or LlamaIndex New collection or table with the new splitter, switch by config or view. You own the switch and rollback.

12. Checklist

Pre-flight

  • Change approved (owners and window named).
  • PB-RAG-01-04 review passed.
  • v1 exported (step 1).
  • Capacity and embedding quota confirmed (step 2).
  • Baseline recorded (step 3).
  • Chunk-key and index-name users planned (step 4).
  • Alerts created and tested.

Post-change

  • Alias and knowledge sources point to v2 (step 14).
  • Probe showed no non-2xx.
  • Golden set through the alias matches step 11.
  • Watch window passed and signed off (step 15).
  • v1 deleted and partitions removed (step 16).

13. Validation record

Pending. Filled in by the dry-runner (see docs/validation-protocol.md).

Sandbox scenario: re-chunk a 1,000-document sandbox corpus from profile A to profile B while a load script queries the alias. Include one rollback to v1 and a second swap forward.

Pass criteria:

  • Zero non-2xx across both swaps and the rollback.
  • Distinct parent counts match.
  • Golden set scores for v1 and v2 recorded.
  • Rollback under 5 minutes from decision to verified.
  • Every command ran unchanged, or the runbook was fixed in the same pull request.

14. Sources

Checked on 26 October 2026. Embedding skill checked on 29 October 2026.