Skip to content
⚠️ Before You Start
Risk: Medium
Duration: 1-4 hours to stabilise
Last Validated: Not yet validated (draft)
Kind: Runbook
Risk Note: Medium (missing content in answers, or a large bill if a full rerun is triggered by mistake)

Handle extraction failures, timeouts and cost overruns in the enrichment pipeline

Draft outline: not validated

Drafted with AI from official sources on 16 Oct 2026. Do not use on a live system until it is written, dry-run by someone other than the author and signed off (planned 2026-11-17). Author: Kandiah Chockalingam · dry-run: Ranjith · sign-off: Yogeshwaran. Work items: RAG-10-T13, RAG-10-T14, RAG-10-T15, RAG-10-T16.

1. Summary

The enrichment indexer reports skill errors, timeouts or throttling, or extraction cost goes above forecast.

Risk: Medium (missing content in answers, or a large bill if a full rerun is triggered by mistake). Typical duration: 1-4 hours to stabilise.

2. When to use and when not to

Use it when: The enrichment indexer reports skill errors, timeouts or throttling, or extraction cost goes above forecast.

Do not use it when: to be written by the author.

3. What you must know first

  • Skill errors appear in the indexer status errors list with the document key
  • Custom Web API skills time out after 30 seconds by default; the maximum is PT230S
  • Content Understanding and Document Layout charge for documents that time out
  • A throttled skill reports 'The cognitive service for skill ... has been throttled'
  • The enrichment cache (preview) can avoid re-billing unchanged documents, but it is not a backup

4. Prerequisites and access

  • Search Service Contributor role
  • Access to Foundry resource metrics and Cost Management
  • Access to the indexer status API
  • Agreed daily cost cap for extraction

5. Steps

Outline of the steps; the author adds the commands and a check after each one.

  1. Read the indexer status and group errors by message.
  2. If cost triggered the alert, disable the indexer schedule first.
  3. For timeouts, find large documents and split or exclude them.
  4. For throttling, lower batchSize and check the Foundry resource quota.
  5. For custom skills, check endpoint logs and raise the timeout within the limit.
  6. Exclude unsupported or locked files with indexer settings and record them.
  7. Run the indexer on a small batch and confirm the errors are gone.
  8. Reprocess only failed documents with reset docs (preview).
  9. Re-enable the schedule and confirm daily cost is back on forecast.

Draft snippet: Create a metric alert for failed skill executions

#!/usr/bin/env bash
# Alert when skill executions fail on the enrichment indexer.
set -euo pipefail
: "${RG:?}" "${SEARCH_SERVICE:?}" "${ACTION_GROUP_ID:?}" "${INDEXER:?}"

SEARCH_ID="$(az search service show --name "${SEARCH_SERVICE}" --resource-group "${RG}" --query id --output tsv)"

# Check the exact dimension values for Failed and IndexerName in Metrics Explorer first.
az monitor metrics alert create \
  --name "skill-failures-${INDEXER}" \
  --resource-group "${RG}" \
  --scopes "${SEARCH_ID}" \
  --condition "total SkillExecutionCount > 10 where Failed includes true and IndexerName includes ${INDEXER}" \
  --window-size 1h \
  --evaluation-frequency 15m \
  --severity 2 \
  --action "${ACTION_GROUP_ID}" \
  --description "Skill executions are failing on ${INDEXER}"

Snippets to add:

  • snippets/ops/alerts/skill-failures.sh: Metric alert on failed skill executions (bash)
  • snippets/defs/indexer-enrichment-parameters.json: Indexer parameters to skip bad files and limit batch size (json)
  • snippets/ops/requests/targeted-reset.http: Reset docs and reset skills requests (preview) (http)

6. Monitoring and alerts

Signal Alert when Action
SkillExecutionCount where Failed = true, by IndexerName More than 10 failed executions in 1 hour Start this runbook.
Daily cost of the Foundry resource (Cost Management budget) Above the agreed daily cap Disable the indexer schedule and investigate.
Indexer status error, or two transientFailure runs in a row Page the on-call engineer.

7. Rollback

To be written by the author: how to undo each risky step.

8. Known issues and fixes

Symptom Likely cause Fix
Could not execute skill because the Web API request failed Custom skill error or unexpected input Debug the skill and check skill input warnings.
Skill did not execute within the time limit Large document or slow back end Raise timeout up to PT230S, lower batchSize, or split documents.
Cost spike with no new documents A reset or skillset edit reprocessed the corpus Check recent resets and skillset changes; use reset skills or reset docs (preview) for targeted reruns.
Expected IndexAction metadata error Cache changed without a reset Reset and rerun the indexer.

9. AWS and open-source equivalents

With Amazon Textract or Amazon Bedrock Data Automation, handle throttling and failed jobs in your orchestration and track cost per page. Open-source parsers move the cost to your own compute, so watch CPU or GPU time instead.

10. Validation record

Pending. Planned dry-run scenario: A second engineer injects a slow custom skill, an oversized file and a throttled deployment into the sandbox pipeline. Pass: the alerts fire, each cause is fixed by following the runbook, and only the failed documents are reprocessed.

11. Sources