Skip to content
⚠️ Before You Start
Risk: Medium
Duration: 1-2 hours
Last Validated: Not yet validated (draft)
Kind: Checklist
Risk Note: Missing capacity limits, vector quota exhaustion or unconfigured alerts cause query throttling, document loss and production outages

Index go-live and capacity checklist

Draft: not validated

Not yet dry-run. Do not use on a live system until the validation record is complete and signed off. Planned validation date: 2026-11-20. Author: Kandiah Chockalingam · Dry-run: Ranjith · Sign-off: Yogeshwaran. Work items: RAG-03-T05, RAG-03-T06, RAG-03-T07, RAG-03-T08.

1. Summary

This checklist gates releases and capacity changes for an Azure AI Search index across three operational milestones: before an index goes live, before a big data load, and monthly for production services. It verifies replica sizing, vector index quota headroom, storage boundaries, index aliasing, Entra ID access controls, diagnostic telemetry, and metric alert rules. Running this checklist prevents query throttling, out-of-quota indexing failures, single-replica availability breaches, and irreversible production lock-in.

Facts checked against Microsoft Learn on 30 October 2026. Latest stable data-plane REST API: 2026-04-01. Document and skill reset operations are preview under API version 2026-05-01-preview.

2. When to use and when not to

Use this checklist when:

  • Releasing a new or re-indexed Azure AI Search index to a staging or production environment.
  • Preparing for a bulk data load or batch re-indexing that increases corpus size by more than 20%.
  • Reviewing monthly capacity, quota consumption, slow queries, and alert thresholds for live search services.
  • Resizing search service scale units (adding or removing replicas or partitions).

Do not use it when:

  • Designing the initial index schema, choosing field analysers, or selecting vector algorithms. Use PB-RAG-03-01 instead.
  • Executing an active zero-downtime reindexing cut-over or indexer rebuild. Follow PB-RAG-09-01 instead.
  • Upgrading embedding models across an entire corpus. Follow PB-RAG-09-02 instead.
  • Evaluating retrieval relevance, nDCG, or regression gates. Follow PB-RAG-02-04 instead.

3. What you must know first

  1. Replicas provide availability and query throughput. High-availability service-level agreements (SLAs) require at least two replicas for read availability (99.9%) and three replicas for read-write availability (99.99%). Adding replicas increases query capacity linearly but does not increase storage or vector quota. Check current SLA requirements in official Azure documentation.
  2. Partitions provide storage and vector quota. Partitions add physical disk space and indexing capacity. Vector index quota is a strict hard limit calculated per partition based on service tier (SKU). When vector quota is exceeded, document indexing fails immediately with HTTP 403 or 400 (QuotaExceeded). Check current tier limits in the official docs before sizing.
  3. Index aliases decouple clients from physical indexes. Applications must always query an index alias (${SEARCH_ALIAS}) rather than the physical index name (${INDEX_NAME}). An alias maps to exactly one index and propagates updates in under 10 seconds, enabling zero-downtime reindexing.
  4. Keyless authentication minimizes credential exposure. Production applications and pipelines should authenticate using Microsoft Entra ID role-based access control (Search Index Data Reader for search clients, Search Index Data Contributor for ingestion pipelines) rather than shared admin keys. Check whether admin keys can be disabled via service configuration.
  5. Diagnostic settings must route to Log Analytics. Raw Azure Monitor platform metrics have limited retention. Diagnostic settings must forward both AzureMetrics and AzureDiagnostics (or resource-specific search logs) to a designated Log Analytics workspace for troubleshooting, alerting, and trend analysis.
  6. Indexer multitenancy limits. Indexers running skillsets operate with a 2-hour maximum execution limit per run. Ingesting large document sets requires change detection and recurring schedules to index incrementally across multiple runs.

Prerequisites and access

Requirement Detail
Role assignments Search Service Contributor for managing service scale, diagnostic settings, and aliases. Search Index Data Reader for application queries. Search Index Data Contributor for ingestion pipelines. Monitoring Contributor for deploying metric alerts.
Tooling Azure CLI (az) 2.60+, curl, jq.
Infrastructure config Target Search service name (${SEARCH_SERVICE}), Log Analytics workspace ID (${WORKSPACE_ID}), Action Group ID (${ACTION_GROUP_ID}), and resource group (${RESOURCE_GROUP}).
Git definitions Target index schema, indexer, datasource, and skillset definitions committed to version control.

Retrieve a data-plane access token using the Azure CLI:

export SEARCH_ENDPOINT="https://${SEARCH_SERVICE}.search.windows.net"
export API_VERSION="2026-04-01"
export TOKEN="$(az account get-access-token --scope https://search.azure.com/.default --query accessToken --output tsv)"

Verify: echo "${TOKEN}" displays a non-empty JWT bearer token.

4. Checklist

Stage 1: Before an index goes live (Go-live readiness gate)

Complete these checks prior to routing production user traffic to the index:

  • Step 1: Check replica count against SLA targets. Verify that the search service has sufficient replicas configured for high availability.
az search service show --name "${SEARCH_SERVICE}" --resource-group "${RESOURCE_GROUP}" \
  --query "{sku:sku.name, replicas:replicaCount, partitions:partitionCount}" -o json

Verify: Output confirms replicas is at least 2 for read SLA (99.9%) or at least 3 for read-write SLA (99.99%).

  • Step 2: Check storage and vector quota headroom. Ensure current consumption allows at least 30% headroom for steady-state growth and a second index copy for zero-downtime reindexing.
curl -sS "${SEARCH_ENDPOINT}/servicestats?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq '{storageUsage: .counters.storageSize.usage, storageQuota: .counters.storageSize.quota, vectorUsage: .counters.vectorIndexSize.usage, vectorQuota: .counters.vectorIndexSize.quota}'

Verify: Both storageUsage / storageQuota and vectorUsage / vectorQuota remain below 70%. Check current tier limits in the official docs.

  • Step 3: Confirm client application queries an index alias. Verify that the application configuration references ${SEARCH_ALIAS} instead of a physical index name.
curl -sS "${SEARCH_ENDPOINT}/aliases('${SEARCH_ALIAS}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" | jq '.'

Verify: The response returns HTTP 200 with "name": "${SEARCH_ALIAS}" and "indexes": ["${INDEX_NAME}"].

  • Step 4: Verify keyless authentication and RBAC bindings. Verify that client workloads use managed identities and have the least-privilege role assigned.
az role assignment list --scope "/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${RESOURCE_GROUP}/providers/Microsoft.Search/searchServices/${SEARCH_SERVICE}" \
  --query "[?roleDefinitionName=='Search Index Data Reader'].{principal:principalName, role:roleDefinitionName}" -o table

Verify: Output lists the client application managed identity with Search Index Data Reader role.

  • Step 5: Validate diagnostic settings routing. Verify that diagnostic settings forward platform metrics and query logs to the designated Log Analytics workspace.
az monitor diagnostic-settings list --resource "/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${RESOURCE_GROUP}/providers/Microsoft.Search/searchServices/${SEARCH_SERVICE}" \
  --query "[].{name:name, workspace:workspaceId, logs:logs[?enabled==\`true\`].categoryGroup, metrics:metrics[?enabled==\`true\`].category}" -o json

Verify: Output confirms diagnostic settings are active and route allLogs (or Audit and QueryRuns) and AllMetrics to ${WORKSPACE_ID}.

  • Step 6: Deploy metric alert rules. Deploy standard alert rules for throttling, search latency, and quota exhaustion using the ARM template snippets/search-alerts.json.
az deployment group create --resource-group "${RESOURCE_GROUP}" \
  --template-file "playbooks/rag/rag-03-index-design-schema-change-and-cut-over/snippets/search-alerts.json" \
  --parameters searchServiceName="${SEARCH_SERVICE}" actionGroupId="${ACTION_GROUP_ID}"

Verify: Deployment succeeds with provisioningState: "Succeeded", and three alert rules appear under Azure Monitor alerts.

  • Step 7: Run synthetic search test and stage alias rollback. Execute a representative test query against the alias, then verify that the instant rollback command is documented.
curl -sS -X POST "${SEARCH_ENDPOINT}/indexes('${SEARCH_ALIAS}')/docs/search?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"search": "connectivity test", "top": 1}' | jq '.value | length'

Verify: The query succeeds with HTTP 200 and returns top search results.

Live alias cut-over carries user impact

Repointing an alias directs live production traffic to the new index. If query latency spikes or relevance drops post-cutover, immediately execute the alias rollback command below.

# Emergency rollback: repoint alias back to previous stable index
curl -sS -X PUT "${SEARCH_ENDPOINT}/aliases('${SEARCH_ALIAS}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"name": "'"${SEARCH_ALIAS}"'", "indexes": ["'"${PREVIOUS_STABLE_INDEX}"'"]}'

Verify: Calling GET on /aliases('${SEARCH_ALIAS}') confirms "indexes": ["${PREVIOUS_STABLE_INDEX}"].

Stage 2: Before a big data load (Ingestion capacity gate)

Complete these checks prior to running a batch reindexing, skillset enrichment, or data load that expands corpus size by more than 20%:

  • Step 1: Calculate vector quota expansion. Estimate total vectors and raw storage required for the new ingestion batch.
  • Formula: Estimated Vectors = Existing Vectors + (New Documents * Average Chunks Per Document).
  • Memory requirement: Estimated Vectors * Vector Dimensions * 4 bytes plus HNSW graph index overhead (~20%).

  • Verify: Available vector index quota on the service exceeds the estimated memory requirement by at least 25%. If insufficient, add partitions before starting. Check current tier limits in the official docs.

  • Step 2: Check physical storage disk capacity. Confirm that partition disk storage can absorb new document content, raw text fields, and inverted indexes.

curl -sS "${SEARCH_ENDPOINT}/servicestats?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq '{storageUsage: .counters.storageSize.usage, storageQuota: .counters.storageSize.quota}'

Verify: storageUsage plus the projected load size (bytes) is less than 80% of storageQuota.

  • Step 3: Confirm deletion and change detection policies. For indexer-driven ingestion from Azure Blob Storage or Azure SQL, verify that soft-delete detection is configured on the data source.
curl -sS "${SEARCH_ENDPOINT}/datasources('${DATASOURCE_NAME}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" | jq '.dataDeletionDetectionPolicy'

Verify: The datasource configuration contains a valid dataDeletionDetectionPolicy (e.g. NativeBlobSoftDeleteDeletionDetectionPolicy). If omitted from the first run, deleted files will remain orphaned in the search index.

  • Step 4: Configure indexer batch size and failure tolerance. Review indexer settings to prevent premature failure during large batches.
curl -sS "${SEARCH_ENDPOINT}/indexers('${INDEXER_NAME}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" | jq '.parameters | {batchSize, maxFailedItems, maxFailedItemsPerBatch}'

Verify: maxFailedItems and maxFailedItemsPerBatch match agreed change tolerances, and batchSize is tuned for target payload size (e.g. 50-100 documents).

Bulk indexing competes with search query capacity

Running unthrottled indexers during peak business hours consumes search unit CPU and memory, doubling query latency and causing HTTP 503 throttling errors for end users.

  • Step 5: Schedule indexer off-peak or scale query replicas. If the indexer runs against a live serving index, schedule the run during off-peak hours or add a temporary query replica.

  • Verify: Maintenance window approved in change ticket, or replica count increased via Azure CLI.

  • Step 6: Stage emergency indexer pause procedure. Ensure the operator can halt runaway indexing without deleting the indexer or losing its change-tracking watermarks.

PUT replaces the full indexer definition — GET the body first

Sending only {"name": ..., "isDisabled": true} to a PUT will either return HTTP 400 (data source and target index are required fields) or silently wipe the existing configuration. Always retrieve the current definition first, patch isDisabled, then PUT the complete body.

# Step 6a: Retrieve the current indexer definition
INDEXER_BODY=$(curl -sS "${SEARCH_ENDPOINT}/indexers('${INDEXER_NAME}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}")

# Step 6b: Patch isDisabled to true and PUT the full body
echo "${INDEXER_BODY}" | jq '.isDisabled = true' | \
curl -sS -X PUT "${SEARCH_ENDPOINT}/indexers('${INDEXER_NAME}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  --data @-

Verify: Calling GET on /indexers('${INDEXER_NAME}') confirms "isDisabled": true. The indexer will not start any new run; in-flight runs complete or time out naturally.

Stage 3: Monthly for production services (Routine capacity & operational audit)

Complete these checks during monthly service reviews:

  • Step 1: Execute search health KQL query. Run the health query from snippets/search-health.kql in the linked Log Analytics workspace.
// 24-hour search service throughput, latency and throttling summary
AzureMetrics
| where TimeGenerated > ago(24h)
| where ResourceProvider == "MICROSOFT.SEARCH"
| where MetricName in ("SearchQueriesPerSecond", "SearchLatency", "ThrottledSearchQueriesPercentage")
| summarize AvgValue = round(avg(Average), 2), MaxValue = round(max(Maximum), 2)
    by MetricName, bin(TimeGenerated, 1h)
| order by MetricName asc, TimeGenerated desc

Verify: ThrottledSearchQueriesPercentage maximum is 0.00% across the entire period. Average SearchLatency conforms to application SLOs (e.g. under 0.2 s — SearchLatency is reported in seconds by the platform metric).

  • Step 2: Inspect 30-day quota consumption trends. Query monthly growth to project partition exhaustion dates.
  • Run Query 4 from snippets/search-health.kql.

  • Verify: IndexStorageUsage and IndexVectorUsage growth rates project at least 90 days before reaching 80% of tier quota.

  • Step 3: Audit slow queries and HTTP 429/503 errors. Identify queries with latency exceeding 500ms or rejected requests.

  • Run Query 3 from snippets/search-health.kql.

  • Verify: Slow queries are identified, catalogued, and shared with the application team for caching, filter optimization, or parameter tuning.

  • Step 4: Check for indexer run failures. Inspect indexer execution logs over the past 30 days.

  • Run Query 5 from snippets/search-health.kql.

  • Verify: No unhandled indexer failures exist. Transient document extraction failures have been retried or logged.

  • Step 5: Audit metric alert rule drift. Verify that active alert rules in Azure Monitor match the defined template in snippets/search-alerts.json.

  • Verify: Alerts for throttling, latency, and vector quota remain enabled with correct notification action groups.

  • Step 6: Reconcile index schemas against Git. Compare live schema definitions with version-controlled files in the repository.

curl -sS "${SEARCH_ENDPOINT}/indexes('${INDEX_NAME}')?api-version=${API_VERSION}" \
  -H "Authorization: Bearer ${TOKEN}" | jq 'del(.name)' > /tmp/live-schema.json
git diff --no-index /tmp/live-schema.json "playbooks/rag/rag-03-index-design-schema-change-and-cut-over/snippets/index-policy.json"

Verify: No untracked schema modifications or undocumented field attribute changes exist on the live service.

5. Known issues and fixes

Symptom Likely cause Fix
Intermittent HTTP 503 or 429 errors during query bursts Insufficient query replicas for concurrency Scale replicas up via az search service update --replica-count <N> or configure client-side exponential backoff with jitter.
Ingestion fails with HTTP 403 or 400 QuotaExceeded Vector size exceeds per-partition vector quota Enable scalar or binary quantization on vector profiles, add partitions, or upgrade service tier. Check current limits in the docs.
Search query latency doubles during scheduled indexer runs Indexer processes compete with queries on shared search unit resources Reschedule indexer to run during off-peak hours, throttle batch size, or add a dedicated query replica.
Source file deletions do not reflect in search results Missing dataDeletionDetectionPolicy on the data source definition A deletion detection policy cannot be added retrospectively to clear existing orphans. Create a new index and rebuild using Path B in PB-RAG-09-01.
Log Analytics workbooks show no query or latency records Diagnostic settings disabled or misconfigured Enable diagnostic settings on the search service. Ensure allLogs and AllMetrics categories forward to ${WORKSPACE_ID}.
Client application breaks during index rebuild Client connects directly to physical index name instead of an index alias Create an index alias pointing to the current physical index and update application configuration strings to target ${SEARCH_ALIAS}.
Indexer halts execution after exactly 2 hours Skillset multitenancy execution limit reached Configure a recurring indexer schedule (e.g. every 15 minutes) with change tracking enabled to process large corpora incrementally across runs.
Metric alert fires false positive during planned reindexing Alert threshold evaluated against temporary batch indexing load Suppress alert rules temporarily during scheduled change windows using Azure Monitor alert processing rules.

6. Monitoring and alerts

The thresholds below are illustrative team defaults, not platform recommendations. Tune them against a measured baseline.

Signal Source Alert when Action
ThrottledSearchQueriesPercentage Platform metric (average) Above 0% for 5 minutes Scale replicas or investigate query traffic spikes.
SearchLatency (seconds) Platform metric (average) Above 1.5x measured baseline for 5 minutes Inspect slow queries, check indexer concurrency, and evaluate replica scaling.
SearchQueriesPerSecond Platform metric (average) Sustained above 80% of benchmarked capacity Plan replica scale-out before query degradation occurs.
IndexStorageUsage Platform metric (maximum) Above 80% of tier partition capacity Add partitions or compress stored fields. Check current tier limits in the docs.
IndexVectorUsage Platform metric (maximum) Above 75% of tier vector index quota Plan partition expansion, move to a higher tier, or apply vector quantization.
FailedIndexerRuns Diagnostic log / alert Greater than 0 failed executions in scheduled window Inspect indexer execution history, check datasource credentials, and reset affected indexer.
SearchServiceAvailability Platform metric (average) Below 99.9% over 15 minutes Escalate to cloud provider incident support and inspect Azure Resource Health.

7. AWS and open-source equivalents

Platform Equivalent Key differences
Amazon OpenSearch Service Multi-AZ deployment with primary and replica shards OpenSearch uses primary and replica shards across data nodes rather than separate partition/replica scaling. Monitor ClusterStatus.red, SearchLatency, and JVMMemoryPressure in CloudWatch. Manage index aliases via POST _aliases.
Amazon Bedrock Knowledge Bases Managed OpenSearch Serverless collection Scaling is automated via OpenSearch Compute Units (OCUs). Monitor vector ingestion and query latency via CloudWatch metrics. Aliasing and direct shard management are abstracted away by the managed service.
Self-hosted OpenSearch / Elasticsearch Clustered nodes with shard allocation Sizing requires manual configuration of JVM heap (50% physical RAM, maximum 32 GB) and disk watermarks (85% low, 90% high, 95% flood-stage). Monitor via Prometheus elasticsearch_exporter and Grafana dashboards.
PostgreSQL + pgvector Tables with HNSW / IVFFlat indexes on dedicated instances Scaled vertically via CPU/RAM or horizontally with read replicas. Monitor memory pressure (shared_buffers, maintenance_work_mem), connection pool limits (PgBouncer), and slow queries via pg_stat_statements.

8. Validation record

Pending. Filled in by the dry-runner (see docs/validation-protocol.md).

Sandbox scenario: Apply this checklist to an active sandbox search service where diagnostic settings are disabled, a single replica is configured, one metric alert is missing, and simulated vector index utilization exceeds 75%. Confirm that the checklist flags all deficiencies and verifies remediation before sign-off.

Pass criteria: The dry-runner confirms that all four planted faults (single replica availability risk, unconfigured diagnostic logging, missing alert rule, and vector quota exhaustion risk) are caught, documented, and remediated.

Record

Item Detail
Playbook PB-RAG-03-04 at commit [hash]
Dry-run by Ranjith
Date YYYY-MM-DD
Sandbox [Resource group or tag, tier and region]
Scenario [Scenario followed]
Time taken [Hands-on time] hands-on, [waiting time] waiting
Result [PASS / PASS WITH NOTES / FAIL]
Notes 1. [Step number: defect or explanation; blocking/non-blocking; fixed/logged]
Signed off by Yogeshwaran

9. Sources

Checked on 30 October 2026; REST response field names re-verified on 2 November 2026.