Skip to content
⚠️ Before You Start
Risk: Medium
Duration: 2-3 days with a small benchmark
Last Validated: Not yet validated (draft)
Kind: Decision guide
Risk Note: Medium (a wrong yes costs weeks of build and LLM spend; a wrong no loses a differentiator)

Decide whether GraphRAG is worth it

Author: Yogeshwaran · dry-run: Gayathri Ramachandran · sign-off: Deeparani. Work items: RAG-11-T01, RAG-11-T02, RAG-11-T03, RAG-11-T04.

1. Summary

This guide decides whether a client's questions need a knowledge graph, and whether the graph is worth its indexing and refresh cost. It compares five options on the client's own labelled questions, from tuned hybrid RAG (Option A, the default) to full GraphRAG (Option D), and ends with a signed decision record. Facts checked on 6 and 11 November 2026: LLM query planning in Azure AI Search agentic retrieval is preview (REST API 2026-08-01-preview), and the Azure AI Evaluation SDK is preview.

2. When to use and when not to

Use this guide when:

  • A client asks for multi-hop, relationship or "connect the dots" questions.
  • Hybrid RAG fails on questions that need a summary of the whole corpus.
  • Someone proposes a graph build before anyone has measured hybrid RAG on the client's questions.
  • A pre-sales proposal includes GraphRAG and needs evidence for the design review.

Do not use it when:

  • You are choosing between keyword, vector, hybrid and semantic ranking. Use PB-RAG-08-01.
  • You are tuning hybrid weights or re-ranking. Use PB-RAG-08-02.
  • Answers fail because chunks cut tables or sections. Use PB-RAG-01-01.
  • You have no labelled question set yet. Build one first with PB-RAG-02-01.
  • The decision is already go and you are building the pilot. Use PB-RAG-11-02.
  • You are costing a go-live. Use PB-RAG-11-03.
  • You are refreshing a graph that already exists. Use PB-RAG-11-04.

3. What you must know first

  1. Three question types decide the answer. One-place questions have their answer in one passage, and hybrid RAG handles them well (see the three question types). Baseline RAG "struggles to connect the dots" and with "summarized semantic concepts over large data collections" (GraphRAG docs). Microsoft Learn recommends graph-augmented RAG for entity relationships and multi-hop reasoning, and vector-only RAG for simple lookups and fast time to production (HorizonDB graph RAG).
  2. Graph indexing is expensive. The GraphRAG README warns that indexing "can be an expensive operation" and says to "start small" (README). In standard indexing, graph extraction is "roughly 75% of indexing cost" (Indexing methods). At query time, global search is "a resource-intensive method" (Query overview).
  3. Microsoft GraphRAG is demonstration code in maintenance mode. The code "is not an officially supported Microsoft offering". The README also says the project "is largely in maintenance mode, and won't be accepting new PRs or implementing new features", with bug fixes and CVE-driven dependency updates only (README). Pin the version you test; the README asks you to re-run graphrag init --force between minor versions.
  4. Cheaper fixes exist; try them first. Agentic retrieval in Azure AI Search breaks a complex query into subqueries, runs them in parallel and merges the results; its LLM query planning is preview (Agentic retrieval). If the relationships already sit in structured data, build the graph from the tables: "If you already have foreign key relationships, you already have a graph" (HorizonDB graph RAG).
  5. A cheaper graph exists, at a quality cost. FastGraphRAG (--method fast) extracts noun phrases with NLP libraries instead of an LLM. It is "much cheaper", but the graph is "quite a bit noisier" and has no entity or relationship descriptions (Indexing methods).
  6. Refresh is a recurring cost. New content can change the community structure, "such that much of an index needs to be re-computed". The update command "computes the deltas" to reduce this, and the LLM cache makes re-runs "significantly faster and cheaper" (GraphRAG 1.0 blog). Model refresh cost before any go.
  7. LazyGraphRAG is not an option for most teams. Microsoft Research reports indexing costs "identical to vector RAG and 0.1% of the costs of full GraphRAG" on its own benchmark (LazyGraphRAG blog). The blog names Microsoft Discovery and Azure Local (public preview) as where it is available. The GraphRAG releases page does not mention it, and the README says no new features will be added (releases).

4. Options

The three question types

Every question in the labelled set gets exactly one type. The type says what kind of search the answer needs, so the scores show exactly which kind of question fails. The examples use a company with 1,000 project reports.

Type (type: value) What it means Example How to spot it Usually answered by
One place (one_place) The whole answer sits in one passage of one document. "What is the budget of Project Alpha?" You can mark the whole answer in one passage of one document. Option A
Connect the dots (connect_the_dots) The answer joins two or more facts from different passages or documents. Each fact leads to the next. "Who manages the team that built the system that failed last week?" One report names the failed system, a second names the team that built it, and a third names the team's manager. You need several passages, and you can only find the second after you have read the first. Options B, E or D
Summary of everything (whole_corpus_summary) The answer is a theme or pattern across most of the corpus, not a fact in any one place. "What are the main risks across all our projects?" There is no passage to mark. You would have to read most of the documents and summarise them. Options C or D

Labelling rules (team rules):

  • Label by what the answer needs, not by how the question is worded. "Tell me about Project Alpha's budget" is still a one-place question.
  • If a question fits two types, choose the harder one: summary of everything, then connect the dots, then one place.
  • A question that asks two separate things, each answered in one place, is not connect the dots. Split it into two one-place questions.
  • Write the expected answer when you label the question. For a summary question, write the key points a good answer must contain.

The option ladder

Think of the options as a ladder. Start with Option A. Climb one rung only when the labelled questions prove that the rung below fails on a question type that matters.

Option A: Tuned hybrid RAG

  • What it is: the best keyword, vector and semantic ranking setup for the corpus, with no graph.
  • When it fits: most questions are one-place questions, or the failures come from retrieval, chunking or prompts.
  • Cost and risk: the lowest. The team usually has it already. A graph does not fix a retrieval or chunking fault.
  • Carried out by: PB-RAG-08-02, with PB-RAG-01-01 for chunking.

Option B: Hybrid RAG with query decomposition

  • What it is: agentic retrieval in Azure AI Search. An LLM splits a complex question into focused subqueries. They run in parallel, each is semantically reranked, and the results are merged with source references.
  • When it fits: questions that ask several things at once, and connect-the-dots questions where each hop can be found by a search.
  • Cost and risk: adds latency. Billed as Azure AI Search retrieval tokens plus Azure OpenAI tokens for query planning. LLM query planning is preview (2026-08-01-preview); the generally available API (2026-04-01) covers minimal, extractive retrieval only. Available in select regions. Do not use a preview feature in production without an agreed fallback.
  • Carried out by: the quickstarts and tutorial linked from Agentic retrieval overview.

Option C: FastGraphRAG

  • What it is: Microsoft GraphRAG with graphrag index --method fast. Entities are noun phrases from NLP libraries; relationships come from co-occurrence in the same text unit.
  • When it fits: whole-corpus summary questions on a tight budget. Microsoft says it "provides high quality summarization with much lower language model cost".
  • Cost and risk: "much cheaper" indexing than Option D, but a noisier graph with no descriptions. The docs recommend much smaller chunks (50-100 tokens). Demonstration code in maintenance mode.
  • Carried out by: PB-RAG-11-02, with --method fast; the pilot steps are in Stage 4 below.

Option D: Full GraphRAG

  • What it is: Microsoft GraphRAG with the standard method. An LLM extracts entities, relationships and descriptions, then writes community summaries. Query with local search (specific entities), global search (the whole dataset) or DRIFT search (local search plus community information).
  • When it fits: connect-the-dots or summary questions where accurate entities matter, after Options A, B and C fail.
  • Cost and risk: the highest indexing cost; global search is resource-intensive; refresh can recompute much of the index. Demonstration code in maintenance mode.
  • Carried out by: PB-RAG-11-02; the pilot steps are in Stage 4 below.

Option E: Graph built from structured data

  • What it is: nodes and edges loaded from existing tables and foreign keys, with no LLM extraction. Retrieval follows the relationships from the entities in the question to the linked documents.
  • When it fits: connect-the-dots questions about people, teams, systems or products whose links already sit in a CRM, ERP, HR or asset system.
  • Cost and risk: no extraction cost. The graph must be kept in sync with the source tables, and that sync is your code.
  • Carried out by: the graph store in PB-RAG-11-02; the schema-driven pattern is shown in HorizonDB graph RAG.

Comparison of options

A: Tuned hybrid B: Query decomposition C: FastGraphRAG D: Full GraphRAG E: Graph from tables
Helps most with One-place questions Multi-ask and some connect-the-dots Whole-corpus summaries Connect-the-dots and summaries Connect-the-dots on known entities
LLM use at indexing None (embeddings only) None (embeddings only) Much lower than D Highest None for the graph
LLM use per query Answer only Query planning and answer Answer; global search costs most Answer; global search costs most Answer only
Preview parts None LLM query planning None None Depends on the store
Supported product Azure services Azure AI Search Demonstration code Demonstration code Depends on the store
Time to try Hours About a day One to two days Two days or more Depends on data access

The times are team estimates for a small slice, not platform facts.

The decision process

Run the stages in order. Stop as soon as a stage gives a clear answer. The commands use Linux shells.

What you need before you start

What Used in
30 or more real client questions, and the client's subject experts to write the expected answers Step 1
Corpus size in tokens, content change rate and monthly query volume Step 2
An approved token budget for any graph pilot Step 3
Azure AI Search with semantic ranker, and permission to create an index Steps 4 to 10
Two Azure OpenAI deployments used only for the evaluation: chat (${EVAL_DEPLOYMENT}) and embedding (${EVAL_EMBEDDING_DEPLOYMENT}), plus a judge model deployment Steps 4 to 8
Permission to read Azure Monitor metrics on the Azure OpenAI resource Steps 7, 13, 14 and 16
For Option B: a search service in a region that supports agentic retrieval Step 10
For a graph pilot: two more deployments used only by the pilot, with the same chat model as ${EVAL_DEPLOYMENT} Steps 12 to 16
Python 3.10-3.12, the azure-ai-evaluation and pyyaml packages, the Azure CLI, and a Linux shell with GNU date All steps

Team defaults

The following thresholds are team defaults, not platform facts. Agree them with the client before Stage 2 and write them in the decision record:

  • At least 10 labelled questions for each question type, and 30 or more in total.
  • Runs: answer and score every question 3 times for each option. The model's answers and the judge's verdicts change from run to run, so one run can mislead.
  • Accuracy target: 80% for each question type that has high-priority questions, measured as the mean of the 3 runs.
  • Go margin: on the failing high-priority types, a graph option must get at least 2 more questions right out of 10 (20 percentage points) than the best cheaper option, in every one of the 3 runs. Otherwise, do not build it. With only 10 questions per type, two equally good options can differ by 2 in a single run by chance. If the result is close, add 10 more questions for the failing type before you decide.

Stage 1: Prepare

Step 1. Label the client's questions.

Copy snippets/eval/question-types.yaml to eval/questions.yaml. Add one entry per real client question with its type, priority, expected_answer and source_documents. Choose the type with the tests and rules in the three question types. Ask the client's subject experts to write the expected answers.

python snippets/eval/score_by_type.py --questions eval/questions.yaml --summary

Verify: the command prints a table with at least 10 questions for each type and 30 or more in total. If a type has fewer, write more questions with the client's experts.

Step 2. Choose the evaluation slice and record the corpus facts.

Copy snippets/decisions/graphrag-go-no-go.md to the project's design records. Choose a slice of the corpus that holds every document in source_documents and is representative of the rest. Use the same slice for every option. Fill section 1 of the record: corpus tokens, slice tokens, content change rate in tokens per month and query volume per month. Count tokens with the tokenizer of your model, not documents.

Verify: the corpus, evaluation slice, content change rate and query volume rows in section 1 of the decision record are filled.

Step 3. Agree the targets and the pilot budget.

Agree the accuracy target per type with the client. Agree a token budget for any graph pilot with the RAG lead. Write both in section 1 of the decision record.

Verify: the record names who approved the budget and when.

Stage 2: Measure the baseline (Option A)

Spends money: evaluation index and runs

Steps 4 to 8 index the slice and answer every question 3 times. This uses embedding and model tokens and creates a search index. Use an index and deployments made only for this evaluation. To clean up after the decision, delete the evaluation index and the evaluation deployments.

Step 4. Index the slice for Option A.

Index the slice into a new search index with the best hybrid configuration (PB-RAG-08-02). Use an embedding deployment that nothing else uses during indexing, ${EVAL_EMBEDDING_DEPLOYMENT}. Write the indexing start and end times in UTC.

Verify: indexing ends without errors, and the index has content from every file in the slice.

Step 5. Run every question through Option A, 3 times.

Generate answers from the new index with a model deployment that nothing else uses during the runs, ${EVAL_DEPLOYMENT}. Write the start and end times of each run in UTC. For run <N> (1, 2 or 3), save each answer as answers/A/run<N>/<question id>.txt and add one row per question to eval/results-run<N>.csv, with the columns question_id,option,passed,latency_ms. Measure latency_ms from sending the question to receiving the full answer. Leave passed empty for now.

Verify: each of the 3 results files has one A row for every question, each with a latency.

Step 6. Score each answer against the expected answer.

Use the same judge model and threshold for every option and every run. ResponseCompletenessEvaluator compares a response with the ground truth and returns pass or fail at a threshold; the SDK default is 3 (RAG evaluators). This guide uses 4 as the team default, because a decision needs complete answers. Set up model_config for the judge as in PB-RAG-02-02.

from azure.ai.evaluation import ResponseCompletenessEvaluator

judge = ResponseCompletenessEvaluator(model_config=model_config, threshold=4)
result = judge(response=answer_text, ground_truth=expected_answer)
passed = result["response_completeness_result"] == "pass"

Write pass or fail into the passed column of each results file. Then have a person check the verdicts. Response completeness measures recall: whether the answer misses points of the expected answer. Precision, meaning no content outside the sources, is a different measure (RAG evaluators). So the judge does not catch wrong facts, and a longer answer covers more points.

  • Read every fail, and correct any wrong verdict.
  • Read every pass for connect-the-dots and summary questions, and five pass answers per run for one-place questions.
  • Change a pass to fail if the answer states a fact that the source documents contradict or do not contain.

Verify: every A row in the 3 results files has pass or fail, and the person's checks are done.

Step 7. Read the tokens used for indexing and for run 1.

Read the input and output tokens twice: for ${EVAL_EMBEDDING_DEPLOYMENT} between the indexing start and end times, and for ${EVAL_DEPLOYMENT} between the start and end times of run 1. Set DEPLOYMENT, START_TIME and END_TIME for each. The metric names and the ModelDeploymentName dimension come from the Azure Monitor metrics reference.

az monitor metrics list \
  --resource "${AOAI_RESOURCE_ID}" \
  --metrics ProcessedPromptTokens GeneratedTokens \
  --aggregation Total \
  --start-time "${START_TIME}" \
  --end-time "${END_TIME}" \
  --filter "ModelDeploymentName eq '${DEPLOYMENT}'" \
  --query "value[].{metric: name.value, total: sum(timeseries[].data[].total)}" \
  --output table

Add two rows to eval/tokens.csv, with the columns option,phase,prompt_tokens,generated_tokens: A,indexing,<ProcessedPromptTokens>,<GeneratedTokens> and A,query,<ProcessedPromptTokens>,<GeneratedTokens>. Keep the prompt and generated tokens apart, because they have different prices. Read query tokens for run 1 only, so that the cost per 1,000 queries counts each question once.

Verify: both prompt-token totals are above zero. Generated tokens can be zero or empty for an embedding deployment; write 0. If a prompt-token total is zero, check the deployment name and that the times are in UTC.

Step 8. Score Option A by question type.

Run the script once for each run. Add --tokens to run 1 only.

python snippets/eval/score_by_type.py --questions eval/questions.yaml \
  --results eval/results-run1.csv --tokens eval/tokens.csv
python snippets/eval/score_by_type.py --questions eval/questions.yaml --results eval/results-run2.csv
python snippets/eval/score_by_type.py --questions eval/questions.yaml --results eval/results-run3.csv

Paste the outputs into sections 3 and 4 of the decision record.

Verify: the accuracy tables show Option A for every question type in all 3 runs. If every type with high-priority questions meets its target (the mean of the 3 runs), stop here: the decision is no-go, stay with Option A. Go to Step 19.

Stage 3: Try the cheaper fixes (Options B and E)

For summary failures, first check whether the whole corpus fits in the answer model's context window (check current limits in Azure OpenAI models). If it does, putting the whole corpus in the prompt can answer summary questions without a graph: see the decision table.

Step 9. Fix retrieval faults before you blame the method.

Read the failed answers. If the right passage was not retrieved for a one-place question, tune hybrid search or chunking first (PB-RAG-08-02, PB-RAG-01-01), then repeat Steps 4 to 8.

Verify: every remaining failure is a connect-the-dots or summary question, or a one-place question that you can explain.

Spends money: agentic retrieval

Agentic retrieval is billed as Azure AI Search retrieval tokens plus Azure OpenAI tokens for query planning. Check the current price before approval. To clean up after the decision, delete the knowledge base you created.

Step 10. Try Option B for multi-ask and connect-the-dots failures.

Set up agentic retrieval over the Option A index with the quickstart linked from the Agentic retrieval overview. Use 2026-08-01-preview for LLM query planning. Do not use answer synthesis. Generate each answer with ${EVAL_DEPLOYMENT} and the same answer prompt as Option A, from the content that retrieval returns, so that only the retrieval differs. Run all questions, not only the failures, and repeat Steps 5 to 8 with option B. Read tokens for both the answer deployment and the deployment the knowledge base uses for query planning. Option B searches the Option A index, so copy Option A's indexing row in eval/tokens.csv as a B row.

Verify: each results file has a scored B row for every question, and the scores tables show Option B.

Spends money: graph store

Loading a graph store creates a billed resource. Check the current price before approval. To clean up after the decision, delete the graph store.

Step 11. Try Option E when the links sit in structured data.

Skip this step if the relationships in the failing questions exist only in document text. Otherwise load the entities and relationships from the source tables into the graph store from PB-RAG-11-02. Link each entity to the documents that mention it. Run all questions and repeat Steps 5 to 8 with option E.

Verify: the scores tables show Option E. If Option B or E meets the target for every high-priority type, go to Step 18.

Stage 4: Run a graph pilot (Options C and D)

Run this stage only if a high-priority type still misses its target. Try Option C first for summary questions. Use Option D when connect-the-dots answers need accurate entities, or when Option C's graph is too noisy.

Spends money: LLM indexing of the slice

Indexing calls the LLM for the slice and cannot be undone or refunded. Run it only with an approved token budget (Step 3) and a model deployment used only by the pilot. If the sample in Step 13 projects more than the budget, stop and re-estimate. To clean up, delete ${PILOT_ROOT} and the pilot deployments.

Step 12. Set up the pilot workspace.

Use Python 3.10-3.12. Install the GraphRAG version you will pin, then create the workspace (Getting started).

python -m pip install "graphrag==${GRAPHRAG_VERSION}"
graphrag init --root "${PILOT_ROOT}" --model "${PILOT_CHAT_MODEL}" --embedding "${PILOT_EMBEDDING_MODEL}"

In ${PILOT_ROOT}/settings.yaml, point the chat and embedding model blocks at Azure OpenAI deployments used only by the pilot. Use managed identity: set auth_method: azure_managed_identity, remove the api_key line, then run az login. Full set-up is in PB-RAG-11-02.

# Fields of the chat model block under models: in settings.yaml.
# Set the same fields in the embedding model block, with the embedding deployment.
model_provider: azure
azure_deployment_name: ${PILOT_CHAT_DEPLOYMENT}
api_base: https://${AOAI_RESOURCE_NAME}.openai.azure.com
api_version: ${AOAI_API_VERSION}
auth_method: azure_managed_identity

Set up the pilot as carefully as Option A, so that only the graph differs:

  • Deploy the same model and model version for ${PILOT_CHAT_DEPLOYMENT} as for ${EVAL_DEPLOYMENT}. A separate deployment keeps the token counts apart.
  • For Option C, set size under chunking in settings.yaml to 50-100 tokens, as the Indexing methods page recommends (YAML configuration).

Verify: ${PILOT_ROOT} contains settings.yaml, .env and an input folder; settings.yaml has no API key; and ${PILOT_CHAT_DEPLOYMENT} runs the same model and version as ${EVAL_DEPLOYMENT}.

Step 13. Tune the prompts, index a small sample and project the cost.

Convert a sample of the slice to plain text files (team default: 10% of the slice tokens) and copy them to ${PILOT_ROOT}/input. Tune the prompts on this sample, as the README recommends: out of the box, GraphRAG may not give the best results on your data. graphrag prompt-tune reads the input files and writes the tuned prompts to the prompts folder (Auto prompt tuning). Point the prompt settings in settings.yaml at the tuned files, for example prompt under extract_graph, as that page shows. Then validate the configuration and index the sample. Use --method fast for Option C or --method standard for Option D.

graphrag prompt-tune --root "${PILOT_ROOT}"
graphrag index --root "${PILOT_ROOT}" --method "${METHOD}" --dry-run
graphrag index --root "${PILOT_ROOT}" --method "${METHOD}"

Read the indexing tokens for the pilot chat and embedding deployments as in Step 7. Project the slice indexing tokens: sample indexing tokens × (slice tokens ÷ sample tokens).

Verify: ${PILOT_ROOT}/prompts holds the tuned prompts and the prompt settings in settings.yaml point at them, the dry run ends without errors, the output folder has parquet files, and the projected slice tokens are within the approved budget. If not, stop and re-estimate with the RAG lead.

Step 14. Index the rest of the slice.

Copy the remaining slice text files to ${PILOT_ROOT}/input and run graphrag index again with the same method. The LLM cache returns cached results for identical requests, so the extraction calls for the sample are not paid for twice (Indexing architecture). The community summaries are rebuilt, because the graph has changed. Read the tokens for both pilot deployments as in Step 7, and add the tokens of both indexing runs as one indexing row in eval/tokens.csv.

Verify: the run ends without errors and the total indexing tokens are within the approved budget.

Step 15. Answer every question with the pilot, 3 times.

Use local search for one-place and connect-the-dots questions and global search for summary questions (Query overview). Set OPTION to C or D. date +%s%3N needs GNU date. The loop prints the UTC start and end time of each run.

python snippets/eval/score_by_type.py --questions eval/questions.yaml --export > eval/questions.tsv
for run in 1 2 3; do
  echo "run ${run} start: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
  mkdir -p "answers/${OPTION}/run${run}"
  while IFS=$'\t' read -r qid qtype question; do
    method=local; [ "${qtype}" = "whole_corpus_summary" ] && method=global
    start=$(date +%s%3N)
    graphrag query "${question}" --root "${PILOT_ROOT}" --method "${method}" < /dev/null > "answers/${OPTION}/run${run}/${qid}.txt"
    echo "${qid},${OPTION},,$(( $(date +%s%3N) - start ))" >> "eval/results-run${run}.csv"
  done < eval/questions.tsv
  echo "run ${run} end: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
done

Verify: each answers/${OPTION}/run<N> folder has one file per question, and each results file has one row per question for the option.

Step 16. Score the pilot answers and read the query tokens.

Repeat Step 6 for the pilot answers in all 3 runs. Then repeat Step 7 for the pilot chat and embedding deployments over run 1, as a query row in eval/tokens.csv.

Verify: every pilot row in the 3 results files has pass or fail, and eval/tokens.csv has indexing and query rows for the option.

Stage 5: Compare and decide

Step 17. Build the comparison tables.

Run the commands from Step 8 again. Paste the outputs into sections 3 and 4 of the decision record.

Verify: the record shows accuracy per question type in all 3 runs, p95 latency and indexing tokens for every option tried.

Step 18. Extrapolate the costs to full scale.

Fill the rest of section 4 of the decision record for every option tried. Keep prompt and generated tokens apart in every row, because they have different prices.

  • Full-corpus indexing tokens = slice indexing tokens × (corpus tokens ÷ slice tokens).
  • Monthly refresh tokens: a best case (only the changed content is re-indexed) and a worst case (a full re-index each refresh). Much of the index may be recomputed, so do not plan on the best case alone.
  • Monthly query tokens = query tokens per 1,000 queries × monthly query volume ÷ 1,000.
  • Add the other charges: Azure AI Search, the graph store and storage.

Convert tokens to cost with current prices. Check the current price before approval, and write the date you checked it.

Verify: section 4 has no empty cell for the options tried, and the price check date is filled.

Step 19. Decide, record and sign off.

Use the decision table in section 5 and the default in section 6. Fill sections 5 to 7 of the decision record. Choose go only when all four are true:

  • A high-priority question type misses its target with Options A, B and E.
  • The graph option gets at least 2 more questions right out of 10 than the best cheaper option on those types, in every one of the 3 runs (the go margin).
  • The full-scale cost, including the worst-case refresh, fits the budget.
  • The RAG lead accepts the risks: preview features, and demonstration code in maintenance mode.

Verify: the reason in section 6 cites the numbers in sections 3 and 4, and the RAG lead has signed section 7.

5. Decision table

Situation Choose Why
Every type with high-priority questions meets its target on Option A A (no-go for a graph) No labelled failures, no graph.
Most failures are one-place questions A, after tuning retrieval and chunking A graph does not fix a retrieval or chunking fault.
Questions ask several things at once B Query decomposition splits them into subqueries. Planning is preview.
Connect-the-dots failures, and the links sit in structured data E The graph comes from the tables, with no extraction cost.
Connect-the-dots failures, links only in text, B still misses the target D (pilot) Only LLM extraction finds links that exist only in text.
Summary failures, and the whole corpus fits in the answer model's context window A, with the whole corpus in the prompt for summary questions No graph to build or refresh. Each summary question pays for the whole corpus in input tokens, so compare the cost per 1,000 queries.
Summary failures on a tight budget C (pilot) Much cheaper indexing, and good summaries.
Summary failures, and C's answers name wrong themes D (pilot) Standard extraction gives richer entities and descriptions.
Connect-the-dots and summary questions are under 20% of real traffic (team default), and none is high priority A or B The graph cost serves too few questions.
The pilot gets fewer than 2 more questions right out of 10 than the best cheaper option, in any of the 3 runs No-go; stay with A or B The gain does not justify the build, refresh and support cost.
The pilot used default prompts, default chunks or a different answer model No decision yet Trap: an untuned pilot against a tuned Option A is not a fair test. Tune it as in Steps 12 and 13.
The client asks for "GraphRAG" but has no labelled questions No decision yet Trap: label the questions first. No evidence, no build.
The pilot looks good, but full-scale and refresh costs are not modelled No decision yet Trap: extrapolate in tokens, not documents, and include the worst-case refresh.
The client needs a supported product with an SLA Record the risk; the RAG lead decides Trap: Microsoft GraphRAG is demonstration code in maintenance mode.
The content changes daily Model the worst-case refresh before any go Trap: new content can force much of the index to be recomputed.

Default: Option A, tuned hybrid RAG. Most enterprise questions are one-place questions, and hybrid RAG is the cheapest to run, refresh and support.

Deviate when the labelled questions show that a high-priority type misses its target, and the evidence points to the next rung:

  • Multi-ask questions fail: try Option B.
  • Links between entities sit in structured data: try Option E.
  • Summary questions fail on a tight budget: pilot Option C.
  • Connect-the-dots or summary questions still fail, the graph gets at least 2 more questions right out of 10 in all 3 runs, and the full-scale cost is affordable: pilot, then go with Option D.

A go rarely replaces hybrid RAG. Keep Option A for one-place questions and add the graph only for the types that need it.

Rule of thumb: no labelled failures, no graph.

7. Known issues and fixes

Symptom Likely cause Fix
The pilot looks good but full-scale cost explodes Indexing cost grows with corpus tokens and extraction calls Extrapolate from the slice in tokens, not documents, and include the worst-case refresh (Step 18).
The go or no-go result changes when you re-run One run of 10 questions per type is noisy Run every option 3 times and apply the go margin to every run (see Team defaults).
A graph option wins mainly with much longer answers A longer answer covers more expected points, and the judge checks coverage only Check the passes by hand and fail answers with wrong or unsupported facts (Step 6). Compare answer lengths per option.
The pilot loses, although the questions clearly need a graph The pilot was not set up as carefully as Option A: default prompts, default chunks or a different answer model Use the same answer model, set the Option C chunk size and tune the prompts (Steps 12 and 13).
The cost per 1,000 queries looks wrong Prompt and generated tokens were added together, or query tokens were read over all 3 runs Keep the two token counts apart, and read query tokens for run 1 only (Step 7).
Graph answers lack citations Community summaries abstract away the source text Keep links from entities and summaries to source chunks, and return them with answers (PB-RAG-11-02).
The pilot wins on the slice but loses at full scale The slice was chosen for convenience and misses some question types Choose a slice that holds every source document and represents the rest (Step 2).
Options cannot be compared Different questions, slice, answer model or judge per option Use the same questions, slice, answer model and judge for every option.
The judge passes answers that a person marks wrong The judge threshold is too low, the expected answer is vague, or the answer adds wrong facts (the judge checks coverage only) Use threshold 4, write specific expected answers, and check the verdicts by hand as in Step 6.
az monitor metrics list shows zero tokens Wrong deployment name in the filter, or local times instead of UTC Check ModelDeploymentName and give the start and end times in UTC (Step 7).
Agentic retrieval query planning is not available The region is not supported, or the API version is not the preview one Check region support in the agentic retrieval docs and use 2026-08-01-preview.
graphrag index fails to connect to Azure OpenAI Managed identity is not signed in, or the settings point at the wrong deployment Run az login, then check api_base and azure_deployment_name in settings.yaml (Step 12).
FastGraphRAG summaries name wrong themes Co-occurrence links are noisy Check a sample of answers by hand; move those questions to an Option D pilot.
Query cost is far above hybrid RAG Global search is resource-intensive Compare query tokens per 1,000 queries per option; use local or DRIFT search where they answer well.
A GraphRAG upgrade breaks the pilot Configuration changes between versions Pin the version; run graphrag init --root <path> --force between minor versions, after backing up settings.yaml and the prompts.

8. Monitoring and alerts

The thresholds below are team defaults, not platform recommendations. Tune them against a measured baseline.

Signal Source Alert when Action
ProcessedPromptTokens plus GeneratedTokens for the pilot deployment (ModelDeploymentName) Platform metric (Total) Above 80% of the approved pilot budget Pause the pilot and re-estimate the remaining tokens with the RAG lead.
The same pilot token total Platform metric (Total) Reaches 100% of the approved pilot budget Stop the indexing job. Do not restart without a new approval.
Share of connect-the-dots and summary questions in a monthly sample of 50 production questions Manual labelling Moves more than 10 percentage points from the share in the decision record Re-run this decision process with the new mix.
Monthly graph refresh tokens, after a go Platform metric (Total) for the refresh deployment Above 125% of the modelled monthly refresh tokens for 2 months in a row Review the refresh plan with PB-RAG-11-04 and update the decision record.
Accuracy per question type on the labelled set, after a go Evaluation run Falls below the target for any high-priority type Re-run Stage 2 and Stage 4 checks and review the decision.

Metric names and dimensions: Azure Monitor metrics reference.

9. Cost and capacity notes

  • Never quote prices. Convert tokens to cost with current prices, and check the current price before approval.
  • Indexing: Option D pays for LLM extraction on every chunk; graph extraction is roughly 75% of its indexing cost. Option C costs much less. Options A, B and E pay only for embeddings.
  • Queries: Option B adds query-planning tokens and Azure AI Search retrieval tokens. Global search in Options C and D is resource-intensive.
  • Refresh: new content can force much of a GraphRAG index to be recomputed. Budget for the worst case.
  • Re-runs: the GraphRAG LLM cache returns cached results for identical requests, so re-runs over unchanged content cost much less.
  • Evaluation runs: 3 runs triple the query tokens of the evaluation, but each option is indexed only once.
  • Token types: prompt (input) and generated (output) tokens have different prices. Keep them apart in every estimate.
  • Other charges: add Azure AI Search, the graph store and storage for each option.

10. AWS and open-source equivalents

Platform Equivalent Key differences
Amazon Bedrock Knowledge Bases GraphRAG with Amazon Neptune Analytics (fully managed) Amazon S3 data source only. The graph build cannot be customised. Up to 1,000 files per data source, which can be raised to 10,000. Available in selected regions. Deleting the knowledge base does not delete the graph, and charges continue until you delete it.
Open source Microsoft GraphRAG library Runs on any cloud with any supported model. Demonstration code in maintenance mode.
Open source Neo4j GraphRAG for Python (neo4j-graphrag) The official Neo4j package. Supports Azure OpenAI through its openai extra and needs Neo4j 5.18.1 or later. You build the extraction and retrieval yourself.

All of them need the same labelled evidence before a go decision. Sources: AWS docs, Neo4j GraphRAG for Python.

11. Validation record

Pending. Filled in by the dry-runner (see docs/validation-protocol.md).

Sandbox scenario: run Steps 1 to 19 on the five open-licence policy documents in data/golden/A-policies/, with the 30 labelled questions in snippets/eval/a-policies-questions.yaml (10 per type, each with its evidence). Use all five documents as the evaluation slice. Compare Option A with an Option C pilot; add Option D if the sandbox budget allows.

Pass criteria:

  • The decision record shows accuracy per question type for Option A and the pilot, in all 3 runs.
  • It shows the indexing cost and the cost per 1,000 queries.
  • The decision follows from these numbers, and every step's Verify line gave the expected result.

Record

Item Detail
Playbook PB-RAG-11-01 at commit [hash]
Dry-run by [Name] (must not be author or reviewer)
Date YYYY-MM-DD
Sandbox [Resource group or tag, tier and region]
Scenario [Scenario followed]
Time taken [Hands-on time] hands-on, [waiting time] waiting
Result [PASS / PASS WITH NOTES / FAIL]
Notes 1. [Step number: defect or explanation; blocking/non-blocking; fixed/logged]
Signed off by [Reviewer name and date]

12. Sources

Checked on 6 November 2026. The GraphRAG CLI, Getting started, YAML configuration and Auto prompt tuning pages, RAG evaluators and Agentic retrieval overview were checked again on 11 November 2026.

  • PB-RAG-01-01: fix chunking before blaming the retrieval method.
  • PB-RAG-02-01 and PB-RAG-02-02: build the labelled question set and score answers.
  • PB-RAG-08-01 and PB-RAG-08-02: choose and tune hybrid RAG (Option A).
  • PB-RAG-11-02: build the graph pilot (Options C, D and E).
  • PB-RAG-11-03: cost model and go-live checklist.
  • PB-RAG-11-04: refresh the knowledge graph after a go.