Skip to content
⚠️ Before You Start
Risk: Medium
Duration: 1-2 weeks for a pilot
Last Validated: Not yet validated (draft)
Kind: How-to
Risk Note: Medium (pilot scope; the main risk is cost overrun and misleading results)

Build a GraphRAG proof of concept with Azure OpenAI and Neo4j or Cosmos DB for Apache Gremlin

Draft outline: not validated

Drafted with AI from official sources on 16 Oct 2026. Do not use on a live system until it is written, dry-run by someone other than the author and signed off (planned 2026-12-14). Author: Deeparani · dry-run: Sivaraj · sign-off: Yogeshwaran. Work items: RAG-11-T05, RAG-11-T06, RAG-11-T07, RAG-11-T08.

1. Summary

After the decision guide shows that a pilot is justified.

Risk: Medium (pilot scope; the main risk is cost overrun and misleading results). Typical duration: 1-2 weeks for a pilot.

2. When to use and when not to

Use it when: After the decision guide shows that a pilot is justified.

Do not use it when: to be written by the author.

3. What you must know first

  • Entity and relationship extraction quality decides graph quality; tune extraction prompts on the domain
  • Neo4j vector indexes need 5.11 or later; Cypher 25 adds a SEARCH clause
  • Cosmos DB for Apache Gremlin is wire-compatible with Apache TinkerPop, but some features differ
  • Keep links from chunks to entities so answers can cite sources
  • Pin library versions; GraphRAG publishes breaking changes between versions

4. Prerequisites and access

  • Azure OpenAI deployments for extraction, chat and embeddings with agreed quota
  • Neo4j 5.11 or later for vector indexes, or Azure Cosmos DB for Apache Gremlin
  • An agreed corpus slice
  • Python environment with library versions pinned

5. Steps

Outline of the steps; the author adds the commands and a check after each one.

  1. Pick the store: Neo4j for Cypher and vector search together, or Cosmos DB for Apache Gremlin if the client standardises on Cosmos DB.
  2. Chunk the corpus slice and store chunks with stable ids.
  3. Extract entities and relationships with a tuned prompt and check a sample by hand.
  4. Load nodes, relationships and chunk links into the graph.
  5. Create a vector index on chunk embeddings.
  6. Implement retrieval: vector match, then expand one or two hops through related entities.
  7. Generate answers with citations to source chunks.
  8. Run the labelled questions and compare with hybrid RAG.
  9. Record token usage and costs for the cost model (PB-RAG-11-03).

Draft snippet: Create a chunk vector index and expand matches one hop in Neo4j

"""Create a chunk vector index and expand matches one hop in Neo4j."""
import os

from neo4j import GraphDatabase

driver = GraphDatabase.driver(os.environ["NEO4J_URI"],
                              auth=(os.environ["NEO4J_USER"], os.environ["NEO4J_PASSWORD"]))

driver.execute_query("""
CREATE VECTOR INDEX chunk_embeddings IF NOT EXISTS
FOR (c:Chunk) ON (c.embedding)
OPTIONS {indexConfig: {`vector.dimensions`: 1536, `vector.similarity_function`: 'cosine'}}
""")


def retrieve(query_vector, k=5):
    records, _, _ = driver.execute_query("""
    CALL db.index.vector.queryNodes('chunk_embeddings', $k, $vector)
    YIELD node AS chunk, score
    OPTIONAL MATCH (chunk)-[:MENTIONS]->(e:Entity)-[r]-(related:Entity)
    RETURN chunk.id AS chunk_id, chunk.text AS text, score,
           collect(DISTINCT e.name + ' ' + type(r) + ' ' + related.name)[..20] AS facts
    """, k=k, vector=query_vector)
    return [record.data() for record in records]

Snippets to add:

  • snippets/graphrag/neo4j_retrieve.py: Neo4j vector index and one-hop retrieval (python)
  • snippets/graphrag/extraction-schema.json: Entity and relationship extraction schema (json)
  • snippets/graphrag/answer.py: Answer generation with chunk citations (python)

6. Verify it works

A second engineer builds the pilot from the pack on a public corpus slice. Pass: the graph loads with a duplicate-entity rate below the agreed limit, the labelled questions run end to end with citations, and token usage is recorded.

7. Known issues and fixes

Symptom Likely cause Fix
Duplicate entities (the same company under different names) No entity resolution Normalise names and merge on stable identifiers.
Graph expansion returns far too much context Unbounded hops or very highly connected nodes Limit hops and neighbours and rank facts before sending them to the model.
A library upgrade breaks the pipeline Breaking changes between versions Pin versions and follow the project's migration notes before upgrading.

8. Monitoring and alerts

Signal Alert when Action
Tokens per indexing run (Azure OpenAI ProcessedPromptTokens and GeneratedTokens) Above the run budget Stop and review chunking and extraction prompts.

9. AWS and open-source equivalents

Amazon Bedrock Knowledge Bases can build the graph in Amazon Neptune Analytics automatically from S3 data, but its graph build options cannot be customised. Microsoft GraphRAG is the open-source reference for community summaries.

10. Validation record

Pending. Planned dry-run scenario: A second engineer builds the pilot from the pack on a public corpus slice. Pass: the graph loads with a duplicate-entity rate below the agreed limit, the labelled questions run end to end with citations, and token usage is recorded.

11. Sources