Skip to content
⚠️ Before You Start
Risk: Medium
Duration: 1-2 days including a sample run
Last Validated: Not yet validated (draft)
Kind: Decision guide
Risk Note: Medium (the wrong path drives cost and rework across the whole corpus)

Choose the extraction path for tables, figures and scans

Draft outline: not validated

Drafted with AI from official sources on 16 Oct 2026. Do not use on a live system until it is written, dry-run by someone other than the author and signed off (planned 2026-12-04). Author: Kandiah Chockalingam · dry-run: Ranjith · sign-off: Yogeshwaran. Work items: RAG-10-T01, RAG-10-T02, RAG-10-T03, RAG-10-T04.

Catalog point to cover

The catalog stack names GPT-4o vision: name the vision-capable Azure OpenAI model used to describe figures and check its retirement date in the docs.

1. Summary

At the start of a document-heavy engagement, or when tables and figures are lost in an existing pipeline.

Risk: Medium (the wrong path drives cost and rework across the whole corpus). Typical duration: 1-2 days including a sample run.

2. When to use and when not to

Use it when: At the start of a document-heavy engagement, or when tables and figures are lost in an existing pipeline.

Do not use it when: to be written by the author.

3. What you must know first

  • The Content Understanding skill is generally available in the 2026-04-01 API, and Microsoft points new skillsets to it
  • Content Understanding image descriptions and semantic chunking are preview (2026-05-01-preview)
  • The Document Layout skill outputs Markdown and remains supported for existing pipelines
  • Document Intelligence Markdown renders tables as HTML and wraps figures in figure tags
  • Image verbalisation gives citable text; multimodal embeddings only give visual similarity
  • Documents needing more than five minutes of processing time out but are still billed

4. Options

Outline of the decision process; the author describes each option.

  1. Classify the documents: digital text, scanned, table-heavy, diagram-heavy or photos.
  2. Run the sample through the Content Understanding skill and the Document Layout skill in a sandbox.
  3. Score table fidelity (cells, merged headers), figure capture and reading order by hand on 20 pages.
  4. Decide how to handle figures: verbalise with an LLM, embed images, or both.
  5. Estimate cost per 1,000 pages for each path, including LLM calls for verbalisation.
  6. Pick the path in the decision table and record the evidence.
  7. Re-check when a preview feature becomes generally available or pricing changes.

Draft snippet: Content Understanding skill with fixed-size chunking, images and location metadata

{
  "skills": [
    {
      "description": "Analyse a document",
      "@odata.type": "#Microsoft.Skills.Util.ContentUnderstandingSkill",
      "context": "/document",
      "extractionOptions": ["images", "locationMetadata"],
      "chunkingProperties": {
        "unit": "characters",
        "maximumLength": 1325,
        "overlapLength": 0
      },
      "inputs": [ { "name": "file_data", "source": "/document/file_data" } ],
      "outputs": [
        { "name": "text_sections", "targetName": "text_sections" },
        { "name": "normalized_images", "targetName": "normalized_images" }
      ]
    }
  ]
}

Snippets to add:

  • snippets/skillsets/content-understanding.json: Content Understanding skill with images and location metadata (json)
  • snippets/decisions/extraction-path.md: Extraction path decision table (markdown)
  • snippets/eval/extraction_cost.py: Cost per 1,000 pages estimator (python)

5. Decision table

Situation Choose Why
to be written

To be written by the author from the outline above.

7. Known issues and fixes

Symptom Likely cause Fix
Tables split across chunks lose their headers Fixed-size chunking cuts through tables Use layout-aware chunking, keep tables whole where possible and repeat headers.
Figures produce no useful text The path extracts images but does not describe them Add image verbalisation for explanation-rich figures.
Unexpected extraction bill Content Understanding has no free allotment, and timed-out documents are still charged Cap pages in the sample run and set a budget alert before scaling.

8. Monitoring and alerts

Signal Alert when Action
Daily cost of the Foundry resource attached to the skillset (Cost Management) 25% above forecast Disable the indexer schedule and review the extraction path.

9. AWS and open-source equivalents

On AWS, Amazon Textract (AnalyzeDocument with tables and layout) and Amazon Bedrock Data Automation cover similar extraction. Open-source parsers can run locally for sensitive data but need the same quality tests.

10. Validation record

Pending. Planned dry-run scenario: A second engineer runs the decision process on a 50-document sample from a real engagement. Pass: the decision table holds table-fidelity scores, figure coverage and cost per 1,000 pages for at least two paths, and the choice follows from them.

11. Sources