⚠️ Before You Start

Risk: High

Duration: 1-2 hours

Last Validated: Not yet validated (draft)

Kind: Checklist

Risk Note: High (missing evidence fails audits)

HITL audit trail and compliance sign-off checklist¶

Draft outline: not validated

Drafted with AI from official sources on 16 Oct 2026; updated on 22 Oct 2026 for task AGT-05-T17. Do not use on a live system until it is written, dry-run by someone other than the author and signed off (planned 2026-12-03). Author: Sanjay Sarvesh · dry-run: Yasar · sign-off: James. Work items: AGT-05-T17, AGT-05-T18, AGT-05-T19, AGT-05-T20.

1. Summary¶

This checklist validates the human-in-the-loop (HITL) audit trail, authorization boundaries, and compliance controls before promoting an agentic approval flow to production. It confirms that every action request, decision, edit, rejection, and timeout produces an immutable, non-repudiable audit record that satisfies regulatory scrutiny and internal risk policies. Conduct this review jointly between the GenAI engineer, Security engineer, and Product owner prior to go-live and during scheduled compliance audits.

Risk: High (missing evidence fails audits and exposes systems to unauthorized agent actions). Typical duration: 1-2 hours.

2. When to use and when not to¶

Use it when:

Conducting the pre-production compliance review for any agent with human-in-the-loop approval gates.
Auditing existing approval workflows after updating model versions, tool definitions, or authorization policies.
Validating regulatory readiness for enterprise standards (such as SOC 2 Type II, ISO 27001, or financial audit mandates).
Investigating suspected authorization bypasses or discrepancies in approval logging.

Do not use it when:

Building the core LangGraph approval interrupt nodes and checkpointer logic: follow PB-AGT-05-01.
Implementing Microsoft Teams or email delivery channels and callback endpoints: follow PB-AGT-05-02.
Triaging an active queue backlog or clearing timed-out approvals during an operational incident: follow PB-AGT-05-03.
Conducting the broader technical go-live review for agent infrastructure, tracing, and eval gates: follow PB-AGT-01-05.

3. What you must know first¶

Mitigate excessive agency with mandatory human gates. OWASP LLM06:2025 identifies excessive autonomy as a core vulnerability; high-impact or irreversible actions must require human review outside the LLM context (OWASP LLM06:2025 Excessive Agency).
Identity must originate from verified IdP claims. Extract the approver identity strictly from cryptographically validated security tokens (oid claim in Microsoft Entra ID, with tid), never from user-supplied card payloads or unauthenticated client request bodies (Access token claims reference).
Audit records must be append-only and tamper-proof. Audit sinks must use Write-Once-Read-Many (WORM) storage with locked retention policies to prevent modification or deletion, even by administrative identities (Azure Blob Storage immutable storage overview, Amazon S3 Object Lock overview).
Enforce segregation of duties (Four-Eyes Principle). The agent system must ensure the user who initiated the original agent run cannot approve their own high-risk action request.
Record evidence across all four resolution paths. Approvals, edits, rejections, and timeouts must all write audit records; unapproved or expired paths require proof of non-execution.
Re-validate edited parameters before execution. When an approver modifies arguments, the system must re-validate the new values against the strict tool schema before dispatching the action.

Prerequisites¶

Audit table schema
Sample decisions
Retention policy

4. Checklist¶

Walk through the checklist items during the compliance sign-off meeting. Verify each control by examining configuration, inspecting database records, and running end-to-end sandbox tests.

Section 4.1: Audit schema and data integrity¶

Request capture: Every interrupt writes an initial audit record containing thread_id, run_id, action name, original tool arguments, initiator identity, and UTC timestamp ${REQUESTED_AT}.
Verify: Inspect the audit store after an interrupt and confirm all fields are populated and match the graph state snapshot, and confirm exactly one request record exists per approval after the thread has been resumed.
Decision capture: Every resolution writes a decision audit record containing thread_id, action name, decision type (approve, edit, reject, timeout), approver identity, comments, and UTC timestamp ${DECIDED_AT}.
Verify: Trigger each decision type in staging and verify the audit sink records matching decision events.
Argument modification diff: For edit decisions, the audit record explicitly captures both the original parameters and the edited parameters in a structured diff.
Verify: Submit an edited approval and confirm the audit payload clearly delineates between original and revised values.
Execution correlation: The downstream execution node logs the outcome and includes an idempotency key ${THREAD_ID}:${ACTION_NAME} linked back to the approval audit record.
Verify: Query downstream execution logs and confirm the idempotency key matches the approved audit record.
Redaction of sensitive data: Audit records scrub API keys, credentials, and customer personal data (PII) before committing to persistent storage.
Verify: Submit a test payload containing synthetic test PII and confirm redacting filters mask the values in the audit store.

Section 4.2: Identity, authorization, and segregation of duties¶

Cryptographic identity extraction: Approver identity is retrieved exclusively from verified identity provider claims (Microsoft Entra ID oid, with tid), not from editable form fields or card payload properties.
Verify: Tamper with the user ID in an adaptive card submission body and confirm the server rejects the request or overrides it with the authenticated token's oid.
Role-based approval gating: The approver belongs to the authorized security group or role required for the specific action class.
Verify: Attempt approval with a user identity lacking the required approval role and confirm the system blocks the decision with a 403 Forbidden error.
Self-approval prevention: The system blocks approval if the approver identity matches the run initiator identity. Compare initiator and approver by oid (not sub, which differs per application).
Verify: Have the test run initiator attempt to approve their own request across different client applications and confirm the action is blocked with an explicit audit log entry by comparing oid rather than application-specific sub claims.
Multi-party approval verification: Actions exceeding critical risk thresholds (such as high-value transfers) require two distinct approvers.
Verify: Check policy rules for Tier-1 sensitive tools and verify the graph remains paused until the second approval is recorded.

Section 4.3: Path validation and failure modes¶

Edited argument re-validation: Edited parameters pass full schema validation and business boundary checks before execution is scheduled.
Verify: Submit edited values exceeding allowed numerical thresholds (such as amount > max limit) and confirm the action is rejected with a validation error.
Timeout path safety: Expired approval requests terminate safely at END without executing side effects and write a timed_out audit record.
Verify: Trigger the timeout monitor against a test thread and verify no downstream action occurs and status is timed_out.
Rejection path closure: Rejected requests write a rejected audit record, notify the caller, and execute zero external side effects.
Verify: Submit an explicit rejection and verify downstream systems receive no calls.
Idempotent resume handling: Submitting duplicate approval callbacks for an already-resolved thread does not trigger duplicate execution.
Verify: Resend an identical approval payload twice and confirm the second call returns HTTP 409 or is safely ignored.

Section 4.4: Immutability, retention, and access controls¶

WORM storage configuration: Audit records are written to immutable storage (such as Azure Blob Storage with time-based retention or AWS S3 Object Lock in Compliance mode).
Verify: Attempt to delete or overwrite an audit record using administrative credentials and confirm the operation is denied by the storage layer.
Retention policy enforcement: Retention period ${RETENTION_PERIOD} matches the retention policy agreed with legal and compliance.
Verify: Inspect the bucket or container policy and confirm the locked retention duration is configured correctly.
Restricted write permissions: Only the agent backend managed identity or dedicated audit ingestion service has write permissions to the audit sink.
Verify: Review IAM / RBAC assignments on ${AUDIT_TABLE} and verify no interactive user accounts hold write access.
Read-only compliance access: Compliance auditors have dedicated read-only access roles (Log Analytics Reader or Storage Blob Data Reader) isolated from production operator credentials.
Verify: Test auditor credentials and confirm read access is granted while write and delete access is completely denied.

Section 4.5: Sign-off and sample reconciliation¶

End-to-end sample reconciliation: An independent reviewer reconciles a sample of staged test decisions covering every decision type (size agreed with the Security engineer) end-to-end against downstream execution receipts.
Verify: Document the reconciliation sheet attaching the sampled thread IDs, audit entries, and downstream transaction receipts.
Sign-off documentation: GenAI engineer, Security engineer, and Product owner review and sign off the compliance register.
Verify: Attach signed compliance checklist to the release ticket ${RELEASE_TICKET_ID}.
Gaps and decision: Record every gap with an owner and due date, then sign off or block go-live.
Verify: The release ticket shows the decision and the open actions.

Draft audit record schema¶

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "HITLAuditRecord",
  "type": "object",
  "required": [
    "audit_id",
    "thread_id",
    "run_id",
    "action_name",
    "event_type",
    "timestamp",
    "initiator_id"
  ],
  "properties": {
    "audit_id": {
      "type": "string"
    },
    "thread_id": {
      "type": "string"
    },
    "run_id": {
      "type": "string"
    },
    "action_name": {
      "type": "string"
    },
    "event_type": {
      "enum": [
        "requested",
        "approved",
        "edited",
        "rejected",
        "timed_out"
      ]
    },
    "timestamp": {
      "type": "string"
    },
    "initiator_id": {
      "type": "string"
    },
    "approver_id": {
      "type": [
        "string",
        "null"
      ]
    },
    "original_arguments": {
      "type": "object"
    },
    "edited_arguments": {
      "type": [
        "object",
        "null"
      ]
    },
    "idempotency_key": {
      "type": "string"
    },
    "comments": {
      "type": [
        "string",
        "null"
      ]
    }
  },
  "additionalProperties": false
}

Snippets to add:

snippets/checklists/hitl-audit.md: HITL audit checklist sheet for sign-off meetings (markdown)
snippets/schemas/approval-audit.json: Audit record JSON schema for automated compliance validation (json)

5. Known issues and fixes¶

Symptom	Likely cause	Fix
Approver identity shows synthetic or client-controlled string	Backend extracted user identity from adaptive card form field rather than verified auth token	Extract user object ID strictly from the authenticated token claims (`oid` in Microsoft Entra ID)
Argument edits cannot be audited after execution	System overwrote state in-place without logging original versus modified parameters	Record both `original_arguments` and `edited_arguments` in the decision audit record
Timeout events produce no audit record	Unhandled timeout dropped out of queue without reaching an audit logging node	Configure the timeout handler worker to write an explicit `timed_out` record before closing the thread
Self-approval succeeds in staging	Review gateway omitted check or compared application-specific `sub` claims instead of consistent user `oid`	Add mandatory inequality check comparing `oid` (with `tid`) in review gateway and return 403 if IDs match
Audit records deleted during routine data housekeeping	Audit store lacked WORM immutability protection	Enable WORM retention lock (Azure Immutable Blob Storage or AWS S3 Object Lock in Compliance mode)
Sensitive customer data or secrets visible in audit store	Raw tool parameters logged without sanitisation or masking	Pass arguments through a redaction filter removing API keys and PII before writing audit records

6. Monitoring and alerts¶

Signal	Alert when	Action
Sensitive action executed without matching approval record	Greater than 0 in any window	Security on-call halts agent execution and investigates potential bypass immediately
Decision submitted without verified approver identity	Greater than 0 in any window	Block callback processing and inspect authentication headers on incoming requests
Self-approval attempt detected	Greater than 0 in any window	Log security warning, reject transaction, and alert compliance officer
Audit store write or delete attempted outside service identity	Greater than 0 in any window	Security operations investigates unauthorized access attempt on audit sink
Audit ingestion failure rate	Any ingestion failure	Alert DevOps; pause agent processing until audit pipeline recovers to prevent unlogged actions

7. AWS and open-source equivalents¶

Azure: Azure Blob Storage with immutable storage (time-based retention policies locked under WORM compliance), Microsoft Entra ID token claims (oid, tid), Azure Monitor Log Analytics workspace with table-level RBAC and diagnostic logging.
AWS: Amazon S3 Object Lock configured in Compliance mode (guaranteeing WORM compliance even against root user deletion), AWS CloudTrail with log file integrity validation enabled, and Amazon Cognito / IAM identity claims.
Open-source & On-Premises: MinIO with Object Locking and retention provides WORM storage. PostgreSQL audit tables with RLS and revoked UPDATE/DELETE grants give append-only access for applications only; administrators can still alter them, so pair them with a WORM export if the audit must be tamper-proof.

8. Validation record¶

Pending. Planned dry-run scenario: A second reviewer audits a sandbox approval flow seeded with a self-approval and a missing edit record, and must flag both. (The unverified card identity and un-redacted secret may be added as extra seeded faults.)