PrototypeAgents

AI-Powered Triage System

Claude · XML-Tagged Structured Output · Audit Trail

100%of triage decisions logged with reference IDs for audit

Streamlit prototype demonstrating AI-powered case triage. Claude processes unstructured case descriptions and returns structured interpretations + priority decisions using XML-tagged output. All decisions are logged with reference IDs and timestamps for audit trails.

Engagement

Status
Prototype / Validation
Primary KPI
0 parsing failures in tested sample set
Starting point
Brittle JSON parsing with occasional preamble failures
Scope
Triage workflow simulation
My role
AI engineer (prompt + parser + audit logging design)

01 Overview

From brittle JSON parsing with occasional preamble failures to 0 parsing failures in tested sample set.

Before

Brittle JSON parsing with occasional preamble failures

After

0 parsing failures in tested sample set

How it works

The triage system uses Anthropic's Claude API to process unstructured case descriptions and return structured decisions. Claude is prompted to produce XML-tagged output separating interpretation (<interpretation>) from priority decision (<triage>), enabling reliable parsing without brittle regex. Each submission generates an MD5-based reference ID from the case content, making decisions idempotent — submitting the same case twice returns the same reference ID for deduplication. All decisions are appended to a CSV-backed audit log with timestamps, and prompt modules are stored as separate Python files for easy domain adaptation.

What I built

  • XML-tagged output parsing (<interpretation>, <triage>) for reliable extraction
  • Reference ID generation and CSV-backed audit trail for all decisions
  • Multi-turn session state management with Streamlit session_state
  • Configurable prompts via external prompt modules for easy adaptation

02 The system

Architecture at a glance.

FIG. 09 — SYSTEM ARCHITECTURECase textunstructuredClaudeXML-tagged promptXML parserdeterministicReference IDMD5 · timestampStreamlittriage decisionAudit logCSV
Claude returns XML-tagged fields that parse deterministically; every decision gets a reference ID and lands in an append-only audit log.
  1. 01

    Claude API Integration

    Sends case descriptions to Anthropic Claude with a structured prompt requesting XML-tagged output for reliable parsing.

  2. 02

    XML Output Parser

    Extracts <interpretation> and <triage> blocks from Claude's response for clean structured data without fragile JSON parsing.

  3. 03

    MD5 Reference ID Generator

    Hashes case content to produce a deterministic reference ID — identical cases produce identical IDs for deduplication.

  4. 04

    CSV Audit Log

    Appends every decision with reference ID, timestamp, case text, interpretation, and triage result for compliance review.

  5. 05

    Configurable Prompt Modules

    Triage criteria and domain context stored in external Python prompt modules — editable by domain experts without code deploys.

  6. 06

    Streamlit Session Management

    Multi-turn session state tracks submission history within a session, supporting case review and comparison.

Stack

Models
Anthropic Claude
Frontend
Streamlit
Backend
Python
Data
Pandas

03 Engineering decisions

The decision that made it production-grade.

Each decision opens with the business problem, contrasts the common approach with what I shipped, then walks through the real code — the highlighted lines follow the step you are reading.

  1. 01Robust Structured Parsing

Decision 01 · LLM Output · Structured Parsing

Robust Structured Parsing

In plain termsThis ensures the AI always returns decisions in a structured, machine-readable format — using strict parsing with fallback recovery so even imperfectly formatted AI outputs can be reliably processed downstream.

The problem

JSON-mode fails when the model adds a reasoning preamble — XML tags degrade gracefully.

When Claude or GPT produces explanatory text before the opening brace of a JSON response, json.loads() raises immediately. XML-tagged output with regex extraction finds the payload regardless of preamble text — and the audit trail requires a deterministic reference ID without a sequence counter.

Before · Common approach

JSON schema in the prompt, json.loads() to parse. Breaks when the model adds preamble reasoning.

✗ ~5% of zero-shot calls add preamble text. No reference ID. No audit trail.

After · This implementation

XML-tagged output (<interpretation>, <triage>), regex extractor, MD5-based reference ID, CSV audit log.

✓ Preamble-resilient parsing. Idempotent reference IDs. Every decision logged with timestamp.

How it worksXML-Tagged LLM Output Pipeline
  1. 01Prompt: wrap fields in <interpretation> + <triage> tagsTags survive reasoning preamble before the opening brace
  2. 02LLM returns free-form text with embedded tagsModel may add reasoning before tags — that is fine
  3. 03re.search() extracts tag contents with DOTALLMulti-line safe; returns None on no match (no throw)
  4. 04MD5 hash of case text → 8-digit reference IDDeterministic: same case always returns same ID

✓ JSON parse failures → 0 · fully idempotent · audit-ready

Code walkthrough — 3 steps · scroll or click

triage-demo/triage_classifier.py
"""
AI triage classifier — structured output via XML-tagged Claude responses.

WHAT IT DEMONSTRATES:
  How to use Claude's free-form generation to produce reliably parseable
  structured output using XML tags. The model wraps each output field in
  named tags (<interpretation>, <triage>), which are extracted with regex.
  All decisions are logged with reference IDs for audit trails.

WHY IT'S INTERESTING:
  Before JSON-mode / tool use were ubiquitous, XML-tagged output was a
  robust way to extract structured data from LLMs — it's resilient to
  preamble text and works across models. The reference ID derived from
  hashing the input ensures idempotent lookups without a database.

NOVELTY:
  XML-tagged output parsing remains relevant even with modern JSON-mode
  because it degrades more gracefully: if the model produces extra reasoning
  text before the tags (common in zero-shot prompts), the regex extractor
  still finds the payload, whereas JSON-mode fails on any non-JSON prefix.
  The MD5-based reference ID (hashed from case description + timestamp) gives
  idempotent audit trail entries without requiring a sequence counter or
  database auto-increment — critical for a stateless prototype that writes
  only to a flat CSV. The design explicitly separates interpretation (what is
  this case?) from triage (what priority?) into distinct XML tags, matching
  the two-question mental model used by human triage reviewers and making it
  straightforward to evaluate each output dimension independently.
"""

from __future__ import annotations

import hashlib
import re
from dataclasses import dataclass
from datetime import date
from typing import Optional

import anthropic
import pandas as pd


@dataclass
class TriageResult:
    reference_id: str
    date: str
    raw_input: str
    interpretation: str
    triage_level: str


class TriageClassifier:
    """
    Classifies unstructured case descriptions into priority triage levels.

    Uses Claude with an XML-tagged output schema so the response can be
    parsed reliably even when the model includes explanatory preamble.

    Output tags:
      <interpretation> — structured summary of the case
      <triage>         — priority label (e.g. P1 Critical / P2 High / P3 Normal)
    """

    SYSTEM_PROMPT = """
You are a triage classification assistant. Given a case description, you must:
1. Interpret the case in a clear, structured summary.
2. Assign a triage priority level.

Always respond using these exact XML tags:
<interpretation>
[Concise structured interpretation of the case]
</interpretation>
<triage>
[Priority level: P1 Critical | P2 High | P3 Normal | P4 Low]
</triage>

Base your decision on urgency, impact, and risk factors described in the input.
"""

    def __init__(self, api_key: str, model: str = "claude-3-5-sonnet-20241022"):
        self.client = anthropic.Anthropic(api_key=api_key)
        self.model = model

    def classify(self, case_description: str) -> TriageResult:
        """Send the case to Claude and parse the XML-tagged response."""
        response = self.client.messages.create(
            model=self.model,
            max_tokens=512,
            system=self.SYSTEM_PROMPT,
            messages=[{"role": "user", "content": case_description}],
        )
        raw_output = response.content[0].text
        interpretation, triage_level = self._parse_xml_output(raw_output)

        return TriageResult(
            reference_id=self._generate_ref_id(case_description),
            date=date.today().isoformat(),
            raw_input=case_description,
            interpretation=interpretation or "[parse error]",
            triage_level=triage_level or "[parse error]",
        )

    # ── Parsing ────────────────────────────────────────────────────────────

    def _parse_xml_output(self, text: str) -> tuple[Optional[str], Optional[str]]:
        """Extract content from <interpretation> and <triage> tags."""
        interp = self._extract_tag(text, "interpretation")
        triage = self._extract_tag(text, "triage")
        return interp, triage

    @staticmethod
    def _extract_tag(text: str, tag: str) -> Optional[str]:
        match = re.search(
            rf"<{tag}>(.*?)</{tag}>", text, re.DOTALL | re.IGNORECASE
        )
        return match.group(1).strip() if match else None

    # ── Audit trail ────────────────────────────────────────────────────────

    @staticmethod
    def _generate_ref_id(text: str) -> str:
        """Deterministic 8-digit reference ID from input hash."""
        return str(int(hashlib.md5(text.encode()).hexdigest(), 16) % 10 ** 8).zfill(8)

    def append_to_log(self, result: TriageResult, log_path: str) -> pd.DataFrame:
        """Persist the triage result to a CSV audit log."""
        row = {
            "Reference ID": result.reference_id,
            "Date": result.date,
            "Interpretation": result.interpretation,
            "Triage Level": result.triage_level,
        }
        try:
            df = pd.read_csv(log_path, dtype=str)
        except FileNotFoundError:
            df = pd.DataFrame(columns=list(row.keys()))

        df = pd.concat([df, pd.DataFrame([row])], ignore_index=True)
        df.to_csv(log_path, index=False)
        return df
import anthropic, json

def classify(self, case_description):
    response = self.client.messages.create(
        model=self.model, max_tokens=512,
        messages=[{
            "role": "user",
            "content": f"Classify: {case_description}"
        }],
    )
    # brittle: json.loads fails on preamble text
    result = json.loads(response.content[0].text)
    return result   # no reference ID, no audit trail

The Impact

0parse failuresXML regex finds payload regardless of preamble text
8digit reference IDDeterministic MD5 hash — same case always gets the same ID
100%decisions loggedEvery triage result appended to CSV with timestamp

XML-tagged output remains relevant even with modern JSON-mode because it degrades gracefully on preamble text. The MD5 reference ID produces idempotent audit entries without a database counter — submitting the same case twice returns the same ID for deduplication.

04 Production hardening

What breaks, why, and what keeps it fixed.

The failure modes this system had to survive — each with its root cause, the fix that shipped, and the guardrail that stops it coming back.

  1. 01
    Failure modeFailure mode · Robust Structured Parsing

    ~5% of zero-shot calls add preamble text. No reference ID. No audit trail.

    Root cause

    JSON-mode fails when the model adds a reasoning preamble — XML tags degrade gracefully.

    Fix shipped

    XML-tagged output (<interpretation>, <triage>), regex extractor, MD5-based reference ID, CSV audit log.

    Guardrail

    JSON parse failures → 0 · fully idempotent · audit-ready

05 Outcomes

What changed for the business.

  1. 01

    Standardized triage interpretation with full decision logging

  2. 02

    Reduced parsing brittleness in structured outputs

  3. 03

    Prototype validation for compliance-first routing workflows.

Key benefits

  • Standardised triage decisions eliminate inter-reviewer variability across case types
  • Full audit trail with reference IDs supports compliance and retrospective quality review
  • 24/7 availability removes human reviewer bottlenecks for initial case routing
  • Idempotent reference IDs prevent duplicate decisions on resubmitted cases
  • Prompt module architecture enables rapid domain adaptation without engineering involvement

Metrics are from project-specific observation windows. Codebases are private; excerpts are curated for illustration.

Want the full story behind AI-Powered Triage System?

I’m happy to walk through the architecture, trade-offs and numbers in an interview.