InvoiceGuard MY - evaluation dashboard

AI-assisted invoice intake and MyInvois validation for Malaysian SMEs - prototype

Invoices received per month

Arrival by day of month (last 5 days highlighted)

Invoice value, RM (log-spaced bins)

Injected error types (ground truth)

Document quality mix

Supplier concentration (cumulative share of invoices)

What the EDA says

    System comparison on the held-out test split

    Wrong auto-approve = auto-approved although the document has an injected error, is high-value (policy: human review), or a critical field was extracted wrongly. Target: ~0. Scan/photo degradation severity is a synthetic design choice, so read the digital-vs-scan gap as direction, not magnitude. STP on error-free docs = share of clean, below-threshold documents approved with no human touch.

    Headline rates

    Wrong auto-approvals (count, lower is better)

    Critical-field exact match by document quality

    Field accuracy: OCR+regex vs LLM stand-in

    Injected-error handling

    A+ cannot fire line-based rules (V01, V03, V06) because regex extraction returns no line items: structured line extraction is where the AI adds value. C's near-perfect catch rate is partly by construction of the simulator; expect lower with a real extractor.

    Human review queue (System C, lowest confidence first)

    Invoice inspector: extracted JSON → rules → decision

    Extracted JSON (editable: change a value, then re-run)

    Printed on the document (ground truth) (amber = differs from the JSON on the left)

    The rules run in your browser and are verified against the Python engine on every invoice (see tests). System C's extraction is a simulated stand-in; A+ is a real OCR measurement. The TIN registry is a mock.

    Inputs (assumptions unless noted)

    Result

    Model counts direct handling labour only (no systems, approvals, or overhead), so it is deliberately conservative versus published fully-loaded benchmarks. Penalty avoidance (RM200-RM20,000 per offence [VERIFY]) is excluded from the base case.