work / AI / ai expense review agent

Rules for the clear cases. AI only for judgment calls.

Reviewing every transaction by hand is slow, and a pure AI reviewer is hard to trust. I designed a hybrid: 13 coded rules handle the clear cases, and AI answers only two yes/no judgment questions.

role
Sole automation engineer
client
Canadian accounting firm
status
Validated prototype

Languages

JavaScript

Tools & platforms

n8nQuickBooks Online APIGoogle Sheets

AI & LLM

in the product
Gemini API (yes/no judgments)
to build it
ClaudeChatGPTPerplexity

Skills

LLM integrationRule designData validationAuditabilityTesting on real data
15

checks: 13 coded rules + 2 AI judgments

1,016

real transactions tested

~7.7%

flagged for human review

01// how I approached it

The thinking behind it

  1. 01 · What I noticed

    Accountants need to explain every flag, so a black-box AI reviewer would never be trusted.

  2. 02 · What I decided

    Use coded rules for everything they can catch, and ask AI only narrow yes/no questions that are logged.

  3. 03 · What I delivered

    A review list a human can trust, validated on 1,016 real transactions.

02// old vs new

Old system vs new system

Old systempeople as the glue
  1. Export transactions
  2. Review every line by hand
  3. Note anything odd
  4. Ask the client
  5. Repeat next period
New systemthe system does the work
  1. Pull the period's transactions
  2. 13 coded rules run
  3. AI answers 2 yes/no checks
  4. Human reviews flagged items only
03// how it works

Watch it run

Click through the four steps. In step 3, click any box to see what it does.

synthetic data · no client screens
Reconstruction with synthetic data · no client screens
04// the hard part

Making AI auditable.

An accountant has to explain every flag. So AI never decides alone: deterministic rules handle everything they can, and AI answers narrow yes/no questions that are logged per transaction. Testing on real data caught a date-handling bug that would have flagged 58% of transactions before it ever reached a reviewer.

05// key decisions

Why it's built this way

decision 1

Deterministic first

Anything a rule can catch is caught by a rule: predictable and explainable.

Trade-off: more rules to write and maintain.
decision 2

AI as yes/no, not free text

Narrow questions are easy to log, check and compare.

Trade-off: AI covers fewer cases on purpose.
decision 3

Validate on real volume

Tested on 1,016 real transactions before anyone relied on it.

Trade-off: slower path to "done".