work / reporting / real-time finance data pipeline

Finance numbers that are never out of date.

Financial reports were assembled by hand and always a little stale. I built a pipeline that moves accounting data into a data warehouse and live dashboards, close to real time.

role
Developer
client
US multi-entity services group
status
Live in production

Languages

SQLJavaScript

Tools & platforms

n8nQuickBooks Online APIBigQueryLooker StudioGoogle Cloud

AI & LLM

to build it
ClaudeChatGPTPerplexity

Skills

Data pipelinesData warehousingDashboard designScheduling
01// how I approached it

The thinking behind it

  1. 01 · What I noticed

    People didn't need prettier reports. They needed numbers they didn't have to ask for.

  2. 02 · What I decided

    Move the data automatically into one warehouse and point every dashboard at it.

  3. 03 · What I delivered

    Finance numbers the whole company can open, synced every minute.

02// old vs new

Old system vs new system

Old systempeople as the glue
  1. Export from accounting
  2. Clean it in spreadsheets
  3. Build the report
  4. Email it around
New systemthe system does the work
  1. Sync runs every minute
  2. Data lands in the warehouse
  3. Dashboards update live
03// how it works

Watch it run

Click through the four steps. In step 3, click any box to see what it does.

synthetic data · no client screens
Reconstruction with synthetic data · no client screens
04// the hard part

Fresh numbers without anyone pressing refresh.

Reports go stale the moment someone has to export them. The pipeline syncs accounting data every minute into one warehouse, so every dashboard reads the same, current numbers.

05// key decisions

Why it's built this way

decision 1

One warehouse as the source of truth

Every dashboard reads from BigQuery instead of its own export.

Trade-off: one more system to run.
decision 2

Sync every minute

Close enough to real time for finance, cheap enough to run all day.

Trade-off: more API calls than a nightly job.
decision 3

Dashboards stay separate from the data

Charts can change without touching the pipeline.

Trade-off: two layers to maintain.
06// lessons

What made it work

  1. 01

    Fix the data, not the report

    Most reporting pain comes from how data moves, not how it looks.

  2. 02

    One source of truth ends arguments

    When everyone reads the same numbers, meetings move faster.

  3. 03

    Automate the boring refresh

    If people refresh it by hand, it will be out of date.