Justin Napolitano

This README documents how to build, run, and sanity-check each service in the headless finance stack. It assumes you’re running Docker Compose locally and using CSV/OFX email ingestion first, with a clean path to add Teller and other providers.

Prerequisites
Environment (.env)
First-time Initialization
Service Matrix
Build, Run, and Sanity Checks
Suggested Run Order
Troubleshooting
- Teller
- Web & Proxy
Cloud Notes

Prerequisites

Docker & Docker Compose installed and running (rootless Docker is fine).
A .env file at repo root (see below).
psql client for DB checks (optional but useful).
curl (and optionally jq) for API/UI checks.

Host port mappings used here (defaults):

Postgres exposed at localhost:5434 → container 5432

API stays internal (no host port) once reverse proxy is used

Proxy exposed at <HOST_LAN_IP>:3020 → serves UI at / and API at /api

Environment (.env)

Create a .env in the repository root. Example:

# DB
POSTGRES_DB=finance
POSTGRES_USER=fin_writer
POSTGRES_PASSWORD=change_me_now
POSTGRES_READONLY_USER=fin_reader
POSTGRES_READONLY_PASSWORD=read_only_please
POSTGRES_HOST=db
POSTGRES_PORT=5432
TZ=America/New_York

# Scheduler (host specifics — adjust for your machine)
HOST_WORKSPACE=/absolute/path/to/your/repo
DOCKER_SOCK_PATH=/run/user/1000/docker.sock
COMPOSE_PROJECT_NAME=finance

# IMAP (for ingestor-email)
IMAP_HOST=imap.gmail.com
IMAP_USER=finance.imports.yourname@gmail.com
IMAP_PASS=your_app_password_here
IMAP_FOLDER=bank-export
RAW_DIR=/data/raw
BANK_NAME=

# Teller (mTLS + Basic)
TELLER_BASE_URL=https://api.teller.io
TELLER_AUTH_STYLE=basic
TELLER_ENROLLMENT_ID=usr_pjhb69paqhfgdf39js000
TELLER_ACCESS_TOKEN=token_4yobfknid5ho4chpxo2drtcz3q

# Mounted inside containers by docker-compose
TELLER_CERT_PATH=/secrets/teller/certificate.pem
TELLER_KEY_PATH=/secrets/teller/private_key.pem

# enroll/sync window
TELLER_SINCE_DAYS=30

Security hygiene on host:

chmod 0400 ./secrets/teller/private_key.pem
chmod 0444 ./secrets/teller/certificate.pem

First-time Initialization

Start Postgres

docker compose up -d db

Run bootstrap (roles, grants, migrations; safe to re-run)

docker compose up --no-deps db-bootstrap

Verify

PGPASSWORD=$POSTGRES_PASSWORD psql -h localhost -p 5434 -U $POSTGRES_USER -d $POSTGRES_DB -c '\dx'
PGPASSWORD=$POSTGRES_PASSWORD psql -h localhost -p 5434 -U $POSTGRES_USER -d $POSTGRES_DB -c 'table schema_migrations;'

Service Matrix

| Service | Path | Build | Run | Purpose | |--------------------|----------------------|--------------------------------------------|--------------------------------------------------|---------| | db | db/ | n/a (official image) | docker compose up -d db | Postgres with extensions | | db-bootstrap | ops/scripts/ | n/a (official image) | docker compose up --no-deps db-bootstrap | Idempotent roles + migrations | | ingestor-email | ingestor-email/ | docker compose build ingestor-email | docker compose run --rm ingestor-email | Pull email attachments (CSV/OFX/QFX) | | normalizer | normalizer/ | docker compose build normalizer | docker compose run --rm normalizer | Normalize raw files to transactions | | classifier | classifier/ | docker compose build classifier | docker compose run --rm classifier | Apply keyword rules → tx_splits | | budgeter | budgeter/ | docker compose build budgeter | docker compose run --rm budgeter | Import monthly budgets from YAML | | api | api/ | docker compose build api | docker compose up -d api | Read-only HTTP API (internal only when proxied) | | scheduler | scheduler/ | docker compose build scheduler | docker compose up -d scheduler | Cron-like orchestration of jobs | | backup | backup/ | docker compose build backup | docker compose up -d backup | Nightly pg_dump rotation | | teller-enroll | teller-sync/ | docker compose build teller-enroll | docker compose run --rm teller-enroll | One-shot: upsert enrollment, fetch accounts, seed jobs | | teller-sync | teller-sync/ | docker compose build teller-sync | docker compose run --rm teller-sync | Pull balances/transactions for seeded accounts | | finance-web | finance-web/ | docker compose build finance-web | internal only (served via proxy) | Next.js UI (built with API base /api) | | reverse-proxy | ops/reverse-proxy/ | n/a (official image) | docker compose up -d reverse-proxy | Serves UI at / and proxies API at /api |

Build, Run, and Sanity Checks

1) Database (`db`)

docker compose up -d db
docker exec -it finance-db pg_isready -U $POSTGRES_USER -d $POSTGRES_DB
PGPASSWORD=$POSTGRES_PASSWORD psql -h localhost -p 5434 -U $POSTGRES_USER -d $POSTGRES_DB -c '\dx'

2) Bootstrap (`db-bootstrap`)

docker compose up --no-deps db-bootstrap
PGPASSWORD=$POSTGRES_PASSWORD psql -h localhost -p 5434 -U $POSTGRES_USER -d $POSTGRES_DB \
  -c "select * from schema_migrations order by applied_at desc;"

3) Email Ingestor (`ingestor-email`)

docker compose build ingestor-email
docker compose run --rm ingestor-email
PGPASSWORD=$POSTGRES_PASSWORD psql -h localhost -p 5434 -U $POSTGRES_USER -d $POSTGRES_DB \
  -c "table ingest_files order by id desc limit 10;"

4) Normalizer (`normalizer`)

docker compose build normalizer
docker compose run --rm normalizer
PGPASSWORD=$POSTGRES_PASSWORD psql -h localhost -p 5434 -U $POSTGRES_USER -d $POSTGRES_DB \
  -c "select id, posted_at, amount, description from transactions order by id desc limit 20;"

5) Classifier (`classifier`)

docker compose build classifier
docker compose run --rm classifier
PGPASSWORD=$POSTGRES_PASSWORD psql -h localhost -p 5434 -U $POSTGRES_USER -d $POSTGRES_DB \
  -c "select t.id, t.posted_at, t.amount, t.description, c.code
      from tx_splits s
      join transactions t on t.id = s.transaction_id
      join categories c on c.id = s.category_id
      order by t.posted_at desc, t.id desc limit 20;"

6) Budget Importer (`budgeter`)

docker compose build budgeter
docker compose run --rm budgeter
PGPASSWORD=$POSTGRES_PASSWORD psql -h localhost -p 5434 -U $POSTGRES_USER -d $POSTGRES_DB \
  -c "select * from budgets order by category_id, period_start limit 20;"

7) Read-only API (`api`)

Keep API internal when proxied (no host port). Health:

docker compose build api
docker compose up -d api
docker exec -it finance-api wget -qO- http://localhost:8000/healthz

8) Scheduler (`scheduler`)

docker compose build scheduler
docker compose up -d scheduler
docker logs -f finance-scheduler

Manual “ingest → normalize → classify”:

docker exec -it finance-scheduler sh -lc \
  'docker compose -p "$COMPOSE_PROJECT_NAME" --project-directory "$HOST_WORKSPACE" -f "$HOST_WORKSPACE/docker-compose.yaml" run --rm --no-deps ingestor-email && \
   docker compose -p "$COMPOSE_PROJECT_NAME" --project-directory "$HOST_WORKSPACE" -f "$HOST_WORKSPACE/docker-compose.yaml" run --rm --no-deps normalizer && \
   docker compose -p "$COMPOSE_PROJECT_NAME" --project-directory "$HOST_WORKSPACE" -f "$HOST_WORKSPACE/docker-compose.yaml" run --rm --no-deps classifier'

9) Backups (`backup`)

docker compose build backup
docker compose up -d backup
docker exec -it finance-backup sh -lc 'ls -lt /backups | head'

10) Teller Enroll (`teller-enroll`)

Sanity

curl -v https://api.teller.io/accounts \
  --cert ./secrets/teller/certificate.pem \
  --key  ./secrets/teller/private_key.pem \
  -H 'Accept: application/json' \
  -H 'User-Agent: finance-os/0.1 (teller-enroll)' \
  -H "X-Enrollment-Id: $TELLER_ENROLLMENT_ID" \
  -u "$TELLER_ACCESS_TOKEN:"

Run

docker compose build teller-enroll
docker compose run --rm teller-enroll

Verify DB

select count(*) from accounts where external_id like 'teller:%';
select count(*) from teller_jobs;

11) Teller Sync (`teller-sync`)

docker compose build teller-sync
docker compose run --rm teller-sync
PGPASSWORD=$POSTGRES_PASSWORD psql -h localhost -p 5434 -U $POSTGRES_USER -d $POSTGRES_DB \
  -c "select posted_at, amount, description from transactions order by posted_at desc, id desc limit 20;"

12) Web UI (`finance-web`)

Build the UI with API base /api so the browser calls the proxy, not the API directly:

# one-time: ensure .dockerignore keeps .next, node_modules, .git out of context
docker compose build finance-web

Do not publish a host port for finance-web. It will be served through the proxy.

13) Reverse Proxy (`finance-proxy`)

The proxy exposes 3020 on the host, serves the UI at /, and forwards API calls at /api to the internal api service.

Bring it up:

docker compose up -d reverse-proxy

Sanity from any device on your LAN (replace with your host IP if different):

curl -i http://192.168.1.115:3020/healthz          # expect 200 ok
curl -I http://192.168.1.115:3020/                  # UI HTML headers
curl -i http://192.168.1.115:3020/api/accounts      # API JSON, no CORS

Open the app:
http://192.168.1.115:3020/

Suggested Run Order

# First-time
docker compose up -d db
docker compose up --no-deps db-bootstrap

# Teller: enroll accounts, then sync
docker compose run --rm teller-enroll
docker compose run --rm teller-sync

# Email ingestion loop (manual)
docker compose run --rm ingestor-email
docker compose run --rm normalizer
docker compose run --rm classifier

# Budgets
docker compose run --rm budgeter

# API (internal only) + UI + Proxy
docker compose up -d api finance-web reverse-proxy

# Scheduler & backup
docker compose up -d scheduler backup

Troubleshooting

API “port already allocated”
If you previously published 8010 and now use the proxy, remove the API host port mapping. Keep API internal.
Classifier can’t find rules
Ensure config/rules.yaml exists and is mounted to /app/config/rules.yaml.
Normalizer ON CONFLICT error
Ensure migrations add:
- unique (name) on institutions
- unique (institution_id, mask) where mask is not null on accounts
Scheduler “permission denied /workspace”
Set a real absolute HOST_WORKSPACE path and pass --project-directory "$HOST_WORKSPACE".
Scheduler “cannot connect to Docker daemon”
Mount the correct rootless Docker socket: DOCKER_SOCK_PATH=/run/user/<uid>/docker.sock.

psql “role root does not exist”
Use credentials explicitly:

PGPASSWORD=$POSTGRES_PASSWORD psql -h localhost -p 5434 -U $POSTGRES_USER -d $POSTGRES_DB -c 'select 1'

Troubleshooting (Teller)

401 Unauthorized
.env must have TELLER_AUTH_STYLE=basic, TELLER_ACCESS_TOKEN set, and you must send X-Enrollment-Id. Re-run teller-enroll.
403 Forbidden
Environment mismatch. Ensure TELLER_BASE_URL matches the token’s environment.

TLS errors
Key/cert mismatch or encrypted key. Verify modulus:

openssl x509 -noout -modulus -in ./secrets/teller/certificate.pem
openssl rsa  -noout -modulus -in ./secrets/teller/private_key.pem

Values must match.

No jobs seeded
Confirm /accounts returns data and the teller_jobs uniqueness constraint exists.

Troubleshooting (Web & Proxy)

CORS errors in browser
You bypass CORS by using the proxy. Ensure the UI was built with NEXT_PUBLIC_API_BASE=/api. Rebuild finance-web if you changed it.

Proxy 502/404
Proxy can’t reach backends. Check service names/ports in the proxy config. From proxy container:

docker exec -it finance-proxy sh -lc 'wget -qO- http://finance-web:3020/healthz && echo && wget -qO- http://finance-api:8000/healthz'

LAN access blocked
If Ubuntu ufw is enabled:

sudo ufw allow from 192.168.1.0/24 to any port 3020 proto tcp

Frontend build failures
Components using usePathname, useRouter, useSearchParams must be Client Components. Add 'use client'; at the top of those files.

Cloud Notes

DB: Managed Postgres (private IP). Run db-bootstrap as a job for grants + migrations.
Runtime: Build images to a registry. Deploy api, web, and proxy. Use a scheduler for ingestion, normalization, rules, and budgets.
Secrets: Move env secrets to a secrets manager and mount/inject at runtime.
Backups: Dump to object storage with lifecycle policies (30–90 days).

Building and Running the Jayburd Accounting Stack

Contents

Prerequisites

Environment (.env)

First-time Initialization

Service Matrix

Build, Run, and Sanity Checks

1) Database (`db`)

2) Bootstrap (`db-bootstrap`)

3) Email Ingestor (`ingestor-email`)

4) Normalizer (`normalizer`)

5) Classifier (`classifier`)

6) Budget Importer (`budgeter`)

7) Read-only API (`api`)

8) Scheduler (`scheduler`)

9) Backups (`backup`)

10) Teller Enroll (`teller-enroll`)

11) Teller Sync (`teller-sync`)

12) Web UI (`finance-web`)

13) Reverse Proxy (`finance-proxy`)

Suggested Run Order

Troubleshooting

Troubleshooting (Teller)

Troubleshooting (Web & Proxy)

Cloud Notes

Contents

Prerequisites

Environment (.env)

First-time Initialization

Service Matrix

Build, Run, and Sanity Checks

1) Database (db)

2) Bootstrap (db-bootstrap)

3) Email Ingestor (ingestor-email)

4) Normalizer (normalizer)

5) Classifier (classifier)

6) Budget Importer (budgeter)

7) Read-only API (api)

8) Scheduler (scheduler)

9) Backups (backup)

10) Teller Enroll (teller-enroll)

11) Teller Sync (teller-sync)

12) Web UI (finance-web)

13) Reverse Proxy (finance-proxy)

Suggested Run Order

Troubleshooting

Troubleshooting (Teller)

Troubleshooting (Web & Proxy)

Cloud Notes

1) Database (`db`)

2) Bootstrap (`db-bootstrap`)

3) Email Ingestor (`ingestor-email`)

4) Normalizer (`normalizer`)

5) Classifier (`classifier`)

6) Budget Importer (`budgeter`)

7) Read-only API (`api`)

8) Scheduler (`scheduler`)

9) Backups (`backup`)

10) Teller Enroll (`teller-enroll`)

11) Teller Sync (`teller-sync`)

12) Web UI (`finance-web`)

13) Reverse Proxy (`finance-proxy`)