CONTINUE — 2026-06-24 — National fares/tolls/parking + GTFS Golden-Source ETL

Host: geocoder (RTX 5080, 172.26.1.95). DB: commute @ 172.26.1.152:5433 (krish, trust-auth). Live: https://multimodal.certihomes.com (GPU map, geo2) · https://commutingcost.com (web geo3 pm2 commutingcost-web:3035 + API geo3 commutingcost-api.service:8096). Repos (all pushed): tlcengine/gpu-pilot @ 1e562cb (via geo bundle), commutingcost-api @ 7daaf23, commutingcost-web @ 8802809 (geo3 direct). Read first: memory gtfs-fare-ingestion, toll-coverage-audit, national-parking-costs, commutingcost-api-service, outbound-isochrone-direction, then this.

DONE this session (6/23–6/24)

A. commutingcost.com cost model fixed + deployed (api 7448dfc/7daaf23): DB-correct Hudson tolls $16.79pk/$14.79off (was bogus $15 OSM-gantry sum), CBD $35 garage + $9 CRZ on drives, Dunellen→zone6 $10.75, peak/off via is_peak; national downtown parking via get_cbd_parking (nearest of 1,515 garages); station-to-station OD fares via get_od_fare.

B. commutingcost.com homepage redesign (web 8802809): 2-col — from/to + 9 mode pills LEFT, "Map & Routes" (maplibre RouteMap + results) RIGHT; per-route cost breakdown so Parking shows for every mode; added P&R/K&R/Uber&Ride/Bike&Ride pills.

C. National DATA layer (loaded into commute DB): - toll_rates 194 facilities / 40 states (was 139/36): NTTA priced, Austin/OK/FL/ CA/CO/GA/WA gaps, RI Pell + KY RiverLink (new states), 6 Canada bridges; stale fixes (Confederation $20, ON 412/418 $0, DE-Newark $4→$5). - gtfs_fares 44,004 rows / 522 feeds / 36,105 OD pairs (was 6/7,057). gtfs_stops 161,554 zoned stops/175 feeds. gtfs_transfers 19,107. gtfs_stop_areas ~16k. pdf_fares 521/10 agencies (VRE 360-pair OD, MARC, Tri-Rail...). cbd_parking 1,515 garages/49 metros + cbd_parking_metro 51. - Built via 8 background agents (toll audit+fill, parking, GTFS sweep ×national+4 regional, PDF) + the ingesters.

D. Ingesters + golden-source (gpu-pilot/gtfs-arrow/fares/, all committed): ingest_gtfs_fares.py (v1, whitespace-fixed), ingest_gtfs_fares_v2.py (v2), ingest_gtfs_transfers.py, ingest_gtfs_stops.py; validate_fix_gtfs.py gained fix_fares+fix_transfers (deployed to canonical gtfs3:/home/krish/, .bak.20260623).

E. ⭐ UNIFIED GTFS GOLDEN-SOURCE ETL (gpu-pilot/gtfs-arrow/etl/, commit 1e562cb): gtfs_golden_etl.py = ONE pipeline per US+CA Mobility-DB feed: download → golden-fix (validate_fix_gtfs) → write cleaned golden zip (gtfs-arrow/golden/, gitignored) → ingest fares(v1+v2)+transfers+stops → idempotent per-feed UPSERT. run_monthly_ etl.sh + cron 15 3 1 * * on geocoder. Tested 12 feeds end-to-end.

F. OD fare wiring LIVE (api): transit/P&R/K&R snap boarding+dest to nearest gtfs_stops in the SAME feed (origin ≤~5km, dest ≤~30km), read origin_zone→dest_zone price; falls back to zone/agency fare. Validated BART $7.15, Caltrain $10.75, GO Union→Oakville $9.70, NJ→Penn $10.75 (no regression).

PENDING (precise, act-cold)

• v2 OD resolver — get_od_fare is v1 only (uses gtfs_stops.zone_id). MBTA/SEPTA are GTFS-Fares v2: origin_id/destination_id are area_ids → resolve via gtfs_stop_areas (stop→area, 1:many), not gtfs_stops.zone_id. Add a v2 branch to get_od_fare.
• GPU-engine fare resolver — the engine still uses static fares.py rail_fare. To use gtfs_fares OD it needs a KDTree over gtfs_stops (the api's 161k-row scan is fine per API call, NOT for the GPU per-cell hot loop). Export gtfs_stops→JSON + KDTree like cbd_parking.
• Sync golden zips to gtfs3 — ETL writes cleaned zips to gpu-pilot/gtfs-arrow/golden/; should rsync to gtfs3:/mnt/drive1/otp/.../gtfs-fixed-v3 (the canonical OTP golden store).
• ~200 no-fare agencies (GTFS ships no fares): CTA/MARTA/WMATA/RTD/UTA/Valley Metro + Canada TTC/STM/TransLink/OC Transpo/Calgary — PDF/agency-page backfill (extend the PDF agent).
• WMATA + AC Transit/SamTrans need a Mobility-DB / 511.org API key.
• #14 DEFERRED per-region GPU hex precompute — now UNBLOCKED (fares+tolls+parking in place). Per metro/CBSA: H3 grid + regional feed union + cross-feed transfers + walk/bike SSSP.

DECISIONS (what the user chose + why)

• Golden ETL = the maintained pipeline: fix + ingest fares/transfers/stops monthly so the cleaned GTFS and the data tables stay current (user: "make these all part of GTFS ETL to Golden Source that fixes all GTFS files and fares and other data to be complete").
• OD fares wired FIRST (before the all-feed sweep) so the loaded data goes live; then 4 parallel regional agents swept the full ~886-feed registry.
• Distance caps on OD snap are REQUIRED — without the origin ≤5km cap a NJ lot snapped to a CA stop (only BART/Caltrain were loaded) → wrong $4 fare. Fallback preserves NJT/LIRR/MNR.
• Parking = SpotHero pre-booked rates (~15-20% below posted drive-up) — right for a "commuter who reserves"; NYC kept at modeled $35.
• Per-region GPU precompute, not one national grid (VRAM); deferred after fares+tolls.

GOTCHAS (don't relearn)

• Ingester whitespace bug (fixed): space-padded headers (Metra " origin_id") silently collapsed OD→flat; all 3 ingesters now .strip() headers+values.
• Idempotent upsert = DELETE WHERE feed IN (batch) then \copy. Re-running keeps totals stable (mdb-NNNN feeds from agents = same data; ETL refreshes them).
• Golden ETL imports the fixer with psycopg2 STUBBED (sys.modules['psycopg2']=...) — the fixer's correction-logging DB is optional for the ETL; DB writes go via psql \copy.
• Agent feed slugs are mdb-NNNN; some dup the 50 friendly feeds (bart/caltrain) = newer snapshots, harmless for the resolver (snaps to nearest stop in either feed).
• GitHub push routes via geo (geocoder has no key): bundle ad516ca..main → geo → git fetch bundle main:refs/temp/x; git push origin refs/temp/x:main; update-ref -d. geo3 repos (api/web) push DIRECT (HTTPS creds present). secret-scan staged diff first.
• commutingcost-api restart is fast (~2s healthy); engine (hexmap-api) ~9s; purge hexmap cache + geo2 adapter cache after cost-logic changes.
• ruff --fix strips an import added BEFORE its first use (broke is_peak once) — add import with the usage. sudo pw /home/krish/secert.txt.