← gpu-pilot · json · source: CONTINUE_7_8_2026.md
▶ CONTINUE 7/8 (Golden-source short-calendar fix · NYC↔Philly whole-corridor nec priority-3 · Zillow pitch + light deck · water masks · res-10 · Kenosha/Gary/Vancouver-WA — newest handoff)
TL;DR
- GOLDEN SHORT-CALENDAR FIX (9062c87): agency snapshots publish trips whose service is ONLY in calendar_dates.txt over a few-month window (NJT 20260513-20261108), with the real service_ids absent from calendar.txt → engine's load-time filter_service_date saw ZERO service on 20260415 = THE real Edison→Philly cause (NOT isolated platform / picker). validate_fix_gtfs.extend_orphan_service_ids now INFERS each orphan's weekly pattern (weekday/Sat/Sun) from its calendar_dates dates and injects a calendar.txt row spanning 2020-2050 (was blanket Mon-Sun = over-serve). refix_orphan_calendars.py re-fixed all 143 golden feeds in-place (0 orphans); refix_regions_runner.sh transit-only rebuilt the 35 affected regions (walk matrix kept). Edison→Philly 80min / NYC 62min on the standard 20260415.
- NYC↔PHILLY WHOLE CORRIDOR (637bffe, Krishna's call): nec is the only region with the full corridor feed set (NYC subway + NJT + SEPTA + Amtrak + Metro-North), so priority raised 1→3 to outrank pinned NYC (2) + midatlantic (1) → it serves the ENTIRE corridor. Validated both ways: Manhattan→Philly 99, Newark→Philly 69, Metropark→Philly 58, Philly→NYC 96 (all were NOT-REACHED); NYC-internal subway intact (Manhattan→Brooklyn 36). TRADEOFF (accepted): NYC now uses nec's flat regional fare, not pinned's subway/PATH/LIRR logic; 08854 flagship routes through nec. Follow-up #77 = give nec NYC-zone fares. Reversible (priority→1).
- WATER MASKS (d772ac9): static per-region OSRM-snap open-water mask (scripts/build_water_masks.py → water_mask.parquet) dropped in _build_nontransit_cells so drive-family never paints bay/ocean/lake — fixes the res-8 coarsening leak the per-request filter missed. 54 regions / 681K water cells. GOTCHA: _is_canada was lon-only → Portland mis-routed to CA OSRM (100% water); now bounded lat+lon boxes.
- res-10 (9eda0ea): engine _refine_to_res10 on the OUTBOUND transit path splits Manhattan-core res-9 cells into res-10 children (walk_matrix_res10_core_v2.parquet); refine='off'|'auto' gated (default off = byte-identical). Validated: 4,298 res-10 children, intra-parent spread 3.8min. Adapter still sends off — wiring auto = follow-up.
- PITCH + DECK: gtm/pitch/ZILLOW_PITCH.md (workflow: 3 researchers→3 angles→3 judges). LIGHT Certihomes deck+one-pager (blue #1E5AA8 on white, CERTIHOMES black, no red/black; gtm/pitch/build_deck_light.py) LIVE at commutingcost.com/partners/certihomes-commute-data-deck.pdf. /map auto-paints on load + ?lat&lon deep links; 51 metro SEO pages link to it.
- GOTCHAS: stops.parquet drops feed prefixes → stop-id COLLISIONS across a multi-feed union (bare 42854 = 2 different stops); walk matrix keys by PREFIXED id via encoding_dict — never match by bare id. filter_service_date bakes the date at LOAD not query. Pause the precompute during region builds (GPU+kernel-lock contention 3×'d chicago + caused validation 429s). Transit-only rebuild (--skip-osm) needs rm transit/+transfers/+manifest first. Metra was NEVER missing (golden mdb-2854; fixer rewrites agency.txt to the mdb code). gpu-pilot main=637bffe.
CONTINUE — 2026-07-08 — Golden-source short-calendar fix · NYC↔Philly whole-corridor (nec priority 3) · Zillow pitch + light deck · water masks · res-10 · Kenosha/Gary/Vancouver-WA
Read first: memory golden-short-calendar-fix, regional-seam-coverage-fixes, res10-dense-core-refine, national-multimodal-engine. Prior handoff: CONTINUE_7_7_2026.md.
Repos: gpu-pilot main = 637bffe (pushed via geo bundle). commutingcost-web/hexmap = geo3 (separate repos, pushed). Front-end map = geo2 tlccommute.
Engine: geocoder :8098 (systemd hexmap-api, uvicorn NO --reload → explicit sudo systemctl restart, sudo pw ~/secert.txt). Registry now includes nec (Northeast Corridor).
DONE this session (large)
- Static per-region OSRM water masks (
d772ac9) —scripts/build_water_masks.py→output/networks/<slug>/water_mask.parquet(+output/nyc_tristate_water_mask.parquet); engine_load_water_bool+_build_nontransit_cellsdrop open-water cells so drive/uber/pnr/knr/ubr never paint bay/ocean/lake. Fixed the res-8 coarsening leak. 54 regions, 681K water cells. GOTCHA fixed:_is_canadawas lon-only → Portland mis-routed to CA OSRM (100% "water"); now bounded (lat,lon) boxes. - Québec City SEO — added to
content/build_metros_seo.py; regenerated (/usr/bin/python3, psycopg2 not in venv); live at commutingcost.com/commute-cost/quebec-city (geo3 web1277072). - res-10 dense-core refine (
9eda0ea) — engine_refine_to_res10on the OUTBOUND transit path (splits Manhattan-core res-9 cells into res-10 children viaoutput/walk_matrix_res10_core_v2.parquet);refine="off"|"auto"through app + cache key; default off = byte-identical. VALIDATED: refine=auto = 4,298 res-10 children, intra-parent spread 3.8 min. (Adapter still sends refine="off"; wiring it on = follow-up.) - Chicago→Gary / Portland→Vancouver-WA / Chicago→Kenosha — all the SAME state-line OSM bug.
_geofabrik_url_for: chicago→[il,in,wi], portland→[or,wa]. Metra was NEVER missing (it's goldenmdb-2854; the fixer rewrites agency.txt to the mdb code, which fooled the search) — refreshed from schedules.metrarail.com. VALIDATED: Gary 80-88, Vancouver-WA 51, Kenosha 122 (Metra UP-N). - Zillow / portals B2B pitch (
4024cca) + light Certihomes deck (4a5909f) —gtm/pitch/ZILLOW_PITCH.md(workflow: 3 researchers→3 angles→3 judges, winner "the commute filter that shows dollars"). LIGHT blue/white deck + one-pager (gtm/pitch/build_deck_light.py, CERTIHOMES wordmark black, NO red/black) LIVE: commutingcost.com/partners/certihomes-commute-data-deck.pdf + /commutingcost-data-onepager.pdf. Board-published. - /map hexmap on commutingcost.com — auto-paints the GPU choropleth on load +
?lat&lon&zdeep links; all 51 metro SEO pages link to it (geo3 hexmape6da688, webfd199e0). - nec Northeast Corridor region (
306dca9,637bffe) — Philadelphia–NYC–New Haven, geofabrik [pa,nj,ny,ct,de], 57 feeds / 76,675 platforms. THE Edison→Philly fix. - ★ GOLDEN-SOURCE SHORT-CALENDAR FIX (
9062c87,6df3432) — the real Edison→Philly root cause. Agency snapshots (NJT NEC mdb-509 + 142 more) define trips' service ONLY incalendar_dates.txtover a short window (NJT: 20260513-20261108), with the real service_ids ABSENT from calendar.txt → ZERO service on the pinned 20260415 (Edison had no rail; NB/Trenton only reached via Amtrak, which skips Edison).validate_fix_gtfs.extend_orphan_service_idsnow INFERS each orphan's weekly pattern from its calendar_dates weekdays (was blanket Mon-Sun) and injects a calendar.txt row spanning 2020-2050.refix_orphan_calendars.pyapplied it in-place to all 143 golden feeds (0 orphans remain).refix_regions_runner.shtransit-only rebuilt the 35 affected regions (walk matrix kept, ~2min each). RevertedREGION_SERVICE_DATEto{}. VALIDATED on 20260415: Edison→Philly 80 / NYC 62. - ★ NYC↔Philly WHOLE-CORRIDOR (
637bffe) — nec priority 1→3 (manifest.json + METROS). nec outranks pinned NYC (2) + midatlantic (1) so it serves the entire corridor. VALIDATED both ways: Manhattan→Philly 99 / Newark→Philly 69 / Philly→NYC 96 (all were NOT-REACHED); NYC-internal subway intact (Manhattan→Brooklyn 36, Flushing 47). - Also this session: per-region
service_datemechanism (kept, override map empty); picker coverage-fallback (_resolve_region_ranked+_select_region: first candidate that_coversthe dest wins → fixed New Haven CT 422→nec 136min); engine path/perf fixes (rp.GPU_PILOT absolute paths;_load_water_boolset-membership not np.isin — the O(N*M) hang).
PENDING (precise)
- #77 nec NYC-zone fares. nec now serves the whole NYC metro at priority 3 but prices transit with the REGIONAL FLAT fare, not pinned's NYC subway/PATH/LIRR/NJT-zone logic (the 08854 flagship routes through nec). Fix: give the nec
_FareContextNYC-aware fares for NYC-area stops (reuse pinned's heuristics, or wirecommute.gtfs_fareszone-OD for the NEC feeds). Also consider UNPINNING the now-idle pinned NYC region to free VRAM (it only serves far-eastern CT east of nec's -72.85). Reversible baseline: nec priority → 1. - res-10 adapter wiring. engine supports
refine="auto"on outbound transit; the geo2 multimodal adapter still sendsrefine="off". Send auto on map clicks → live res-10 in the Manhattan core. National res-10 needs per-metro core matrices. - #69 Losing Ground narrative. SEO metros done (incl QC). The coverage/reach "new data" is a Losing Ground NARRATIVE refresh (
reports/LOSING-GROUND-2026.md+ json + PDF; losingground-cities app on geo :3037) — not yet done. - Precompute re-warm (running).
hexmap-api/precompute_national.pyrelaunched FRESH after nec priority-3 (busted the whole nec bbox). ~28 GPU-h. Monitortail hexmap-api/precompute.log.
DECISIONS (user chose + why)
- Whole NYC↔Philly corridor → nec (priority 3) over "NJ strip only" or "fix fares first" — user wants every hex NYC↔Philly reachable both ways because people commute between the cities. Accepted the NYC flat-fare tradeoff (→ #77).
- Fix the short-calendar issue at the SOURCE (golden fixer + re-fix corpus) over the per-region service_date workaround — "the schedules are short but they go longer, this is the golden source issue."
- Light Certihomes deck (blue + white, wordmark black, no red/black).
- nec region (Philly–NYC–New Haven) for Edison→Philly, over extending the pinned tri-state grid south (byte-identity).
GOTCHAS (don't relearn)
- Golden short-calendar = THE Edison→Philly root cause (NOT isolated platform, NOT the picker). 143/935 feeds had it; fixer now DOW-aware. Transit-only rebuild (
--skip-osm) reuses the walk matrix (fix changes calendars not stops).refix_orphan_calendars.pyis in-place (.bak-orphancal). - stops.parquet DROPS feed prefixes → stop-id COLLISIONS across a multi-feed union (bare "42854"=EDISON in one feed AND a Newark stop mdb-508:42854 in another). The walk matrix keys by PREFIXED id via
encoding_dict(code→value) — never match stops by bare id across a multi-feed region. filter_service_datebakes the date at network LOAD, not query (engine.py ~496) — a feed with no service on the pinned date is invisible; extend the calendar or set a per-region date.- Pause the national precompute during region builds/validation — GPU + kernel-lock contention 3×'d the chicago matrix stage AND caused 429s on validation queries.
- Engine NO --reload →
sudo systemctl restart hexmap-api; then busthexmap-api/cache/iso_*for tested cells. NEVER restart mid-region-build-batch (breaks regions being rebuilt). --forcedoes NOT rebuild a stage whose output exists —rm -rf output/networks/<slug>/transit(+ transfers, manifest) for a transit-only refresh; +rm osm/<slug>_clip.osm.pbfwhen the bbox/state-list grew.- gpu-pilot pushes route via geo bundle (geocoder has no GitHub deploy key):
git bundle create x.bundle main --not <base>→ scp geo (172.26.1.45) →git fetch bundle main→git push origin FETCH_HEAD:main. geocoder'sorigin/maintracking ref stays STALE (false "unpushed") — trust the push confirmations. - OSRM snap distance is the correct water signal (walk-node distance is WRONG — drops rural car-only land).
_is_canadaneeds BOTH-axis boxes (lon-only mis-routes Portland).