← gpu-pilot · json · source: CONTINUE_6_28_2026.md

▶ CONTINUE 6/28 (LOS AI-1003 moved to gtfs3's dedicated RTX 5060 Ti / qwen2.5-coder:14b — newest handoff)

Newest handoff: moved the mortgage LOS AI-1003 assembly off geo2's shared GPU onto gtfs3's new dedicated RTX 5060 Ti (16GB) running qwen2.5-coder:14b (an upgrade over the 7b) — always-on, quota-free, warm ~0.6s. Fixed the new GPU with no reboot / no commute-DB downtime (595-open driver hot-swap), installed Ollama LAN-bound, and freed geo2. macstudio was investigated + rejected (contended); geo<->Mac passwordless SSH staged. · updated 2026-06-28 14:10

TL;DR

  • LOS AI host move (certihomes-los 4b11981 on geo2): OLLAMA_BASE=http://172.26.1.152:11434, OLLAMA_MODEL=qwen2.5-coder:14b. Verified live: /health=qwen2.5-coder:14b@ollama, 'assembled via ai:qwen2.5-coder:14b' (real URLA), warm ~0.6s, ~9GB/16GB resident. geo2's 7b removed -> its 32GB GPU freed for ComfyUI/avatar.
  • gtfs3 GPU brought up with NO reboot / NO DB downtime: RTX 5060 Ti (10de:2d04) wasn't enumerated by driver 580 -> installed nvidia-driver-595-open (DKMS, kernel 6.8.0-124) -> hot-swapped modules (rmmod nvidia stack -> modprobe nvidia) -> nvidia-smi lists it. Ollama installed (systemd, 0.0.0.0:11434, KEEP_ALIVE=-1, CUDA 13). Persists across reboots.
  • Model-fit decision: 16GB card -> qwen2.5-coder:14b (~9-12GB, fits fully = sweet spot). 32b (~20GB) does NOT fit 16GB (needs a 24GB+ card / RTX 5090); Q2/Q3-cramming loses more than it gains; CPU-offload too slow. 'A model that fits fully in VRAM beats a bigger one that spills to CPU.'
  • GOTCHAS: new Blackwell 'No devices found' = driver too old (595-open for 5060 Ti); hot-swap GPU driver w/o reboot on headless boxes; curl|sh installers need run-as-root via 'sudo -S bash file < ~/secert.txt' over non-interactive SSH; Ollama LAN bind override; manage models via HTTP API (no SSH); geo's :11434 firewalled from peers; macstudio SSH locked + contended; 172.26.1.152 = gtfs3 (NOT macmini).

CONTINUE — 2026-06-28 — Moved the LOS AI-1003 to gtfs3's dedicated RTX 5060 Ti (qwen2.5-coder:14b)

Read first: memory fleet-llm-inference-hosts, certihomes-poslos-deck-pipeline, certihomes-raise-stack-live. Prior handoff: CONTINUE_6_27_2026.md (deck→A, POS/LOS built, Qwen wiring). Repos: gpu-pilot main = (this wrap commit); certihomes-los main = 4b11981 (on geo2, pushes direct to GitHub https).

DONE this session

  • Moved the LOS AI-1003 assembly off geo2 → gtfs3's dedicated RTX 5060 Ti 16GB running qwen2.5-coder:14b (an upgrade over the geo2 7b). certihomes-los 4b11981: OLLAMA_BASE=http://172.26.1.152:11434, OLLAMA_MODEL=qwen2.5-coder:14b. Verified live: /health=qwen2.5-coder:14b@ollama, assembled via ai:qwen2.5-coder:14b (real URLA), warm ~0.6s, ~9GB/16GB resident.
  • Fixed gtfs3's new GPU with NO reboot / NO DB downtime: the RTX 5060 Ti (10de:2d04) wasn't enumerated by driver 580 → installed nvidia-driver-595-open (DKMS for kernel 6.8.0-124), then hot-swapped modules (rmmod nvidia_uvm nvidia_drm nvidia_modeset nvidia → modprobe nvidia) → nvidia-smi lists the card. Persists across reboots (DKMS + initramfs).
  • Installed Ollama on gtfs3 (systemd, LAN-bound 0.0.0.0:11434, OLLAMA_KEEP_ALIVE=-1, GPU-detected CUDA 13). Pulled qwen2.5-coder:14b (+7b).
  • Freed geo2: ollama rm qwen2.5-coder:7b — geo2's 32GB GPU (RTX 5090, was ~maxed by ComfyUI/avatar) no longer at risk of a LOS-triggered model load evicting those processes.
  • Investigated macstudio (172.26.1.90) first — rejected: Ollama LAN-reachable + has coder 14b/32b, but a pinned qwen2.5vl:7b (14GB) means it can't keep a coder warm (thrash → timeouts); SSH is locked (API-only). gtfs3 is the better home.
  • Staged passwordless SSH from geo + geo2 → macstudio/macmini: ensured ed25519 keys on geo (existing) + geo2 (new); macstudio/macmini aliases already in both ~/.ssh/config. Printed both pubkeys for the user to add to the Macs' authorized_keys (the user does that leg — needs the Mac login password; I'm API/SSH-locked out of the Macs).

PENDING (precise)

  • Add geo/geo2 pubkeys to macstudio + macmini ~/.ssh/authorized_keys (user-side; commands + the 2 pubkeys were provided). Then ssh -o BatchMode=yes macstudio hostname from geo/geo2 should work passwordlessly. (macmini real IP TBD — 172.26.1.152 in old notes is actually gtfs3.)
  • POS→LOS live wiring: import posSubmit.js into AFLConfirmationForm on pos.tlcengine.com (1 import + 1 call) so the borrower POS submits into the LOS (today the full flow runs via los.tlcengine.com's own intake tab).
  • Real AUS (Fannie DU/Freddie LPA): needs a sponsoring lender — sandbox until then.
  • Drive copy of the deck: still not automated (no Drive rclone remote; Drive MCP base64 too big for context). Deck live at losingground.certihomes.com/certihomes-investor-deck-d286894e.pdf.
  • Durable-A commercial gates (signed pilot, parking the 100-repo sprawl) + #61/#62.

DECISIONS (user chose + why)

  • gtfs3's dedicated RTX 5060 Ti (16GB) is the LLM-inference home, running qwen2.5-coder:14b — "a model that fits fully in VRAM beats a bigger one that spills to CPU." 14b@Q4 (~9-12GB) fits with headroom; 32b (~20GB) does NOT fit 16GB (would need a 24GB+ card / the RTX 5090) — Q2/Q3-cramming loses more than it gains, CPU-offload is too slow.
  • Self-hosted Qwen over the Claude proxy (weekly-quota-limited) and over a paid API key — always-on, zero per-call cost, reinforces "runs on our own GPU fleet."
  • No reboot of gtfs3 (it's the commute DB host) — hot-swapped the driver modules instead; zero DB downtime.

GOTCHAS (don't relearn) — full detail in memory fleet-llm-inference-hosts

  • New Blackwell card "No devices found" with a driver loaded = driver predates the SKU → nvidia-driver-595-open for the 5060 Ti (ubuntu-drivers devices gives the recommended).
  • Hot-swap a GPU driver w/o reboot on a headless box: rmmod the nvidia stack → modprobe nvidia (works because nothing pins nvidia_drm).
  • curl|sh installers fail over non-interactive SSH ("sudo: a terminal is required"); run as root: sudo -S -p "" bash /tmp/x.sh < ~/secert.txt.
  • Ollama LAN bind: systemd override Environment="OLLAMA_HOST=0.0.0.0:11434" + OLLAMA_KEEP_ALIVE=-1; manage models without SSH via the HTTP API (/api/pull, /api/chat, /api/ps).
  • geo's Ollama :11434 is firewalled from peer hosts; macstudio SSH is locked (API-only) + contended. sudo pw file is ~/secert.txt (geocoder + gtfs3).
  • 172.26.1.152 = gtfs3 (commute DB + now the inference GPU) — NOT macmini (old trip/macmini=152 note is stale).

Comments