Four agents run a 4-GPU vision lab: experiment triage, distillation calls, paper sweeps
A computer-vision researcher — semantic segmentation, knowledge distillation and vision foundation models (SAM) — runs training on a 4-GPU workstation. Over 17 days, four agents (Claude, Codex, DeepSeek, OpenCode) processed 418 events as his AI lab assistants, spanning experiment management and academic literature work.
Background
Daily research means juggling parallel training runs on different GPUs, tracking each experiment's progress and validation mIoU, judging new directions (is NanoSAM distillation on VOC worth starting?), and keeping up with CVPR/ECCV/ICCV output — all scattered across terminals, logs, paper sites and local folders.
The bottlenecks
Scattered multi-GPU management
Knowing what each card is running and how each experiment is doing meant logging in, listing processes and grepping logs one by one.
Manual experiment comparison
Deciding which runs to continue or kill requires comparing mIoU and progress across scattered log files.
Slow feasibility assessment
Every new research direction demanded from-scratch investigation, stretching decision cycles.
Literature-tracking load
Filtering SAM-related work out of top-conference volume, downloading and digesting PDFs, is a continuous drain.
How it unfolded
Direction 1 — GPU experiment management and analysis
Agents read GPU process and load data to produce a live overview of what each of the 4 cards is running; compiled training progress and validation mIoU across experiment lines and recommended which to continue and which to stop to free compute; and tracked the SAM distillation work — MobileSAM training time and checkpoint updates — while assessing the feasibility of NanoSAM distillation on VOC against current data and literature.
Direction 2 — Literature search and organization
The plan: search 2025–2026 CVPR/ECCV/ICCV for SAM-related papers, batch-download PDFs, parse abstracts and file everything. The retrieval list and organization scheme landed; the batch download-and-parse step didn't finish in this cycle — Codex crashed repeatedly when spawning sub-agents to parallelize it.
Inside the Workspace
What changed
One view over the whole lab
Process-by-process, log-by-log checking became a structured overview: what's running on each card, how it's going, what to stop.
Data-backed continue/kill calls
Continue-or-terminate recommendations grounded in actual training curves cut gut-feel decisions and wasted compute.
Faster feasibility answers
The NanoSAM assessment arrived pre-assembled — training status plus literature — before committing GPUs to it.
A first integration of the research toolchain
GPU monitoring, experiment analysis and paper search converged into one Workspace — fewer context switches across terminals, logs and websites.
Delivered in this period
- Live status overview of experiments on 4 GPUs
- Cross-experiment training-progress and mIoU comparison
- Continue/terminate recommendations for running experiments
- MobileSAM training and checkpoint tracking
- NanoSAM-on-VOC distillation feasibility assessment
- SAM-paper retrieval list and organization scheme (execution incomplete)
Takeaway
The agents stopped being question-answering tools far from the research environment and entered the real experiment-management loop. In ML research settings, that targets the genuine bottlenecks — experiment management, literature tracking, compute monitoring — and gives researchers back time for experiment design, analysis and writing.
Put agents to work in your team
Start from one painful, repetitive workflow — the way the teams in these stories did.
