last_modified_at: 2026-09-06

Hermes Agent for Big Computer Vision Projects — Production-Scale CV Engineering with Autonomous Agents — https://www.pirahansiah.com/notes/slides/presentation-cv/ Presentation: Setup, bots, multi-agent orchestration, research, profiling, cron jobs, messaging, group bots, and artifacts for large CV teams.

Last updated: 2026-09-06.

← Tap Left / Press Left Arrow | Tap Right / Press Right Arrow →
COMPUTER VISION ENGINEERING AT SCALE

Hermes Agent for Big Computer Vision Projects

Setup · Bots · Multi-Agent · Profiling · Cron · Messaging · Artifacts

Dr. Farshid Pirahansiah
AI & Computer Vision Engineer • pirahansiah.com

Big CV Projects: Why Standard Tooling Fails

⚠️ Scale & Complexity

• 100+ camera multi-stream ingestion pipelines.

• Model zoo sprawl: YOLO, SAM2, Depth Anything, custom transformers.

• GPU scheduling across clusters & edge nodes.

• Repetitive ops: labeling QA, dataset diffs, eval reports.

🤖 Hermes as CV Control Plane

• One agent orchestrates code, GPUs, data, comms.

• Persistent memory of your pipeline topology.

• Parallel subagents per model/camera group.

• Group bots push results to Slack/Telegram/Discord.

Hermes CV Control-Plane Architecture

Gateway

Single control plane for CLI, GUI, Web, Messaging bots.

Memory

Pipeline topology, GPU inventory, dataset schemas.

Subagents

Per-model / per-camera swarm workers.

Cron

Nightly eval, retraining triggers, report pushes.

Local LLMs (oMLX) for private code; Cloud for heavy reasoning — same agent core.

Project Setup & Configuration

# Install + init a CV project workspace
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup             # pick provider (local oMLX / Gemini / OpenAI)
hermes doctor            # verify ffmpeg, CUDA, Python, torch

# ~/.hermes/config.yaml (CV-tuned)
model:
  default: Qwen3.5-4B-OptiQ-4bit
  base_url: "http://127.0.0.1:8000/v1"
  aliases:
    coder: deepseek/deepseek-coder
    vision: gemini/gemini-2.0-flash-lite
agent:
  max_turns: 40
  tools: [terminal, search_files, browser, vision_analyze]

Building Reusable CV Skills

Teach Hermes your pipeline once — reuse forever via skill_manage

🎯 Detection

• YOLOv8/v11 finetune & inference skill.

• Auto-annotate + label-QA loop.

🧩 Segmentation

• SAM2 prompt + box masking skill.

• COCO→custom mask converter.

📷 Calibration

• Multi-camera intrinsic/extrinsic solver.

• Charuco board + stereo rectify.

Skills load automatically when the task matches — no re-prompting.

Multi-Agent: CV Swarm Orchestration

# One coordinator dispatches parallel CV workers
delegate_task(tasks=[
  {"goal": "Fine-tune YOLOv11 on /data/coco-subset, export ONNX",
   "context": "imgsz=640, epochs=50, device=cuda:0"},
  {"goal": "Run SAM2 on /data/video/*.mp4, emit mask JSON",
   "context": "checkpoint=sam2_hiera_large"},
  {"goal": "Profile GPU memory of inference server under 50 streams",
   "context": "tool=nvidia-smi + py-spy"}
])
# Each returns a verified summary; coordinator merges + writes report.

Profiling CV Pipelines

⚡ GPU & Inference

• nvidia-smi dmon loop captured by agent.

• py-spy dump for Python GIL stalls.

• torch.profiler trace → flamegraph SVG.

• Memory ceiling detection before OOM.

📊 Throughput

• FPS per model under N streams.

• Batch-size sweep (1→32) auto-table.

• Edge vs Datacenter latency compare.

• Bottleneck callout in final report.

Scheduled Jobs: Nightly CV Ops

# Nightly eval + dataset drift report at 02:00
hermes cron create "0 2 * * *" \
  --name "Nightly CV Eval" \
  --script nightly_eval.py \
  --deliver telegram

# Weekly retrain trigger if mAP drops > 2%
hermes cron create "0 4 * * 1" \
  --name "Weekly Retrain Gate" \
  --no-agent --script retrain_gate.sh

Cron survives reboots; results delivered to chat or messaging automatically.

Messaging & Group Bots

Telegram

Channel & group posting of eval dashboards.

Discord

Slash commands: /eval, /profile, /status.

Slack

Webhook alerts on training failure.

# Connect a bot (one-time OAuth / token)
hermes auth add telegram     # or discord / slack
hermes bot create cv-alerts --platform telegram --group "CV War Room"

Artifact Management

📦 What Hermes Produces

• Trained weights (ONNX, GGUF, safetensors).

• Benchmarks: CSV + interactive HTML charts.

• Diagrams: pipeline SVG, architecture graphs.

• Datasets: diff reports, schema validation.

☁️ Storage & Traceability

• Auto-push to HuggingFace Datasets / Models.

• Git-LFS for large binaries.

• Every artifact tagged with commit + eval hash.

• Reproducible via skill_manage replay.

Closing the Research → Production Loop

1. Lit

ArXiv sweep → SOTA table.

2. Proto

Reproduce repo, patch bugs.

3. Train

Subagent swarm fine-tunes.

4. Ship

Artifact + bot alert + doc.

Memory retains lessons; next cycle starts faster.

Case Study: 100-Camera Edge CV Fleet

Challenge

Real-time detection across 100 USB cameras on edge NPUs; central eval nightly.

Hermes role: spawn 10 subagents (10 cams each), profile NPU memory, push daily mAP to Telegram group.

Outcome

✓ 3× faster camera onboarding.

✓ OOM eliminated via profiling skill.

✓ Zero manual eval reports.

✓ Full artifact lineage per site.

Key Takeaways

✔ Setup once: config + CV skills become permanent lab memory.

✔ Orchestrate at scale: multi-agent swarms per model/camera group.

✔ Operate autonomously: cron jobs + group bots run the ops loop.

✔ Profile & ship: bottleneck calls + traced artifact lineage.

pirahansiah.com/notes/docs/cv/

Thank You & Discussion

Hermes Agent for Big Computer Vision Projects

CV Documentation • GitHub