From d5485212a7124958c63eb464663a0b5c319f41c4 Mon Sep 17 00:00:00 2001
From: Holden Salomon <holden@arch.fyi>
Date: Sat, 13 Jun 2026 22:13:33 +0000
Subject: [PATCH] Add GPU benchmark results for InsightFace and SigLIP
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

RTX 2070 SUPER results on CUDA 12.8.1:
- InsightFace: 12.8 ms / 78 img/s (8× CPU)
- SigLIP batch 32: 5.4 ms/img / 187 img/s (33× CPU)

Also fix the nvidia-smi verify command to use CUDA 12.8.1 image.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---
 Benchmarks.md | 33 +++++++++++++++------------------
 Setup.md      |  2 +-
 2 files changed, 16 insertions(+), 19 deletions(-)

diff --git a/Benchmarks.md b/Benchmarks.md
index 7ee9812..71cb86c 100644
--- a/Benchmarks.md
+++ b/Benchmarks.md
@@ -24,16 +24,15 @@ Detection (RetinaFace det_10g) + ArcFace embedding (w600k_r50) on a 640×640 ima
 
 | Mode | Model load | Median latency | Throughput |
 | :-- | --: | --: | --: |
-| GPU (`FORCE_CPU=false`) | — | — | — |
+| GPU (`FORCE_CPU=false`) | 2.5 s | 12.8 ms | 78.3 img/s |
 | CPU (`FORCE_CPU=true`) | 4.3 s | 102 ms | 9.8 img/s |
 
-*GPU results pending CUDA 12.8.1 image build.*
-
 ### Notes
 
 - Latency measured over 30 runs after 5 warmup iterations.
 - Input: 640×640 synthetic image. The detection network processes the full input regardless of whether faces are found; timing is representative of real-world single-image throughput.
-- 320×320 input: CPU median 101 ms — detection runtime is dominated by the fixed model overhead, not image size at these resolutions.
+- 320×320 input: CPU 101 ms, GPU 13.4 ms — detection runtime is dominated by fixed model overhead, not image size at these resolutions.
+- GPU is **8× faster** than CPU for InsightFace (12.8 ms vs 102 ms).
 
 ---
 
@@ -47,27 +46,25 @@ Detection (RetinaFace det_10g) + ArcFace embedding (w600k_r50) on a 640×640 ima
 
 | Batch | ms/batch | ms/img | img/s | p95/img |
 | --: | --: | --: | --: | --: |
-| 1 | — | — | — | — |
-| 4 | — | — | — | — |
-| 8 | — | — | — | — |
-| 16 | — | — | — | — |
-| 32 | — | — | — | — |
-
-*GPU results pending.*
+| 1 | 13.1 | 13.13 | 76.1 | 14.2 |
+| 4 | 24.4 | 6.10 | 163.9 | 6.2 |
+| 8 | 45.4 | 5.67 | 176.3 | 5.7 |
+| 16 | 87.7 | 5.48 | 182.4 | 5.5 |
+| 32 | 171.5 | 5.36 | 186.6 | 5.4 |
 
 #### CPU (`FORCE_CPU=true`)
 
 | Batch | ms/batch | ms/img | img/s | p95/img |
 | --: | --: | --: | --: | --: |
-| 1 | 253 | 253 | 4.0 | 315 |
-| 4 | 821 | 205 | 4.9 | 214 |
-| 8 | 1566 | 196 | 5.1 | 214 |
-| 16 | 3181 | 199 | 5.0 | 205 |
-| 32 | 6175 | 193 | 5.2 | 199 |
+| 1 | 216 | 216 | 4.6 | 243 |
+| 4 | 757 | 189 | 5.3 | 192 |
+| 8 | 1450 | 181 | 5.5 | 202 |
+| 16 | 2846 | 178 | 5.6 | 188 |
+| 32 | 5683 | 178 | 5.6 | 188 |
 
-Model load: **18.8 s** (CPU; first load, no cache)
+Model load: **16.1 s** (CPU; first load, no cache)
 
-CPU batching saturates quickly — throughput barely improves past batch 4 (~5 img/s ceiling). On GPU, large batches are expected to see significant throughput gains.
+CPU batching saturates quickly — throughput barely improves past batch 4 (~5.5 img/s ceiling). GPU shows 33× speedup at batch 32 (186 img/s vs 5.6 img/s).
 
 ---
 
diff --git a/Setup.md b/Setup.md
index 83ab2c8..e1501ad 100644
--- a/Setup.md
+++ b/Setup.md
@@ -122,7 +122,7 @@ deploy:
 
 Verify GPU access:
 ```bash
-docker run --rm --gpus all nvidia/cuda:13.3.0-base-ubuntu22.04 nvidia-smi
+docker run --rm --gpus all nvidia/cuda:12.8.1-base-ubuntu22.04 nvidia-smi
 ```
 
 ### AMD (ROCm)