flan/winnow
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add GPU benchmark results for InsightFace and SigLIP 

RTX 2070 SUPER results on CUDA 12.8.1:
- InsightFace: 12.8 ms / 78 img/s (8× CPU)
- SigLIP batch 32: 5.4 ms/img / 187 img/s (33× CPU)

Also fix the nvidia-smi verify command to use CUDA 12.8.1 image.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
flan
2026-06-13 22:13:33 +00:00
co-authored by Claude Sonnet 4.6
parent 5836573b90
commit d5485212a7
2 changed files with 16 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Benchmarks.md
+15 -18
@@ -24,16 +24,15 @@ Detection (RetinaFace det_10g) + ArcFace embedding (w600k_r50) on a 640×640 ima
| Mode | Model load | Median latency | Throughput |
| :-- | --: | --: | --: |
| GPU (`FORCE_CPU=false`) | — | — | — |
| GPU (`FORCE_CPU=false`) | 2.5 s | 12.8 ms | 78.3 img/s |
| CPU (`FORCE_CPU=true`) | 4.3 s | 102 ms | 9.8 img/s |
*GPU results pending CUDA 12.8.1 image build.*
### Notes
- Latency measured over 30 runs after 5 warmup iterations.
- Input: 640×640 synthetic image. The detection network processes the full input regardless of whether faces are found; timing is representative of real-world single-image throughput.
- 320×320 input: CPU median 101 ms — detection runtime is dominated by the fixed model overhead, not image size at these resolutions.
- 320×320 input: CPU 101 ms, GPU 13.4 ms — detection runtime is dominated by fixed model overhead, not image size at these resolutions.
- GPU is **8× faster** than CPU for InsightFace (12.8 ms vs 102 ms).
---
@@ -47,27 +46,25 @@ Detection (RetinaFace det_10g) + ArcFace embedding (w600k_r50) on a 640×640 ima
| Batch | ms/batch | ms/img | img/s | p95/img |
| --: | --: | --: | --: | --: |
| 1 | — | — | — | — |
| 4 | — | — | — | — |
| 8 | — | — | — | — |
| 16 | — | — | — | — |
| 32 | — | — | — | — |
*GPU results pending.*
| 1 | 13.1 | 13.13 | 76.1 | 14.2 |
| 4 | 24.4 | 6.10 | 163.9 | 6.2 |
| 8 | 45.4 | 5.67 | 176.3 | 5.7 |
| 16 | 87.7 | 5.48 | 182.4 | 5.5 |
| 32 | 171.5 | 5.36 | 186.6 | 5.4 |
#### CPU (`FORCE_CPU=true`)
| Batch | ms/batch | ms/img | img/s | p95/img |
| --: | --: | --: | --: | --: |
| 1 | 253 | 253 | 4.0 | 315 |
| 4 | 821 | 205 | 4.9 | 214 |
| 8 | 1566 | 196 | 5.1 | 214 |
| 16 | 3181 | 199 | 5.0 | 205 |
| 32 | 6175 | 193 | 5.2 | 199 |
| 1 | 216 | 216 | 4.6 | 243 |
| 4 | 757 | 189 | 5.3 | 192 |
| 8 | 1450 | 181 | 5.5 | 202 |
| 16 | 2846 | 178 | 5.6 | 188 |
| 32 | 5683 | 178 | 5.6 | 188 |
Model load: **18.8 s** (CPU; first load, no cache)
Model load: **16.1 s** (CPU; first load, no cache)
CPU batching saturates quickly — throughput barely improves past batch 4 (~5 img/s ceiling). On GPU, large batches are expected to see significant throughput gains.
CPU batching saturates quickly — throughput barely improves past batch 4 (~5.5 img/s ceiling). GPU shows 33× speedup at batch 32 (186 img/s vs 5.6 img/s).
---
Setup.md
+1 -1
@@ -122,7 +122,7 @@ deploy:
Verify GPU access:
```bash
docker run --rm --gpus all nvidia/cuda:13.3.0-base-ubuntu22.04 nvidia-smi
docker run --rm --gpus all nvidia/cuda:12.8.1-base-ubuntu22.04 nvidia-smi
```
### AMD (ROCm)