YOLO-NAS Strikes Back: Still the Fastest Detector on Hardware You Already Own

TL;DR: quantized to INT8 at 320px, YOLO-NAS-S does 201 FPS on a laptop CPU and 148 FPS on the integrated GPU next to it, all measured below. On a mid-range laptop dGPU through TensorRT it reaches 1071 FPS at FP16. It was designed to survive quantization and it does: on the full COCO val2017 set INT8 costs it 0.4 AP (47.3 → 46.9), and it drops into Frigate with preprocessing and NMS inside the exported graph. What aged badly is the tooling, not the model: no release since April 2024 and a weights host that no longer resolves. So I rebuilt the architecture on a modern stack and checked it against the original, where class scores come out bit-identical. The pretrained COCO weights are still non-commercial, and open replacements are in training.

Updated 23–24 Sep 2026. The latency tables now come from the companion repo’s regenerable runtime matrix, at 320px and 640px instead of 256px and 640px, and each cell is the median of three sessions in performance mode, with the range in Limitations. The headline FPS moved with the input size and the benchmark script. The COCO figure is the repo’s current measurement, and the library now returns supervision detections and ships a tracker, measured on 45 SportsMOT sequences; what changed is listed at the end.

Most detector write-ups compare COCO mAP and stop there. That ranking answers a question almost nobody deploying to the edge is actually asking, which is not “what is the most accurate model” but “what is the most accurate model that fits in my latency budget, on my hardware, at the precision I can afford to run”.

Change the question and the ranking changes with it. This post makes that case in four steps: why YOLO-NAS is still the right answer for a lot of edge work, why running it in 2026 is harder than it should be, what is actually inside the model, and how I rebuilt it in a way you can check rather than trust.

---

Part 1: The case for YOLO-NAS on the edge

Three properties, none of them about leaderboard position.

It was built to be quantized

Most networks are quantized as an afterthought and lose accuracy unevenly. YOLO-NAS’s backbone uses quantization-aware RepVGG blocks, restructured specifically so the fused weights do not develop the extreme value ranges that wreck INT8 [4]. We will see exactly what that restructuring does in Part 3; the effect is not subtle.

Measured on the full COCO val2017 set (5000 images), with the OpenVINO IR the companion repo exports, FP16 against INT8:

YOLO-NAS-S	FP16 AP	INT8 AP	AP lost to INT8
320×320	38.36	37.92	−0.44 pt (−1.1%)
640×640	47.32	46.91	−0.41 pt (−0.9%)

Under half a point of AP at either size. For contrast, my head-to-head benchmark put both models through the same OpenVINO pipeline on an Intel iGPU, on a 500-image COCO subset at 256px:

Model	FP32 mAP	INT8 mAP	mAP lost to INT8
RF-DETR Nano	0.428	0.338	−21.2%
YOLO-NAS-S	0.345	0.336	−2.5%

RF-DETR is the more accurate model in float. At INT8 the two land within noise of each other, because one of them was designed for this and the other was not. The two tables use different protocols, so read the second one for the gap between the models and the first for what YOLO-NAS loses on its own. If your deployment target is INT8, and on edge hardware it usually is, that is the whole argument.

It is fast on hardware you already have

Here is the part that surprised me. I measured YOLO-NAS-S across three devices and three precisions on one laptop. Model inference only (NMS left to the caller), batch 1, single stream, synchronous, median of 30 runs after 8 warmup runs, on a real COCO frame. I ran the whole matrix in three separate sessions, and each cell below is the median of the three:

Input 320×320

Runtime	Device	FP32	FP16	INT8
PyTorch	CPU i7-12700H	38.41 ms · 26 FPS	—	—
OpenVINO	CPU i7-12700H	17.13 ms · 58 FPS	16.60 ms · 60 FPS	4.98 ms · 201 FPS
OpenVINO	iGPU Iris Xe	7.85 ms · 127 FPS	7.30 ms · 137 FPS	6.76 ms · 148 FPS
PyTorch	dGPU RTX 3060	4.76 ms · 210 FPS	5.12 ms · 195 FPS	—
TensorRT	dGPU RTX 3060	2.31 ms · 433 FPS	0.93 ms · 1071 FPS	not measured

Input 640×640

Runtime	Device	FP32	FP16	INT8
PyTorch	CPU i7-12700H	118.65 ms · 8 FPS	—	—
OpenVINO	CPU i7-12700H	65.63 ms · 15 FPS	66.44 ms · 15 FPS	20.34 ms · 49 FPS
OpenVINO	iGPU Iris Xe	20.24 ms · 49 FPS	20.16 ms · 50 FPS	12.55 ms · 80 FPS
PyTorch	dGPU RTX 3060	9.68 ms · 103 FPS	6.25 ms · 160 FPS	—
TensorRT	dGPU RTX 3060	5.59 ms · 179 FPS	2.18 ms · 459 FPS	not measured

FPS is 1000 / latency on one synchronous stream, which is not throughput. Four things worth pulling out.

The runtime matters more than the hardware. Swapping PyTorch for OpenVINO on the same CPU is a 2.2x speedup at 320px, before touching precision. Add INT8 and it becomes 7.7x: 38.41 ms down to 4.98 ms. No new silicon, and the accuracy cost is the 0.44 AP from the table above. At 640px the same INT8 path is a 5.8x win over PyTorch.

A laptop CPU at 201 FPS is enough for most edge work. If you are running four camera streams at 15 FPS, that is 60 FPS of demand against a 201 FPS budget, on a machine with no accelerator.

FP16 is not automatically a win. On the dGPU at 320px, PyTorch FP16 is slower than FP32 (5.12 vs 4.76 ms), and it was slower in each of the three sessions separately, so the gap is not drift: the tensors are too small for the conversion overhead to pay off. At 640px the same swap is a clear win (6.25 vs 9.68 ms). On the OpenVINO CPU plugin FP16 does essentially nothing, which is expected, since it upconverts to FP32 internally.

On the iGPU, INT8 only pays at the larger input. At 640px it cuts latency from 20.16 ms (FP16) to 12.55 ms, 38% less. At 320px the three precisions sit within 1.1 ms of each other (6.76 to 7.85 ms), and the INT8 cell alone moved between 5.93 and 6.98 ms across sessions, so something other than arithmetic sets the floor there. I have not profiled what. An earlier version of this post showed the opposite at 640px, INT8 slower than FP16, from a benchmark script that has since been removed from the repo because its table could not be regenerated. The repo’s own INT8 export does not reproduce that result. Machine load does not explain it either: across the three sessions and the single everyday-mode run that preceded them, INT8 at 640px ran between 12.04 and 12.62 ms and FP16 between 19.25 and 20.18 ms, so INT8 was the faster of the two every time.

These are latency numbers: one image at a time, waiting for each result. A throughput pipeline with async requests and several streams will report higher FPS on the same hardware, which is why they are not directly comparable to the FPS in my earlier benchmark.

The clip below is the other kind of number: everything from video decode to tracking, with four inference requests in flight, timed end to end.

It drops into the tools edge people actually run

Frigate, the self-hosted NVR, supports YOLO-NAS natively, and modern-yolonas can export a graph with preprocessing and NMS already inside it. That turns integration into a config file rather than a service. The details are in Part 5, once the pieces it depends on have been introduced.

I am not claiming YOLO-NAS beats every 2026 detector on accuracy. It does not, and my own benchmark says so. The claim is narrower and more useful: on quantized edge hardware, the accuracy you keep matters more than the accuracy you started with, and very few models keep as much of it.

So the model is worth having. The problem is getting hold of it.

---

Part 2: Why that is harder than it should be

Every item below is a wall someone hits on their first afternoon with YOLO-NAS. None of them are arguments against the architecture; they are arguments for giving it a maintained home.

The releases stopped

super-gradients’ last PyPI release is 3.7.1, uploaded 2024-04-08 [1]. The GitHub repository is not abandoned; it was pushed as recently as February 2026, so “dead project” is the wrong description. The releases simply stopped.

The dependencies contradict each other

It pins onnxruntime==1.15.0, which has no wheel for CPython 3.12 or 3.13. On 3.11 it fails for a better reason: super-gradients pins numpy<=1.23 while its own pinned onnxruntime requires numpy>=1.24.2. The requirement set contradicts itself. It survives on Python 3.10 only by accident of per-wheel metadata: onnxruntime 1.15.0’s cp310 wheel asks for numpy>=1.21.6, while its cp311 wheel asks for numpy>=1.24.2.

Python	pip install super-gradients==3.7.1
3.13	✗ no onnxruntime wheel
3.12	✗ no onnxruntime wheel
3.11	✗ numpy<=1.23 vs numpy>=1.24.2, unsatisfiable
3.10	✓ installs (152 packages)

Even then it installs but will not import: it uses pkg_resources, removed in setuptools 81. Pin setuptools<81 and it comes back.

The weights no longer download

This is the one that actually stops you. models.get("yolo_nas_s", pretrained_weights="coco") fetches https://sghub.deci.ai/models/yolo_nas_s_coco.pth. That hostname returns no A record, no AAAA record and no CNAME, from Cloudflare (1.1.1.1), Google (8.8.8.8) and Quad9 (9.9.9.9) alike. deci.ai itself resolves fine from all three, and the zone’s SOA serial was 2026070101 when I checked: an actively maintained zone with that subdomain removed from it.

I did not find an official alternative host. What I can say is that the documented download path does not work, so the ordinary way of obtaining these weights no longer functions. modern-yolonas ships them as safetensors from the Hugging Face Hub instead.

The licence, and what we are doing about it

The super-gradients code is Apache-2.0. The YOLO-NAS pretrained weights are not. They ship under the Super Gradients Model EULA, which restricts them to non-commercial use [2]. Because the repository’s LICENSE file says Apache-2.0 and the weights arrive through the same API as everything else, it is easy to assume the licence flows through. It does not, and this is the single most expensive misunderstanding available in this whole topic.

It applies to any mirror, including mine. Converting a checkpoint to safetensors and putting it on the Hugging Face Hub changes the file format, not the licence, so the loader prints the restriction every time it fetches weights rather than burying it in a README.

We are training open-licensed COCO weights so this stops being permanent. That work is in progress and I am not going to put a date on it; until it lands, the checkpoints modern-yolonas distributes are still Deci’s, under Deci’s terms. The architecture has always been free. It is the trained parameters that carry the restriction, and that is a solvable problem.

---

Part 3: Inside the model

To rebuild something faithfully you have to understand it, and the three ideas below are the ones that matter: the block that makes it quantize well, the head that makes it accurate, and the rule that decides what it learns from.

Meet Gate-Cam

One toy scenario, reused for everything that follows.

Gate-Cam is a camera bolted above a driveway, running on a mini-PC with no discrete GPU. Its whole job is to notice the delivery van. The frame is 640×640 after letterboxing, and for every box the detector has to say where each edge is and how sure it is. We will follow the van’s right edge, measured rightwards from the cell that predicts it.

Gate-Cam	value
Input frame	640 × 640
Feature levels (strides)	8, 16, 32
Cells per level	80², 40², 20²
Box predictions per frame	8400
Distribution bins per box side	17

Every number in that table comes straight out of the model, and each one is derived below.

Before the maths, here is the whole network. Click any block to see what it receives, what it emits, and how much of the 19.05M parameter budget it owns. The shapes are not illustrative: they were captured with forward hooks on the real model.

Two things jump out once you can click through it. Stage 4 alone holds 9.0M parameters, 47% of the model, because channels are widest exactly where the feature map is smallest. And the neck blocks take three inputs each, not one: that is the PAN pattern, pulling skip connections straight from the backbone so fine detail survives the trip down and back up again.

The backbone: two shapes, one function

Intuition first: a network can wear two different shapes: a wide, branchy one that is easy to train, and a narrow, fast one that computes exactly the same function. RepVGG trains in the first shape and deploys in the second [3].

During training, a RepVGG block runs three parallel branches on the same input: a 3×3 convolution, a 1×1 convolution, and an identity shortcut. At inference all three collapse into a single 3×3 convolution. Not approximately. Exactly. Convolution is linear, so a sum of convolutions is a convolution:

$W_{\text{fused}} = W_{3\times3} + \text{pad}(W_{1\times1}) + W_{\text{identity}}$

The 1×1 kernel is zero-padded into the centre of a 3×3, and the identity branch becomes a 3×3 with a single 1 in the middle of its own input channel.

The reimplementation carries both shapes and the conversion between them, because the checkpoints store the unfused branches.

The head: a distribution, not a number

That backbone feeds a head, and the head is where the two remaining Gate-Cam numbers, 8400 and 17, come from.

Intuition first: asking a network “how far away is the box edge?” invites a single confident number even when the network is not sure. Asking it “what does your belief about the distance look like?” lets it answer “probably 6 cells, maybe 7”, or admit it is torn between two answers. Distribution Focal Loss does the second [5].

The head is anchor-free: one prediction per feature-map cell, over three levels. Szeliski’s §6.3 (Object detection) is the clearest short account of why the field moved off anchor boxes [7]. The particular head YOLO-NAS uses, pairing a distribution-based box branch with task-aligned assignment, comes from the PP-YOLOE line of work [8].

Now the distance. Each box has four sides, and each side gets 17 bins (reg_max=16, so bins 0 through 16). The network emits a probability over those bins, and the decoded distance is the expected value:

$d = \sum_{i=0}^{16} i \cdot p_i \times \text{stride}$

You can see this in the tensor shapes too: the raw regression output is (1, 8400, 68), and $68 = 4 \text{ sides} \times 17 \text{ bins}$.

The interesting failure is what happens when the network is torn. Drag the bars below, and try the Two minds preset, where the head splits its vote between bin 2 and bin 12. The expectation lands near bin 7, where almost no probability mass lives, so the decoded edge goes somewhere the network never argued for.

That is the honest limitation of an expectation-decoded head: it reports a mean, and a mean is a poor summary of a bimodal belief. In practice unimodal distributions dominate and the mean is fine. But when you see a box edge sitting confidently in empty space, this is a mechanism worth suspecting.

Assignment: which cell owns the van

The head produces 8400 candidates. Training needs a rule for which of them should be pushed toward the answer, and that rule shapes what the detector becomes.

YOLO-NAS uses task-aligned assignment [6]: score each candidate by combining how well it classifies with how well it localizes, then let the best ones own the object.

$t = s^{\alpha} \cdot \text{IoU}^{\beta}$

Intuition first: a box that overlaps the van beautifully but calls it a mailbox is not a good prediction, and neither is one that says “van!” while boxing the hedge. The product forces a candidate to be good at both jobs at once.

Since IoU is doing most of the work in that formula, it is worth having a feel for how fast it moves. Drag the boxes:

That is the whole model: a backbone that fuses, a head that predicts distributions, and an assignment rule that rewards placement over confidence. Enough to rebuild it.

---

Part 4: Rebuilding it, and proving it right

The state_dict is the API

A state_dict is a dictionary of tensors with names like backbone.stem.conv.branch_3x3.conv.weight. Those names are not decoration. They are the load-bearing interface. If your module hierarchy uses the same attribute names, the checkpoint drops in; if it does not, nothing loads.

So the reimplementation is constrained in an unusual way: I was not free to design a clean class layout, because the layout is the API. yolo_nas_s has 921 tensors, and all 921 have to land in the right place.

That constraint is also what makes the work checkable. Either all 921 keys match with zero missing and zero unexpected, or they do not.

Variant	Checkpoint tensors	Loads strict=True?
yolo_nas_s	921	✓ 0 missing / 0 unexpected
yolo_nas_m	965	✓ 0 missing / 0 unexpected
yolo_nas_l	1097	✓ 0 missing / 0 unexpected

Loading cleanly is necessary but nowhere near sufficient. Every tensor could be in the right slot and the forward pass could still compute something subtly different.

Designing a parity test when the weights host is gone

The obvious experiment, loading pretrained weights into both libraries and comparing, is blocked by Part 2: the host is gone. So I side-loaded. Build the super-gradients architecture with models.get(..., num_classes=80), which needs no download, then push the same safetensors tensors into both implementations.

That turns out to be a better test than the obvious one. Both sides get byte-identical weights, so the comparison isolates the architecture from weight loading. And super-gradients on Python 3.10 resolves to the same torch build I was already using, so both models run in one process, on one torch, against one input tensor in memory. No serialization, no cross-environment drift.

What the comparison says

Variant	Class scores, worst |Δ|	Boxes, worst |Δ| relative to scale
yolo_nas_s	0.000e+00	1.853e-07
yolo_nas_m	0.000e+00	1.134e-07
yolo_nas_l	0.000e+00	7.473e-08

Class scores matching bit for bit across 672,000 values per image is a strong statement: the backbone, the neck and the classification branch produce identical bits. Whatever differs is confined to the box-regression decode.

But “the boxes differ by a tiny amount” is easy to wave through and hard to defend. A tiny difference could still be a real formula difference that happens to be small on these inputs. So I ran the same comparison in double precision, at matched input counts:

dtype	worst box |Δ|	relative to scale
float32	6.104e-05	9.265e-08
float64	2.274e-13	3.452e-16

Interpretation: the relative error falls by roughly $2^{28}$ when the mantissa goes from 23 bits to 52. That tracks the precision change almost exactly, which is the signature of rounding. A genuine difference in the maths would have kept roughly the same relative error in float64, because more precision does not repair a wrong formula. The two implementations are doing the same arithmetic in a different summation order, which is unsurprising given the decode sums 17 weighted bins and adds an anchor offset.

One caveat worth stating rather than burying: it would be wrong to summarise this as “every difference is within one ULP”. Around 8,180 elements differ by more than one unit-in-the-last-place of their own magnitude, because a near-zero output computed from large intermediates inherits absolute error from the larger scale. The defensible claims are the relative-to-scale figure and the float64 collapse, not a ULP count.

The external check

Parity against super-gradients proves the two implementations agree. It does not prove either is correct, since a shared misunderstanding would be invisible to it.

So, independently, the rebuilt pipeline goes through pycocotools on the full 5000-image COCO val2017 set, letterboxed, with NMS at IoU 0.70:

Variant	Measured AP	Deci’s published AP	Gap
YOLO-NAS-S	47.3	47.5	−0.2
YOLO-NAS-M	51.3	51.5	−0.2
YOLO-NAS-L	52.0	52.2	−0.2

The same gap across all three variants, and I do not know its cause. The candidates tested and ruled out, one variable at a time on YOLO-NAS-S: crowd annotations (stripping them lowers AP by 0.43, so the evaluator already handles them), padding placement (super-gradients’ bottom-right padding scores 0.2 worse than centred), and the NMS threshold (0.70 is the best of 0.65, 0.70 and 0.80). Closing the last 0.2 needs Deci’s own evaluation code. An earlier version of this post quoted 0.4761 for YOLO-NAS-S. I cannot trace that figure to a saved artifact, so it is withdrawn; the table above is the one that regenerates.

You can reproduce the parity tables with examples/parity_check.py, which writes them to output/parity.md, and the AP table with examples/model_table.py.

---

Part 5: Deploying to Frigate

Which brings us back to where Part 1 left off. Frigate supports YOLO-NAS natively, and the export puts preprocessing and NMS inside the graph, so the model takes raw uint8 frames and emits a flat [D, 7] tensor of [batch, x1, y1, x2, y2, confidence, class_id]:

yolonas export --model yolo_nas_s --format openvino --target frigate --input-size 320

detectors:
  ov:
    type: openvino
    device: GPU
model:
  model_type: yolonas
  width: 320
  height: 320
  input_tensor: nchw
  input_pixel_format: bgr
  path: /config/model_frigate.xml

That is the whole integration. No custom postprocessing service, no NMS in Python, no glue to maintain. Frigate hands the graph raw BGR frames and the channel swap happens inside the exported model, which is why the config says bgr even though the weights want RGB.

The convenience has a price, and it is worth knowing. ONNX’s NonMaxSuppression operator is slower than torchvision’s: on the dGPU through TensorRT FP16, NMS inside the graph costs 49% more than the model followed by torchvision NMS at 320px (1.97 vs 1.32 ms), and 35% more at 640px (3.40 vs 2.51 ms). What it buys is one self-contained file with no Python in the inference path, which is the thing Frigate needs.

Combined with the INT8 numbers from Part 1, that is a driveway camera doing real-time detection on integrated graphics, with a config file instead of an integration project.

Picking the confidence threshold is the actual work of deploying a detector, and this street shows why no benchmark table picks it for you. YOLO-NAS-S, on CPU, returns 19 detections at 0.40 and above, and 13 at 0.6 and above. The six in between, the amber boxes:

Detection	Score	What it is
person	0.58	real, half hidden behind the man in the striped shirt
umbrella	0.56	real, the market umbrella
person	0.52	real, far back above the silver car
motorcycle	0.51	a duplicate box spanning both parked scooters
umbrella	0.47	real, the second umbrella
motorcycle	0.40	real, the second scooter

Raising the threshold from 0.40 to 0.6 removes one wrong box and five real objects with it. Below 0.40 the list keeps going: the next detection down is a car at 0.36, real and partly hidden behind the SUV, and I did not audit the seven boxes between it and 0.25. Which side of the trade you want depends on whether a miss or a false alarm is the expensive mistake in your application.

What changed since August

The library kept moving after this post went up. The changes that affect how you use it:

Detections are supervision objects (0.4.0). YoloNASDetector (renamed from Detector in 0.5.0) returns an sv.Detections, so filtering is slicing and every annotator, zone and tracker in supervision reads the output with no adapter. Field names follow supervision: xyxy, confidence, class_id, plus data["class_name"].

import cv2
from modern_yolonas import YoloNASDetector

detector = YoloNASDetector("yolo_nas_s")
detections = detector(cv2.imread("driveway.jpg"))   # sv.Detections

vans = detections[detections.data["class_name"] == "truck"]
confident = detections[detections.confidence > 0.5]

Training runs on PyTorch Lightning, and the source licence is Apache-2.0 (0.5.0). PTQ and QAT are CLI commands (yolonas quantize, yolonas qat). The Apache-2.0 licence covers the code. The pretrained weights are still Deci’s, under Deci’s terms.

On main, not yet in a PyPI release: TensorRT export, the working OpenVINO INT8 export that produced the tables above, pre-exported ONNX, OpenVINO and TensorRT files on the Hugging Face Hub, feature embeddings from the backbone (per image or per box, from the same forward pass as the detections), and a Deep HM-SORT tracker [9] that associates on those per-box embeddings. It follows supervision’s update_with_detections contract, so it drops in wherever a supervision tracker did.

The tracker, measured

A demo clip settles nothing, so I measured the tracker on all 45 validation sequences of SportsMOT [10], basketball, football and volleyball broadcast footage, against ByteTrack [11] from roboflow/trackers. Every tracker replayed the same cached YOLO-NAS-L detections, so a difference can only come from how it associates them, and TrackEval scored the result:

Tracker, real detections	HOTA	ID switches
ByteTrack	53.7	1,885
Deep HM-SORT, motion only, on ByteTrack’s thresholds	49.3	2,385
Deep HM-SORT, motion only	47.5	3,080
Deep HM-SORT, its default (appearance on)	47.6	3,172

ByteTrack wins, in every sport, so it is now the default tracker on the repo’s development branch, with Deep HM-SORT one flag away. Giving Deep HM-SORT ByteTrack’s detection thresholds closes 1.8 of the 6.2 points between the two motion-only trackers. The other 4.4 come from some difference I have not isolated; the largest one is that ByteTrack predicts where a player moves with a Kalman filter and Deep HM-SORT does not.

The appearance cue is the more interesting result, because it does not help. On ground-truth boxes, where association is the only source of error, the same tracker scores lower with appearance than without:

Tracker, ground-truth boxes	HOTA
Deep HM-SORT, motion only	89.7
ByteTrack	87.5
Deep HM-SORT, its default (appearance on)	86.1

The reason is in the embeddings themselves. In the detector’s feature space, two different players in the same frame sit at a median cosine distance of 0.092, and the same player one second later at 0.059 (tracking_separability.json in the repo, over 295,573 boxes). Features trained to find and classify objects were never trained to tell two players apart, and here they barely do, so the tracker fuses in a cost that carries almost no identity. The full tables, the per-sport split and the conditions are in the tracking benchmark. SportsMOT is CC BY-NC 4.0; these are metrics on its validation split, and none of its frames are redistributed.

The open-licence weights have a written plan: COCO-only training data, DINOv3 distilled into the backbone, and pseudo-labels on COCO’s unlabelled split from an open-licensed teacher. Still no date.

Limitations

The weights are still non-commercial today. Open replacements are in training, but they are not published yet, and a faithful reimplementation of the architecture does not launder the licence on Deci’s COCO checkpoints. For commercial work right now, train from scratch.

Latency is not throughput, and the power profile dominates both. Every figure in the matrix is single-stream latency: one image at a time, waiting for each result. An async pipeline with several inference requests in flight will report higher FPS on the same hardware. Worse for anyone benchmarking casually: I first measured this laptop on the powersave profile with cores pinned near 1 GHz and got 300 ms per inference at 256px on CPU. The same code on the balanced profile gives 31 ms. A 10x swing, same machine, same commit. If you are comparing detectors on a laptop, check your power profile before you trust a single number, including mine.

The INT8 calibration images overlap the evaluation set. The INT8 files were calibrated on 128 images drawn from COCO val2017 and then scored on all 5000 images of the same split, so about 2.6% of the evaluation set also shaped the quantization ranges. That flatters the −0.4 AP, probably slightly; I have not re-run it with a calibration set taken from train2017, which is the clean version of this measurement.

Every latency figure comes from one laptop, and its numbers drift. An i7-12700H with an Iris Xe and an RTX 3060 Laptop GPU, on mains, in performance mode, but not idle: the load average sat between 2 and 5 across the three sessions. The median of 30 runs absorbs noise within a session and not drift between sessions, which is why the tables report the median of three sessions. The spread, in ms, beside the single run that the first version of this update used (same commit, same versions, everyday power mode):

YOLO-NAS-S cell	Median of 3	Min	Max	Earlier single run
OpenVINO INT8, CPU, 320	4.98	4.77	5.58	5.19
OpenVINO INT8, CPU, 640	20.34	19.46	20.65	19.87
OpenVINO INT8, iGPU, 320	6.76	5.93	6.98	6.83
OpenVINO INT8, iGPU, 640	12.55	12.48	12.62	12.04
OpenVINO FP32, CPU, 640	65.63	62.18	76.49	65.02
PyTorch FP32, CPU, 320	38.41	31.74	44.70	33.57
TensorRT FP16, dGPU, 320	0.93	0.93	1.17	1.17
TensorRT FP16, dGPU, 640	2.18	2.14	2.25	2.12

Performance mode did not make the numbers reliably faster: the everyday-mode run sits inside or next to the range almost everywhere, and the CPU legs moved most with whatever else the machine was doing. The three sessions share one set of exported files (ONNX, OpenVINO IR and the TensorRT engine were built in the first and reused), so the TensorRT spread at 320 is one engine file timed in different sessions: 20% of drift with nothing about the model changing. The TensorRT FP16 median of 0.93 ms comes from two sessions that agreed to the hundredth and one that did not, so 1071 FPS is the typical figure rather than a floor. Read gaps smaller than these ranges (the iGPU precisions at 320, or the old 256px tables against these) as noise, and the large ones (INT8 against FP32 on the CPU, TensorRT against PyTorch) as the finding. Other hardware will shift them. The RF-DETR comparison in Part 1 is from a different post with a different protocol (256×256, 500-image subset). The −2.5% versus −21% quantization gap there is large enough that I would expect the direction to hold, but it is one machine.

The tracking result is one dataset and one detector. SportsMOT is broadcast sport with a mostly steady camera, and the detector is YOLO-NAS-L on COCO weights, not fine-tuned on the sport, which misses a lot of players at this threshold. A fine-tuned detector, or pedestrian or drone footage, could move the gap between the trackers either way. Both ran with their library defaults and no tuning beyond the one threshold-matched row.

Parity was proven for the architecture, not for every code path. The bit-identical result covers a forward pass on an identical input tensor for S, M and L. Training, export, quantization and the video pipeline are not covered by that comparison.

This is one person’s reimplementation. It has tests and a parity proof, but it has not been battle-tested by a community the way super-gradients was.

Reproducibility

Parameter	Value
CPU	12th Gen Intel Core i7-12700H (20 threads)
dGPU	NVIDIA GeForce RTX 3060 Laptop
iGPU	Intel Iris Xe (Alder Lake-P)
RAM / OS	31 GB · Linux 6.8.0 (Ubuntu)
GPU driver	NVIDIA 580.173.02
Python	3.10.16 (parity, to host super-gradients) · 3.12.9 (latency and AP)
torch	2.13.0+cu130 in the parity env, identical build on both sides of the test · 2.10.0+cu130 for latency and AP
numpy	1.23.0 in the parity env (forced by super-gradients’ pin)
super-gradients	3.7.1, with setuptools==80.10.2 (pinned <81 or it will not import)
modern-yolonas	commit 3d8daba for parity; commit dbd6619 (19 Sep 2026, on main, after the 0.5.0 release) for the AP tables; f505310 (same benchmark code) for the latency tables
Model / checkpoint	yolo-nas-{s,m,l}.safetensors, HF CondadosAI/detectors snapshot edaed6b1, Super Gradients Model EULA [2]
Parity inputs	15 × torch.randn(1,3,640,640) per variant, seed 1234, CPU, eval(), no_grad()
Precision sweep	float32 and float64 at 3 inputs each, so the ratio is comparable
AP dataset	COCO val2017, all 5000 images, letterboxed, predictions mapped back to original image coordinates
AP tool	pycocotools COCOeval, bbox, confidence 0.001, NMS IoU 0.70
Commands	examples/parity_check.py --precision-sweep · examples/model_table.py --input-size {320,640} --half --nms-iou 0.70 · examples/runtime_matrix.py --models yolo_nas_s --sizes 320,640 --calibration-dir <coco>/images/val2017 --sample-image <coco>/images/val2017/000000435081.jpg, three times · examples/runtime_accuracy.py
Artifacts	output/parity.md, output/parity_summary.csv · docs/benchmarks/runtime_accuracy.json, model_table.json and docs/benchmarks/sessions-2026-09-23/session{1,2,3}.json (latency, one file per session; on the dev branch until the next release) in the companion repo
Excluded from parity	preprocessing, NMS, training, export
Latency benchmark	model inference only (no preprocessing; NMS excluded except in the Part 5 end2end comparison), batch 1, single stream, synchronous, median of 30 runs after 8 warmup runs discarded, GPU synchronised before the clock is read, on a real COCO frame with 62 annotated objects
Runtimes benchmarked	PyTorch 2.10.0+cu130 · OpenVINO 2026.3.1 · TensorRT 11.3.0.99 · ONNX Runtime 1.24.3
INT8 calibration	NNCF PTQ through yolonas export --precision int8, 128 images from COCO val2017 (see Limitations), head decode tail excluded from quantization
TensorRT INT8	not measured — TRT 11 removed the INT8 calibrator API; it needs a QDQ ONNX, which I did not build
Power and load	mains, recorded by the harness per run; performance power profile; load average 2 to 5, not an idle machine (see Limitations)
Sessions	three, 23 Sep 2026, run back to back; each table cell is the median of the three session medians, ranges in Limitations
Threshold figure	Porlamar street photo by Wilfredor, CC0, docs/assets/street.jpg in the companion repo; YoloNASDetector("yolo_nas_s", device="cpu"), conf_threshold=0.25, commit f505310
Tracking benchmark	SportsMOT val, all 45 sequences, CC BY-NC 4.0, fetched from the MCG-NJU Hugging Face repo, not redistributed · YOLO-NAS-L, COCO weights, input 640, confidence floor 0.1 · roboflow/trackers 2.6.0 library defaults · TrackEval 1.3.0 · yolonas benchmark-tracking cache / evaluate, detections cached on a RunPod A40 · artifacts docs/benchmarks/tracking.json and tracking_separability.json on the companion repo’s dev branch
Shibuya clips	Basile Morin, CC BY-SA 4.0, via Wikimedia Commons; the annotated clips are docs/assets/demo_video.mp4 and tracking_demo.mp4 in the companion repo, which document the crop, frame sampling and regeneration commands

Further reading

• Szeliski, Computer Vision, §6.3 (Object detection) [7] — the best free account of where anchor-free heads sit among the alternatives. Read this if the 8400-candidates-then-NMS design still feels arbitrary.
• The QARepVGG paper [4] — the argument for why a block can be restructured to survive INT8, which is the property the whole edge case for YOLO-NAS rests on.
• RF-DETR vs YOLO-NAS: a practical edge benchmark — where the INT8, CPU and iGPU numbers in Part 1 come from, measured end to end.
• Object detection metrics: mAP explained — if the difference between mAP@0.50 and mAP@0.50:0.95 is not second nature yet.

References

[1] Deci AI. super-gradients (PyPI release history; latest 3.7.1, 2024-04-08). PyPI release history

[2] Deci AI. Super Gradients Model EULA (YOLO-NAS weights licence). LICENSE.YOLONAS.md

[3] Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., Sun, J. RepVGG: Making VGG-style ConvNets Great Again. CVPR, 2021. arXiv:2101.03697

[4] Chu, X., Li, L., Zhang, B. Make RepVGG Greater Again: A Quantization-aware Approach. 2022. arXiv:2212.01593

[5] Li, X., Wang, W., Wu, L., Chen, S., Hu, X., Li, J., Tang, J., Yang, J. Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection. NeurIPS, 2020. arXiv:2006.04388

[6] Feng, C., Zhong, Y., Gao, Y., Scott, M. R., Huang, W. TOOD: Task-aligned One-stage Object Detection. ICCV, 2021. arXiv:2108.07755

[7] Szeliski, R. Computer Vision: Algorithms and Applications, 2nd ed. Springer, 2022. §6.3 (Object detection). szeliski.org/Book

[8] Xu, S., Wang, X., Lv, W., Chang, Q., Cui, C., Deng, K., et al. PP-YOLOE: An evolved version of YOLO. 2022. arXiv:2203.16250

[9] Gran-Henriksen, M., Lindgaard, H. A., Kiss, G., Lindseth, F. Deep HM-SORT: Enhancing Multi-Object Tracking in Sports with Deep Features, Harmonic Mean, and Expansion IOU. 2024. arXiv:2406.12081

[10] Cui, Y., Zeng, C., Zhao, X., et al. SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes. ICCV, 2023. arXiv:2304.05170

[11] Zhang, Y., Sun, P., Jiang, Y., et al. ByteTrack: Multi-Object Tracking by Associating Every Detection Box. ECCV, 2022. arXiv:2110.06864