benchmark
8 posts · browse all tags
Models & Detection A Traffic Camera in a Simulated New York: Exact Labels at 0.75 s a Frame
A pole camera at a real Harlem junction, rendered by the Valdrada simulator on one desktop GPU: every car and pedestrian comes with its box, its mask, its depth and how much of it is hidden, at 0.75 s per labelled frame when four cameras share a capture. What it took to run headless on Linux, what a simulated label does and does not tell you, how far a COCO detector gets on it, and a recording format that lets the same code read a real camera later.
Edge Deployment EmbeddingGemma 2 Video Search on a Laptop iGPU
Google's open EmbeddingGemma 2 finds the right clip for a sentence 22 times out of 23, fully offline, at 2.3 s per clip on the Intel Iris Xe inside a laptop with OpenVINO INT8. Then the same vectors are asked which way time runs: a clip and its reverse land at cosine 0.997, and neither EmbeddingGemma 2 nor VideoPrism, a model built for video, can tell them apart.
Edge Deployment 3D Vehicle Detection on a Laptop iGPU
UrbanOmniDetect-2 draws a 3D box around every vehicle from one uncalibrated camera and places it on a top-down map of the road. On the Intel Iris Xe inside a 12th-gen laptop, quantized to INT8 with OpenVINO and with its final layer kept in float, the medium model runs at 33 ms per frame (30 fps), and its box corners stay within 0.4 px of the original model on two traffic cameras.
Evaluation How many good parts does it take to beat a System One AI model? Between one and sixteen, on VisA
System One models, the idea TypeSafe launched with Jev in September 2026, answer a question with a decision in a single pass. Jev-Omni, an open one, and the untouched Gemma 4 12B it was built from inspect all 2,162 VisA test images without seeing a good part. PatchCore, the standard industrial anomaly detector, is given k good parts per product: at 256 pixels it passes Jev-Omni at k = 8 and Gemma 4 at k = 16, at 512 pixels at k = 1 and k = 4. Also: the base model beats its System One fine-tune, and a day-0 demo on a lime line.
Edge Deployment Marigold V2 at four bits: what quantizing a 20-billion-parameter depth model costs
Marigold V2 ships as a 17 GB inference job, but the released checkpoint is a rank-128 LoRA over a frozen 20.4 B backbone, so the backbone is swappable. Four configurations on one rented A40: going 4-bit costs 10% of the time and saves 65% of the memory, picking GGUF over bitsandbytes costs another 1.87x and saves nothing, and streaming the weights from host RAM costs 3.33x.
Edge Deployment Build a computer-vision plugin for OBS Studio, and measure what every frame costs
A step-by-step OBS Studio video filter in C++ on OpenVINO, split so the model swaps without touching the plumbing, and measured stage by stage: getting pixels off the GPU, where inference has to live so OBS never drops a frame, and what the trip from camera to virtual camera costs.
Edge Deployment RF-DETR vs YOLO-NAS: A Practical Benchmark for Edge Deployment: CPU, GPU, and Intel iGPU Compared
RF-DETR Nano vs YOLO-NAS-S on COCO across CPU, CUDA GPU, and Intel Iris Xe iGPU, across two resolutions, OpenVINO FP32/FP16/INT8, and a custom fine-tune, all under one consistent evaluation. RF-DETR wins accuracy; YOLO-NAS wins latency, efficiency, and INT8.
Models & Detection CNN vs Transformer, quantized: YOLO26-seg and RF-DETR-Seg race for instance masks on an Intel iGPU
Two opposite small segmentation models, a 2026-era CNN and a DETR transformer, both quantized to INT8 with NNCF and run on a laptop Intel iGPU. YOLO26-seg's forward pass is ~7.4× faster; RF-DETR-Seg keeps its masks more faithful under INT8 on the CPU, and its fully-quantized mask head breaks on the iGPU. The whole shootout, numbers first.