Reproducing a published pupil-diameter model, and running it in the browser

The model and the dataset are not ours. They are the work of Vijul Shah, Ko Watanabe, Brian B. Moser and Andreas Dengel (DFKI and RPTU Kaiserslautern-Landau), published as EyeDentify [1] and PupilSense [2] and released openly. Paper, trained checkpoints, dataset and source code are all public [3]. Nothing is trained here and no architecture is proposed; that half of the credit is entirely theirs.

Everything downstream of the checkpoint is ours, and it is most of this post: the independent run over all 424,000 crops and the error analysis that comes out of it, the scale experiments, the ONNX conversion and the quantisation study behind it, the reimplementation of the input transform in JavaScript, and and the browser application at the top of this page, which is a different thing from the authors’ Streamlit app and was built from scratch to run entirely client-side.

Their work: paper · code · weights · data, both CC BY-NC 4.0. Ours: the repository for this post, Apache-2.0.

Running both released checkpoints over all 424,000 EyeDentify crops reproduces errors in the range the paper reports: 0.104 mm on one eye and 0.155 mm on the other, against a 0.25 mm floor for predicting the mean. Converting them to run in a browser costs 51 MB of assets, ~330 ms per reading, and a preprocessing port that had to match PIL’s bicubic to 1.4 × 10⁻⁴ mm. The interesting part is what the numbers say about the design: the error is a per-person offset rather than scatter, because nothing in the pipeline knows how large anything is. That points at a cheaper and steadier way to build the same thing.

The model is right here, so try it before reading about it.

Try it on your own face

This is the released checkpoint, running in this page. MediaPipe’s face landmarker [7] finds your eyes, ONNX Runtime Web [8] runs the two regressors on them, and the trace shows the last ten seconds. The models and runtimes are 51 MB and download only when you press start. No frame is uploaded anywhere; there is no server in this.

The magnified strip shows what the landmarker found: the eye contours, the iris rings, and the dashed rectangle that is the model’s entire input. Look at the crop panel beside it before believing any millimetre. Thirty-two pixels by sixteen is all the network ever sees.

Then turn a lamp on, or cover half the screen, and watch the trace follow. Responding to light is the part this model does well.

Now lean towards the camera. Your pupil has not changed and the number moves anyway. The panel underneath says why: it reports your pupil-to-pupil span and iris width in pixels, both of which grow as you approach, and neither of which the regressors receive. They get the crop and nothing else.

The oval is the cheapest fix for that, and it is the reason it is there. Line your face up inside it and your apparent scale stops drifting, so a change in the trace is a change in your pupil rather than a change in your posture. The band it enforces, 48 to 62 pixels between the pupils at 640×480, sits around the mean of the clips the model was recorded on. The rest of this post is where those millimetres come from, how far they can be trusted, and why leaning in changes them.

What the model is, exactly

There are two networks, one per eye, with separate weights and no sharing between them. Each takes an image of one eye and returns one number.

Input, as cut from the frame	one RGB crop, 32 px wide × 16 px tall, at the frame’s own resolution
Input, as the network receives it	that crop bicubic-resampled to 64 × 32, scaled to [0, 1], no normalisation. Tensor float32[1, 3, 32, 64], channels-first, named eye_crop
Inside forward	zero-padded to 224 × 224 before the ResNet stem, so ~96% of what the backbone processes is black
Output	float32[1, 1], named diameter_mm. One scalar: the pupil diameter in millimetres
Weights	11.69 M parameters per eye; 46.8 MB fp32, 11.8 MB int8
What it does not return	no confidence, no uncertainty, no way to abstain

That last row is a practical constraint rather than a curiosity. Hand the network a black frame and it answers 2.29 mm with the same composure as it answers a real eye, so anything built on it has to gate on the face detector, never on the number.

Three details about the architecture matter, because a reimplementation can get each of them wrong, still run, and still produce plausible millimetres.

The backbone is a stock torchvision ResNet-18 with its 1000-way ImageNet classifier still attached, followed by Linear(1000, 1). The obvious way to build a regressor is to replace resnet.fc, and doing that makes the released checkpoint unloadable.

There is no ImageNet normalisation anywhere in the pipeline. The crop is scaled to [0, 1] by ToTensor() and handed straight to the network. Adding the Normalize that almost every ResNet pipeline carries shifts every prediction the model makes.

And the padding happens inside the model rather than in the transform, which is easy to miss when reading the training code and impossible to notice from the outputs.

class PupilRegressor(nn.Module):
    def __init__(self, arch="ResNet18"):
        super().__init__()
        self.resnet = models.resnet18(weights=None)   # fc intact, 1000 outputs
        self.regression_head = nn.Linear(1000, 1)

    def forward(self, x):                              # x is 1x3x32x64
        h, w = x.shape[2], x.shape[3]
        pad_h, pad_w = (224 - h) // 2, (224 - w) // 2
        x = F.pad(x, (pad_w, pad_w, pad_h, pad_h), value=0)
        return self.regression_head(self.resnet(x))

Getting all three right is what makes load_state_dict(..., strict=True) succeed. That strictness is the check: a loose load would silently leave half the weights at their random initialisation and still return numbers in the 2–3 mm range.

The scenario: 51 people looking at a coloured screen

Everything below is computed on EyeDentify [1], so the running example is the dataset itself rather than an invented one. Fifty-one participants sat in front of a webcam while a web app cycled the screen through white, black, red, blue, yellow, green and grey to drive their pupils through a range of sizes. Each of the 50 sessions per person is a 3-second clip at 30 fps, recorded alongside a Tobii tracker sampling at 90 Hz. The published archive contains the eye crops rather than the faces: participants consented to release the crops, which is also why a reproduction can skip face detection entirely and feed the network exactly what it was trained on.

That gives 212,073 crops per eye, which matches the count the paper states after blink filtering.

Two participants carry the rest of this article.

	participant 4	participant 50
frames	4,407	4,335
true mean diameter	2.135 mm	2.103 mm
MAE	0.322 mm	0.043 mm
bias	+0.322 mm	−0.014 mm
correlation with the tracker	0.615	0.617

They have nearly the same pupil size and nearly the same correlation with the ground truth. One of them has seven and a half times the error of the other.

Reproducing the published number

Running both checkpoints over the whole dataset:

	MAE	RMSE	bias	r	predict-the-mean MAE	n
left eye	0.1041 mm	0.1447 mm	−0.018 mm	0.876	0.2369 mm	211,988
right eye	0.1546 mm	0.2010 mm	−0.117 mm	0.859	0.2557 mm	211,919
both	0.1293 mm	0.1751 mm	−0.067 mm	0.851	0.2465 mm	423,907

The last column is the floor. A model that ignores the image entirely and always answers the dataset mean gets 0.2465 mm of error over both eyes, because pupils do not vary much. The network beats that by 2.28× on the left eye and 1.65× on the right, with correlations of 0.88 and 0.86, so it is reading the eye rather than memorising the average.

The paper reports 0.1340 ± 0.0196 mm and 0.1403 ± 0.0328 mm for the two eyes [1]. Our numbers sit around there, and they are not the same quantity. The paper’s figure is a five-fold cross-validated test error, where each fold’s model is scored only on participants it never saw during training. What is published on Hugging Face is a single checkpoint whose training split is not documented, so an error computed over all 51 participants includes whoever it was fitted on. I looked for a bimodal split in the per-participant errors that would give the training set away, and there is not one: the distribution is continuous from 0.043 mm to 0.322 mm. So this is a reproduction of the pipeline and a measurement of the released artefact. It is not a reproduction of the paper’s cross-validation, and I cannot make it into one from what is public.

The aggregate hides the shape of the error

An MAE says how far off the model is. It does not say whether it is off in a random direction each frame, or off in the same direction every frame for one person. Those are different failures and they call for different fixes.

Splitting them apart is one subtraction. For each participant, take the mean signed error over all their frames, which is their offset, and subtract it from every frame. Whatever error remains is scatter.

Doing that for all 51 participants at once:

	MAE as measured	after one global offset	after one offset per person
left eye	0.1041 mm	0.1031 mm	0.0837 mm
right eye	0.1546 mm	0.1161 mm	0.0878 mm

The right eye loses 43% of its error to a single number per person, the left eye 20%. Once both are de-biased they are within 0.004 mm of each other, so the two checkpoints read the pupil about equally well and differ in where they start from. The median participant has 93% of their right-eye error sitting in the offset.

One caveat that matters: those offsets are fitted on the ground truth. Nobody can compute them without a tracker to calibrate against, so 0.0837 mm is not an accuracy anyone can reach in the field. It is a statement about where the error lives, which is what tells you what to do about it. Calibrate once per person against a known reference and the model becomes useful for relative change; skip that and the absolute millimetres carry a per-person offset of unknown size.

Where the offset comes from

Nothing in the pipeline knows how big anything is.

The landmark model [7] runs on the whole frame, capped at 640×480. The eye crop is then a fixed 32×16 pixel window centred on the eye-contour bounding box. It is a window, not a resize of that box: the crop is taken at the frame’s own resolution, so a person sitting closer fills more of those 512 pixels with iris, and a person further away fills fewer. The 256×256 face crop in the original application is only ever drawn on screen; it is not what the eye coordinates are measured in.

This is the scale–depth ambiguity, and it is not specific to eyes: a projected size is the physical size divided by the distance, so one image is consistent with a whole family of (size, distance) pairs. Gonzalez and Woods treat the resampling side of it in §2.4 (Image Sampling and Quantization), where Image Interpolation begins on p. 61, and again in §4.5 (Extensions to Functions of Two Variables) where Image Resampling and Interpolation on p. 283 covers what aliasing does to it [5].

Now you try

It starts on two people who cannot be told apart. A 2.4 mm pupil at 45 cm and a 4.0 mm pupil at 75 cm both project to a radius of 2.67 px, so the same photograph is consistent with either. Drag the sliders and watch how little it takes to bring any two back into agreement. Note the size of the numbers while you are there: a webcam pupil is a couple of pixels across, and that is the whole measurement.

Measuring it

If that reading is right, apparent scale alone should move the model’s answer, with the real pupil held completely still. It can be tested on a recording: rescale each frame before the landmarker sees it, and nothing changes except how many pixels the eye occupies.

Here is one clip, rescaled from 0.6× to 1.6×, with the pupil in front of the camera never moving:

frame scale	0.6	0.7	0.8	0.9	1.0	1.1	1.2	1.4	1.6
predicted diameter, mm	2.368	2.484	2.629	2.732	2.597	2.519	2.404	2.482	2.542
eye box fills window	34%	41%	47%	52%	57%	63%	69%	81%	93%

Across all eight sample clips the swing runs from 0.13 mm to 0.56 mm, which is 1.0 to 4.2 times the model’s own reported error:

clip	left eye	right eye	left peak at
All Smiles Ahead	0.496 mm	0.562 mm	1.0×
Funny Talks	0.484 mm	0.419 mm	1.6×
Infinite Blue	0.479 mm	0.224 mm	1.2×
Red Ross	0.442 mm	0.133 mm	1.2×
And it was all Yellow	0.420 mm	0.237 mm	1.4×
Focus Pocus	0.364 mm	0.346 mm	0.9×
Blink It Like Brian	0.276 mm	0.412 mm	1.2×
I like to move it move it	0.254 mm	0.231 mm	1.4×

The magnitude replicates: every clip moves by at least as much as the published error. The shape does not replicate at all. The peak lands at 0.9×, 1.0×, 1.2×, 1.4× or 1.6× depending on the clip, and seven of the eight peak somewhere in the middle of the range rather than at an end. So this is not a systematic drift with a direction; it is a model responding erratically to an input nobody is controlling.

The obvious explanation for the non-monotonicity is that the eye eventually outgrows the 32×16 window and the crop saturates on iris. The second row of the first table rules that out: even at 1.6× the eye’s landmark box only reaches 93% of the window’s width, so the whole eye is still inside it. I do not have a mechanism for the shape of these curves and I did not test one, so what follows rests on the magnitude alone.

What the measurement does support is narrower and enough: the model’s output depends strongly on a variable that nothing in the pipeline controls, at a magnitude several times its stated accuracy. A per-person offset is what that looks like on a dataset where each person sat at their own fixed distance.

Exporting it to run in a browser

The checkpoint is 46.8 MB of fp32 weights. Getting it into a page means ONNX, a runtime that can execute it there [8], and a size decision.

The zero-padding lives inside forward, so tracing captures it and the exported graph accepts the same 1×3×32×64 crop the PyTorch model does. That is deliberate: it means the JavaScript side never reimplements the padding, which is one fewer place for the two to disagree.

torch.onnx.export(
    model, (torch.zeros(1, 3, 32, 64),), "left_eye_fp32.onnx",
    input_names=["eye_crop"], output_names=["diameter_mm"], opset_version=17,
    dynamic_axes={"eye_crop": {0: "batch"}, "diameter_mm": {0: "batch"}},
)

Then the format. The obvious way to compare fp32, fp16 and int8 is to run them on the same crops and see how far the outputs drift from PyTorch. That is worth doing, because it catches an operator that was fused differently, but it does not answer the question, because an output that drifts by 0.03 mm might land on the side that was already wrong. So each format is re-scored against the Tobii ground truth on 20,000 crops drawn evenly across the 51 participants, which is the same metric the rest of this post reports.

eye	format	file	MAE	change	bias	one run in the browser
left	fp32	46.8 MB	0.1042 mm	n/a	−0.019 mm	n/a
left	fp16	23.4 MB	0.1042 mm	+0.04%	−0.020 mm	324 ms
left	int8	11.8 MB	0.1041 mm	−0.08%	+0.014 mm	177 ms
right	fp32	46.8 MB	0.1529 mm	n/a	−0.116 mm	n/a
right	fp16	23.4 MB	0.1530 mm	+0.03%	−0.116 mm	324 ms
right	int8	11.8 MB	0.1765 mm	+15.4%	−0.149 mm	177 ms

fp16 is free on both eyes. int8 is free on one and costs 15% on the other, from the same graph, the same calibration procedure and the same 512-crop calibration set.

The bias column says why. Quantisation moves each model’s bias by about 0.03 mm, and the two do not move the same way: the left eye goes from −0.019 mm to +0.014 mm, the right from −0.116 mm to −0.149 mm. Same magnitude, opposite signs, which is what you would expect from two separately trained sets of weights whose rounding error has no reason to agree. What differs in the outcome is what that shift lands on. The left eye’s offset was small enough that 0.03 mm carries it through zero and out the other side, so the absolute error stands still. The right eye’s offset was already most of its error, and the shift went the wrong way and made it bigger.

The −0.08% on the left is not an improvement. It is the same number with noise on it, and there is no mechanism by which throwing away precision makes a model better.

Which is a good argument for shipping fp16 on the right eye, and it loses anyway. In the browser the fp16 graph runs in 324 ms against int8’s 177 ms, because the wasm backend casts back to fp32 to do the work. Paying 11.6 MB of download and 147 ms per reading to recover 0.024 mm, on a page that spends two sections explaining why the absolute millimetre should not be trusted, is the wrong trade. The lab ships int8 for both eyes.

The general form of that: a quantisation format is not good or bad on its own. Score it on the metric you publish, per model, and the answer can differ between two checkpoints that came out of the same training script.

The resize is the part that breaks quietly

The crop is resampled from 32×16 to 64×32 before the network sees it, and the checkpoints were trained on PIL’s bicubic through torchvision. A browser port that scales through canvas.drawImage gets a different filter, implementation-defined and generally bilinear-ish, and feeds the network pixels it was never fitted on. So the lab carries its own resampler.

Matching PIL turned out to hinge on one detail. PIL resamples separably, horizontal pass then vertical, and for an 8-bit image it writes the intermediate back into an 8-bit buffer. Keeping that intermediate in floating point is the more accurate thing to do and it gives different pixels: it cost 23% of the exact matches until the port rounded between passes the way PIL does.

With that fixed, on real EyeDentify crops, 99.97% of pixels come out identical to 8 bits and the two paths predict the same diameter to within 1.4 × 10⁻⁴ mm.

What costs what

The obvious guess is that MediaPipe dominates, being the bigger and more famous model. On this machine in Chrome it is not close:

stage	per pass
face landmarker, 640×480 frame	25–45 ms
the two ResNet-18 regressors, int8	~330 ms

The regressors take roughly ten times the landmarker, and the reason is the padding rather than the architecture. A ResNet-18 is a small network, but this one is run at 224×224 while the crop it was given occupies 64×32. That is 24.5 times the pixels, of which 96% are the zeros the model pads with. Timed on CPU in PyTorch, the same backbone takes 59.3 ms at 224×224 and 15.6 ms at 64×32. That is 3.8 times rather than 24.5, because at this size you are paying for memory traffic and layer overhead more than arithmetic, but the direction is clear enough.

Nothing can be done about it without changing the numbers. The padded region is not inert: batch-norm shifts turn those zeros into non-zero activations, and the global average pool at the end mixes all of them into the answer. Feeding the network a 64×32 input would be 25 times cheaper and would not be the same model.

Getting it to run without freezing the page

The lab at the top of this post puts the regressors in a Web Worker, which is not a detail worth mentioning unless it changed something. It did. onnxruntime-web’s wasm backend is single-threaded unless the page is cross-origin-isolated, and single-threaded wasm runs on whatever thread calls it, so awaiting inference inside the render loop froze the video for the length of every pass and took the preview from 50 fps to 2. Enabling cross-origin isolation was the first thing measured and it bought 19%, because the work is one small model rather than something that parallelises. That is not enough to justify COOP and COEP headers on the site. Moving the work off the main thread was the fix.

What I would change

Everything above is the released model, reproduced and moved to a browser without altering it. This section is what I would do next, and it is separated from the rest because none of it is measured end to end. Where a piece is measured, the number is here.

Stop predicting millimetres; measure pixels and convert

The single design decision that causes the per-person offset is that the network is asked for a physical length from an image with no physical reference in it. That is not a solvable inference problem. It is only ever recoverable from a prior about the person, and the network’s prior is whoever it was trained on.

Splitting the job in two removes it:

1. Measure the pupil in pixels. Segmenting a dark disc inside an iris is a much easier task than regressing millimetres, and it is scale-free by construction.
2. Convert with a reference you can see. Anything of known length in the same image plane gives px/mm. A printed marker held at the cheek, a credit card (85.60 × 53.98 mm by ISO/IEC 7810 ID-1, which is why it is the usual prop), or the eye’s own iris.

The lab above already computes the third option, which is why the panel under the video reports an implied scale. The horizontal visible iris diameter averages 11.71 ± 0.42 mm across 743 eyes of 390 healthy adults [6], so it works as a ruler that is always in frame.

It is worth checking that such a ruler measures anything real before building on it. Run it over the eight upstream sample clips, eight different people recorded at apparent scales spanning 1.67×, and convert the pupil-to-pupil span it implies into millimetres:

clip	pupils apart	iris	implied separation
Focus Pocus	44.7 px	8.95 px	58.5 mm
All Smiles Ahead	45.2 px	8.93 px	59.3 mm
Funny Talks	45.6 px	8.96 px	59.6 mm
Red Ross	53.1 px	10.45 px	59.5 mm
And it was all Yellow	53.9 px	10.80 px	58.4 mm
Infinite Blue	54.6 px	10.89 px	58.7 mm
Blink It Like Brian	68.4 px	12.24 px	65.5 mm
I like to move it move it	74.9 px	14.37 px	61.0 mm

The pixel column varies by 67%. The millimetre column varies by 12%, which is the real spread of eight people’s faces rather than an artefact of where they sat. It lands between 58.4 and 65.5 mm for all eight, inside the ordinary adult range of roughly 54 to 74 mm. Eight people, four apparent scales, one conversion, and the physical answer holds. That is the property the regressor does not have.

That 1.67× spread is also a fact about the recordings. The clips are not all at one distance, and rescaling one of them from 0.9× to 1.6× moves it across 40 to 72 px between the pupils, which is most of the range the clips themselves cover. So the scale sensitivity in the previous section shows up inside the span the recordings span, not only when the model is pushed outside it.

The iris ruler has two limits worth stating. It measures a projected length, so it shrinks when the head turns and the iris foreshortens, which makes the estimate trustworthy only near frontal gaze or with a head-pose correction. And at these resolutions the ruler is nine pixels long, so a one-pixel landmark error is 11% of the scale. A physical marker of known size, larger in the frame, is the more accurate option whenever you can ask someone to hold one up.

Make it much smaller

At 11.7 M parameters and 224×224, the model is enormous relative to its job. Three directions, cheapest first:

Retrain without the padding. The single biggest waste is structural: 96% of what the backbone processes is the zeros the graph pads with. A network taking the 64×32 crop directly would be around 25× fewer pixels; measured on CPU the same backbone runs in 15.6 ms instead of 59.3 ms. That needs training, so it was out of scope here, but it is the first thing I would try.

Use a backbone sized for the input. A 32×16 crop does not need a full ResNet-18. A purpose-built convolutional stack of a few hundred thousand parameters, or a MobileNetV3-Small, would likely fit in a megabyte or two instead of 11.8 MB, which also removes the argument for quantising at all.

Know what the 51 MB is before optimising it. It splits almost evenly: 24.9 MB of onnxruntime and MediaPipe binaries against 26.2 MB of weights, of which 22.6 MB is the two regressors and 3.6 MB the landmarker. So shrinking the regressors to nothing would still leave 28 MB on the page. Getting substantially below that means dropping a runtime, not just a model: running the landmarks through a smaller custom detector, or moving to WebGPU, whose bundle is larger still but moves the convolutions off the CPU. Each of those is a trade worth measuring rather than assuming.

What I measured and rejected

Halving the model payload by using one checkpoint for both eyes looks obvious. Eyes are near mirror images, so the left model should read a flipped right eye. It does not:

what runs on the right-eye crops	MAE
the right-eye checkpoint (baseline)	0.1542 mm
the left-eye checkpoint, crop mirrored	0.2464 mm
the left-eye checkpoint, crop unmirrored	0.2242 mm

Mirroring costs 60% more error and lands close to the 0.2557 mm floor for predicting the mean. It is also worse than not mirroring, which is the opposite of what the symmetry argument predicts, and says the two checkpoints have each fitted their own offset rather than a shared notion of an eye. Measured on 5,995 crops sampled across all 51 participants.

The system we built

The lab at the top of this page is not the authors’ application. Theirs is a Streamlit app that uploads a video and processes it on a server; this one runs the whole pipeline inside the browser tab, on a live camera, with no server involved at any point. Nine pieces, in the order a frame passes through them.

1. Camera capture. getUserMedia is asked for 640×480 specifically. Upstream shrinks every frame to fit that box before doing anything else, so requesting it directly makes that step a no-op instead of a resampling nobody accounted for.

2. Face landmarks. MediaPipe’s FaceLandmarker [7] in VIDEO mode, one face, refined landmarks on so the iris ring is included. It runs on the whole frame, which is what upstream does; the 256×256 face crop in their application is only ever displayed.

3. The eye window. The bounding box of sixteen eye-contour landmarks, then a fixed 32×16 pixel window centred on it, with the same edge clamping upstream uses. Reproducing this exactly is what makes the browser numbers comparable to the Python ones, and it is also the step that throws the scale information away.

4. The input transform. A port of PIL’s bicubic resampler, described in its own section above, then [0, 1] scaling and a channels-first layout. Verified against the Python reference to 1.4 × 10⁻⁴ mm.

5. Inference, in a worker. Two int8 ONNX sessions on onnxruntime-web’s wasm backend [8], in a Worker, talking to the page over postMessage with the tensors transferred rather than copied. The worker exists for one reason, given in the section below.

6. The render/analyse split. draw repaints every animation frame from the last known landmarks. analyse runs the landmarker, cuts the crops, waits for the worker, then re-arms itself. Neither waits for the other, which is why the preview holds ~50 fps while readings arrive two or three times a second.

7. The HUD. The mirrored preview, the magnified detection strip, the two crops at pixel scale, and a panel reporting the measurements the model never receives: iris width, pupil-to-pupil span in pixels and in millimetres, and the implied px/mm.

8. The framing guide. An oval plus a pupil-to-pupil band of 48 to 62 pixels. It is the cheapest available fix for the article’s main finding: hold the scale still and the remaining variation in the trace is the pupil. While it is on, everything outside the oval is blurred, which keeps the attention on the target and keeps the reader’s room out of their own screen.

9. Asset delivery. 51 MB of weights and runtimes, fetched only after the reader presses start, stored in the site’s repository through Git LFS and resolved at build time. A page view that never starts the lab downloads none of it.

The mirroring is display-only. Every landmark, crop and millimetre comes from the unmirrored frame; only the painting is flipped, because a camera that shows you un-mirrored sends your image the wrong way when you move and makes the thing unusable.

One naming trap comes with that. Upstream calls the MediaPipe index set [33, …] LEFT_EYE, and its crops are paired with the tracker’s left_pupil column, so the checkpoint named left_eye was fitted on that pairing and this reproduction keeps it. In our capture, which is not mirrored, that same index set lands on the image-left of the frame, which is the viewer’s right eye. So the panel labelled your left eye is served by right_eye, and the labels in the lab follow the person rather than the filename.

Reproducibility

Parameter	Value
CPU	12th Gen Intel Core i7-12700H, 20 threads
GPU	NVIDIA GeForce RTX 3060 Laptop, 6 GB
RAM	31 GB
OS	Ubuntu 22.04, Linux 6.8.0-136
Key versions	torch 2.13.0+cu130, torchvision 0.28.0+cu130, onnx 1.22.0, onnxruntime 1.29.0, mediapipe 0.10.14, Pillow 12.3.0, numpy 2.4.6, pandas 3.0.5, onnxruntime-web 1.29.0, @mediapipe/tasks-vision 1.0.1
Model	PupilSense ResNet-18, left_eye.pt / right_eye.pt from the vijulshah/pupilsense Space repo, CC BY-NC 4.0. sha256 98fb2c78…7dd894e6 and 68e2928f…09337668, fetched 24 Aug 2026
Data	EyeDentify, Kaggle vijuls/PupilDiameterDatasets, CC BY-NC 4.0. 51 participants, 212,073 crops per eye, Tobii ground truth in session_data.csv. EyeDentify++ not used
Commands	pupil fetch-weights · pupil prepare-data · pupil predict --eye {left,right}_eye · pupil evaluate · pupil export · python scripts/scale_sensitivity.py · python scripts/scale_reference.py · python scripts/quantisation_cost.py
Runs	Single deterministic pass over every crop; no sampling, no warmup, nothing timed
Excluded	239 frames whose Tobii reading is empty, dropped and counted
Quantisation	ONNX QDQ static, per-channel int8 weights, uint8 activations, calibrated on 512 crops drawn evenly across all 51 participants; error scored on 256 further crops held out from calibration

Every millimetre in this post comes from one of four saved artifacts in the companion repo: output/summary.json, output/per_participant_{left,right}_eye.csv, output/quantisation_cost.json, output/scale_reference.json and output/scale_*.csv.

Four numbers are not in those files, because they measure code rather than the model, and each has a command that regenerates it instead. The agreement between the JavaScript and Python input transforms (99.97% of pixels, 1.4 × 10⁻⁴ mm) comes from scripts/verify_js_transform.py. The browser figures (177 ms for int8, 324 ms for fp16, the 19% that cross-origin isolation bought and the drop from 50 fps to 2 when inference ran on the main thread) were measured on this machine in Chrome 151 on the site’s own /labs/pupil-selftest page and in the lab itself. Browser timings move with the machine and the browser; treat them as the reason for a design decision, not as a benchmark.

Limitations & caveats

• This is not the paper’s number. The published MAE is five-fold cross-validated on held-out participants; the released checkpoint has an undocumented training split. The comparison is honest only as “the released artefact scores this on the released data”.
• The de-biased figures are an oracle. Per-person offsets are fitted on the Tobii ground truth. They describe the error, they are not an accuracy anyone can obtain without a reference instrument.
• The scale experiment rescales frames, it does not move a person. Someone actually leaning in also changes perspective, focus and sensor noise. The measurement isolates apparent size deliberately, so it reports that term and nothing else.
• I have no mechanism for the non-monotonic response to scale. I ruled out the crop saturating on iris; I did not test anything else.
• The lab’s millimetres are uncalibrated for you. Different camera, different distance, different eye. Watch the number change, not its value.
• Both the weights and the dataset are non-commercial. CC BY-NC 4.0 on each. An Apache-2.0 licence on the code that loads them does not lift that.
• The blink gate in the lab is partial. Upstream falls back to a ViT classifier in the ambiguous eye-aspect-ratio band between 0.22 and 0.25; the lab reports that band as uncertain instead of shipping a second model.
• Everything here is one architecture. ResNet-50 checkpoints are published too and I did not run them.
• Nothing in “What I would change” is measured end to end. The mirror experiment is, and the padded-versus-unpadded timing is; the iris-as-ruler pipeline and the smaller backbones are reasoning from the numbers above, not results. Do not read that section as a benchmark.
• The iris constant is a population mean, not your iris. 11.71 ± 0.42 mm, measured on healthy white adults aged 10–80 [6]. It also foreshortens off-axis, and at these resolutions the ruler is about nine pixels long.

Takeaways

• The released checkpoints reproduce sensible errors on the released data: 0.104 mm and 0.155 mm, against floors of 0.237 mm and 0.256 mm for predicting each eye’s mean. That is checkable because the authors published paper, code, weights and data [3], which is rarer than it should be.
• A published model becomes a browser demo for 51 MB and a few hundred milliseconds, and most of both is runtime rather than weights.
• Porting a preprocessing step is where a reproduction leaks accuracy quietly. The bicubic here had to match PIL down to the 8-bit intermediate between passes.
• Pick a quantisation format per model, on the metric you publish. int8 cost one eye nothing and the other 15.4%, and fp16 lost anyway on latency.
• The error is a per-person offset rather than scatter, because nothing in the pipeline knows how large anything is. Use the model for relative change at a fixed distance, and do not read the absolute millimetres without calibrating that person first.
• If you were building this from scratch: measure the pupil in pixels and convert with a reference you can see. It removes the scale confound instead of learning around it, and it is far cheaper than a ResNet-18 evaluated over 96% padding.

Further reading

• Go deeper on resampling: Gonzalez & Woods, Digital Image Processing (4th ed.), §2.4 for interpolation itself and §4.5 for what aliasing does to it. The theory behind why a 32×16 crop is a lossy measurement instrument [5].
• The follow-up from the same group: Webcam-based Pupil Diameter Prediction Benefits from Upscaling [4] asks whether super-resolving the crop before the regressor helps, which is the natural next question after seeing how small the input is.
• The original work, in full: the authors’ repository [3] carries the training code, the dataset construction and the Streamlit application, which is far more than the slice reproduced here.
• Related on CondadosAI: the image as data for what a pixel value is before any model sees it.

References

The first three entries are the work this post reproduces. Paper, code, weights and data were all released by its authors, which is the only reason a reproduction like this is possible.

[1] Shah, V., Watanabe, K., Moser, B. B., & Dengel, A. (2024). EyeDentify: A Dataset for Pupil Diameter Estimation based on Webcam Images. arXiv:2407.11204v1. Mind the version: v2 of the same arXiv identifier is a different paper, [2] below. The MAE quoted here (0.1340 ± 0.0196 and 0.1403 ± 0.0328 mm) is v1’s. Project page: vijulshah.github.io/eyedentify.

[2] Shah, V., Watanabe, K., Moser, B. B., & Dengel, A. (2025). PupilSense: A Novel Application for Webcam-Based Pupil Diameter Estimation. ETRA 2025 (Symposium on Eye Tracking Research and Applications). doi:10.1145/3715669.3723125. The application paper, and the origin of the Streamlit app this post ports.

[3] Shah, V., et al. Released artefacts, all CC BY-NC 4.0: source code at vijulshah/webcam-based-pupil-diameter-estimation, the ResNet-18 and ResNet-50 checkpoints inside the PupilSense Space under pre_trained_models/, and the EyeDentify and EyeDentify++ datasets on Kaggle.

[4] Shah, V., Moser, B. B., Watanabe, K., & Dengel, A. (2025). Webcam-based Pupil Diameter Prediction Benefits from Upscaling. ICAART 2025. arXiv:2408.10397.

[5] Gonzalez, R. C., & Woods, R. E. (2018). Digital Image Processing (4th ed.), §2.4 (Image Sampling and Quantization), Image Interpolation p. 61; §4.5 (Extensions to Functions of Two Variables), Image Resampling and Interpolation p. 283. Pearson. Detailed table of contents.

[6] Rüfer, F., Schröder, A., & Erb, C. (2005). White-to-white corneal diameter: normal values in healthy humans obtained with the Orbscan II topography system. Cornea, 24(3), 259–261. doi:10.1097/01.ico.0000148312.01805.53. Mean 11.71 ± 0.42 mm over 743 eyes of 390 subjects aged 10–80. The population was healthy white adults, so treat the constant as approximate outside it.

[7] Google. MediaPipe Face Landmarker. MediaPipe Solutions, tasks-vision 1.0.1. Docs.

[8] Microsoft. ONNX Runtime Web. Version 1.29.0. Docs.