How to build a rep counter that does not count your rest

Counting repetitions from a pose model is four stages and 125 lines of logic. Getting it right is four decisions, and this post makes each one with a measurement rather than a preference. The two that matter most are free: read the joint angle from world landmarks, because the projected one moves 43.6 degrees while the elbow holds still, and use two thresholds instead of one, because a single threshold turned a clip of six push-ups into a count of twenty. The filter everyone argues about is third in size, and the one that wins is not the usual choice.

The naive version is short enough to write from memory. Take a pose model, compute the elbow angle, count the times it crosses a line:

def naive_count(angles, threshold=110.0):
    reps, phase = 0, "up"
    for angle in angles:
        if angle < threshold and phase == "up":
            phase = "down"
        elif angle > threshold and phase == "down":
            phase = "up"
            reps += 1
    return reps

It works on the first video you try. Then the complaints arrive: counts that climb while the user is resting, half repetitions registering as whole ones, the same repetition counted twice. This post builds the version that does not do that, one stage at a time.

What you are building

The finished thing, running here on your camera. The landmarker, the filter and the counter all run in the tab and no frame leaves your machine.

The second counter is the point of it. It reads the identical filtered angle, frame for frame, through the naive rule above. Do slow controlled repetitions and the two agree. Then stop with your arm halfway and hold it there, or do half repetitions, and watch where they part.

Everything below is how it got there. The code is JavaScript where it runs in the browser and Python where the measurements were made; both live in the companion repository.

Every number here is measured on one clip, named once so it can be checked: Interval Push-ups by Taco Fleur, CC BY-SA 4.0 on Wikimedia Commons [7]. 51.4 s at 640×480 and 30 fps, a backlit silhouette in profile, six push-ups with a plank held between them. Six is the hand-counted truth, established from the video before any of this was run, because the pipeline under test cannot be the source of its own ground truth.

Step 1: landmarks, and the two settings that matter

MediaPipe’s Pose Landmarker gives 33 body points per frame [1], from the BlazePose family [6]. In the browser:

import { FilesetResolver, PoseLandmarker } from '@mediapipe/tasks-vision';

const CDN = 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@1.0.1';
const MODEL = 'https://storage.googleapis.com/mediapipe-models/pose_landmarker/'
            + 'pose_landmarker_lite/float16/latest/pose_landmarker_lite.task';

const fileset = await FilesetResolver.forVisionTasks(`${CDN}/wasm`);
const landmarker = await PoseLandmarker.createFromOptions(fileset, {
  baseOptions: { modelAssetPath: MODEL, delegate: 'GPU' },
  runningMode: 'VIDEO',       // IMAGE would re-detect from scratch every frame
  numPoses: 1,
});

Two things about that call are worth knowing before you write the rest.

The GPU delegate is not guaranteed, and asking for it fails loudly. createFromOptions rejects when the delegate is missing instead of falling back, so a page that only asks for GPU does not start at all under a software renderer or on a locked-down machine. Try both:

let landmarker = null;
for (const delegate of ['GPU', 'CPU']) {
  try {
    landmarker = await PoseLandmarker.createFromOptions(fileset, {
      baseOptions: { modelAssetPath: MODEL, delegate },
      runningMode: 'VIDEO', numPoses: 1,
    });
    break;
  } catch (err) {
    if (delegate === 'CPU') throw err;      // nothing left to fall back to
  }
}

VIDEO mode needs strictly increasing timestamps in milliseconds, and it is the caller’s job to supply them. A repeat or a step backwards breaks the internal tracking. Two frames delivered inside the same millisecond is enough to do it:

let lastStamp = -1;

function detect(video) {
  let stamp = Math.round(performance.now());
  if (stamp <= lastStamp) stamp = lastStamp + 1;   // the guard that gets forgotten
  lastStamp = stamp;
  return landmarker.detectForVideo(video, stamp);
}

Offline, over a recorded clip, take the timestamp from the frame index and the declared frame rate. Never from the wall clock: a wall-clock stamp on a frame that arrived late tells the tracker the subject moved further than they did.

timestamp_ms = int(round(1000.0 * frame_index / fps))
result = landmarker.detect_for_video(mp_image, timestamp_ms)

There is no smoothing option to turn on. The whole of PoseLandmarkerOptions is the running mode, the number of poses, three confidence thresholds, the segmentation-mask switch and a result callback. In Python those names read running_mode, num_poses, min_pose_detection_confidence, min_pose_presence_confidence, min_tracking_confidence, output_segmentation_masks and result_callback; the JavaScript names are the same list in camel case. The smooth_landmarks flag people remember belongs to the retired mp.solutions.pose API. Anything built on the current one filters its own signal or does not filter.

How much jitter is there before you do anything

Worth knowing early, because it sets a floor on what any filter can achieve. Decode one frame, then hand the landmarker that same array of pixels sixty times.

In IMAGE mode the landmarks come back bit for bit identical, across all three published model sizes. In VIDEO mode they do not.

model	IMAGE mode	VIDEO mode, identical pixels
lite	0.0000 px	0.418 px RMS (max 0.995)
full	0.0000 px	0.340 px RMS (max 0.650)
heavy	0.0000 px	0.207 px RMS (max 0.536)

Per-landmark RMS deviation about each landmark’s own mean, 60 repeats of one 640 px frame. Nine conditions in total, three model sizes across three resolutions, in output/tracker_self_jitter.json.

A photograph, fed in repeatedly, produces landmarks that wander by a third of a pixel. Nothing about the scene changed, so this is not the sensor, the lighting or the encoder. VIDEO mode crops each frame from the region of interest it derived from the previous one, so its own output is part of its next input and the loop never settles. That is the design that makes the mode fast enough to run at frame rate, and the cost is usually invisible.

The practical consequence: there is a floor no camera, lens or light gets below. The heavy model sits 2.0 times lower than the lite one (0.418 against 0.207 in the table above), so capacity moves the floor and does not remove it.

Step 2: from 33 points to one number

The counter reads one scalar per frame, the joint angle. Filtering that instead of all 33 landmarks is a thirty-third of the arithmetic, and a threshold in degrees means the same thing whether the subject is at arm’s length or across the room, which is not true of any threshold in pixels.

The angle at b in the chain a-b-c is the usual dot product:

$\theta = \arccos\!\left(\frac{(a-b)\cdot(c-b)}{\lVert a-b\rVert\,\lVert c-b\rVert}\right)$

function angleAt(a, b, c, useZ) {
  const u = [a[0] - b[0], a[1] - b[1], useZ ? a[2] - b[2] : 0];
  const v = [c[0] - b[0], c[1] - b[1], useZ ? c[2] - b[2] : 0];
  const nu = Math.hypot(...u), nv = Math.hypot(...v);
  if (nu === 0 || nv === 0) return NaN;          // a leg with no length
  const cos = (u[0]*v[0] + u[1]*v[1] + u[2]*v[2]) / (nu * nv);
  return Math.acos(Math.max(-1, Math.min(1, cos))) * 180 / Math.PI;
}

// BlazePose 33-point chains. Verified against the documentation, not recalled.
const ELBOW = { left: [11, 13, 15], right: [12, 14, 16] };
const KNEE  = { left: [23, 25, 27], right: [24, 26, 28] };

Two traps live in the three lines that feed it.

Normalized landmarks are x / width and y / height. On a 16:9 frame one unit of x is 1.78 times as many pixels as one unit of y, so an angle taken straight from the normalized values is the angle of a horizontally stretched picture of the arm. Multiply back into pixels first.

The z you get from landmarks is not metric. MediaPipe returns a second set, worldLandmarks, which is in metres and centred on the hips [5]. Use those, and the reason is the next section.

Why the angle comes from world landmarks

An angle computed on image coordinates is the angle of the limb’s shadow on the sensor. Turn the person and the shadow changes shape while the elbow holds still.

This can be checked exactly. Take a real skeleton from a real frame, rotate the whole thing about its own spine as a rigid body, and reproject it through a pinhole. Rotating a rigid body cannot change any angle inside it, so the world angle is constant by construction and every degree the projected angle moves is projection.

elbow held at	projected at −45°	at 0°	at +45°	full swing over 120°
45°	39.9°	9.5°	25.4°	43.6°
90°	101.7°	92.1°	117.7°	41.5°
135°	143.4°	136.6°	155.2°	28.2°
170°	174.1°	174.6°	167.2°	13.3°

The empirical half of this corrected the hypothesis, and the correction is the useful part. Binning every frame of every clip by how far the shoulders are turned from square put the worst disagreement at the smallest turn, which is backwards. A push-up filmed from the side has the torso fully turned and the arm swinging in a plane parallel to the sensor, so the projection is faithful. What distorts a projected angle is how much of the limb points at the camera, not how far the person turned.

Re-binned on the arm’s own tilt out of the image plane, over 23,059 frames (output/angle_2d_vs_3d.json):

predictor	rank correlation with the 2D-versus-3D disagreement
the arm’s tilt out of the image plane	+0.376
torso yaw, the first guess	−0.393

The tilt correlation is positive inside every one of the seven clips taken separately, from +0.14 to +0.80, so it is not an artefact of which clip supplied which frames.

Step 3: decide whether to filter, and which

This is the stage everyone argues about, so it is worth doing in order: look at the noise, work out what a filter costs, then pick one.

What the noise looks like

Filters are usually chosen against white noise, because white noise is what is easy to generate. Every still segment across three clips, meaning every stretch where the limb is locked out and the angle is flat, gives 232 seconds of residual once a straight line is removed from each segment. Its autocorrelation:

Correlated by 0.694 from one frame to the next, and still outside the white-noise band at five frames (output/noise_profile.json). The memory runs out somewhere around a quarter of a second.

This matters for a practical reason. A low-pass filter separates signal from noise by frequency, and noise with that much memory has most of its energy at low frequencies, which is where the movement lives too. The same filter that removes white noise cheaply has to eat into the signal to remove this.

The size of the effect is measurable. Take the same residual samples and shuffle them: the distribution is untouched and the memory is gone, which is the signal a white-noise benchmark assumes. Across 65 filter configurations, filters leave 1.78 times more jitter behind on the real residual than on the shuffled version, with a range of 1.07 to 2.21 (output/filter_sweep.json). A benchmark run on simulated white noise reports a filter that is 78 percent better than the one you deploy.

What an exponential moving average costs

The filter people reach for first is one line:

$y[n] = \alpha\,x[n] + (1-\alpha)\,y[n-1]$

Each output is a blend of the newest reading and where the filter already was, so a small $\alpha$ means the filter mostly ignores the newest reading. That buys smoothness by being slow to believe anything, which is the same thing as being late.

There are two ways to quantify how late. Across the settings swept here they differ by a factor of 2.4 to 3.3, which matters because people quote one while meaning the other. The group delay at DC is the mean delay of the impulse response [2]:

$\tau_{\text{group}} = \frac{1-\alpha}{\alpha}\ \text{samples}$

The step response is $1-(1-\alpha)^n$, so the time to arrive at 90 percent of a change is:

$n_{90} = \frac{\ln(0.1)}{\ln(1-\alpha)}\ \text{samples}$

The comparison, and the winner

The 1 euro filter of Casiez, Roussel and Vogel [3] raises its cutoff with the estimated speed of the signal: heavy smoothing while the arm is still, and it gets out of the way while the arm moves.

Both axes were measured on the condition where each one matters, and neither was simulated. Jitter is the RMS of each filter’s output over the real residual above, where the true value is a constant so anything the filter emits is error. Lag is measured against a zero-phase reference, the same low-pass run forwards and then backwards so its phase cancels, on a real descent from the push-up clip.

The first version of the lag measurement used an ideal step and had to be thrown out. A step is 120 degrees inside one frame, a speed-adaptive filter opens its cutoff all the way for it, and the step reports 2.53 times the lag of a real descent (step_vs_movement_lag_ratio in output/filter_sweep.json). That would have credited the 1 euro filter with a movement no arm can make.

lag budget	best EMA	best 1 euro	best Kalman	1 euro advantage
50 ms	1.710°	1.316°	1.708°	23.0%
100 ms	1.464°	0.819°	1.499°	44.1%
200 ms	1.192°	0.819°	1.245°	31.3%
400 ms	0.863°	0.819°	0.770°	5.1%

Unfiltered residual: 2.206° RMS. Best settings under a 120 ms budget: EMA $\alpha = 0.25$, 1 euro with min cutoff 0.05 Hz and $\beta = 0.005$, Kalman with $q = 100$ and $r = 1$. Full sweep in output/filter_sweep.csv.

Three readings, in order of how useful they are.

The 1 euro filter wins where a real-time app has to live. At a 100 ms budget it leaves 44 percent less jitter than the best EMA, and the gap closes to 5 percent by 400 ms. My own line is that a counter reporting a repetition 400 ms after it happened is not one I would ship, which puts the region where the two filters tie outside the part of this figure I would use.

The constant-velocity Kalman filter tracks the EMA closely enough to make its extra state hard to justify. Below 300 ms it never beats the EMA by more than 1.8 percent and it is 4.5 percent worse at a 200 ms budget. It does pull ahead at 400 ms, by 10.8 percent, in the same region where the EMA and the 1 euro filter converge and where the delay is long enough to notice. Forsyth and Ponce set out the constant-velocity model and where it is appropriate [4]; a repetition reverses its angular rate at the top and the bottom of every rep, which is where a constant-velocity model is at its worst. One clip is not a proof of mechanism, so that connection is the likely reason rather than a demonstrated one.

The dashed lines matter for anyone reading a filter comparison elsewhere. They sit far below the solid ones because they answer an easier question.

The filter, in code

Two details a from-memory rewrite tends to lose, both of which still produce a plausible-looking output: the derivative is taken against the previous filtered value, and the sampling frequency is re-estimated from consecutive timestamps so the filter behaves under a dropped frame.

class OneEuroFilter {
  constructor(minCutoff = 0.05, beta = 0.005, freq = 30, dCutoff = 1) {
    Object.assign(this, { minCutoff, beta, freq, dCutoff });
    this.x = null; this.dx = 0; this.tPrev = null;
  }
  alpha(cutoff) {
    const te = 1 / this.freq;
    return 1 / (1 + (1 / (2 * Math.PI * cutoff)) / te);
  }
  filter(value, t) {
    if (this.tPrev !== null && t > this.tPrev) this.freq = 1 / (t - this.tPrev);
    this.tPrev = t;
    const rate = this.x === null ? 0 : (value - this.x) * this.freq;
    const a = this.alpha(this.dCutoff);
    this.dx = a * rate + (1 - a) * this.dx;
    const b = this.alpha(this.minCutoff + this.beta * Math.abs(this.dx));
    this.x = this.x === null ? value : b * value + (1 - b) * this.x;
    return this.x;
  }
}

Check any port you write. The authors publish a groundTruth.csv for this purpose, and the Python side of the companion repository replays it: agreement to 4 × 10⁻⁶, where the residual is the CSV’s six-significant-figure printing accumulating through a recursive filter.

The tuning order is the authors’: set beta to zero and lower the cutoff until slow-motion jitter is gone, then raise beta until fast motion stops lagging.

Step 4: the state machine

This is where counting is won or lost, and it is the cheapest stage to get right.

class RepCounter {
  constructor({ flexBelow = 80, extendAbove = 150, confirmS = 0.1,
                minVisibility = 0.5 } = {}) {
    Object.assign(this, { flexBelow, extendAbove, confirmS, minVisibility });
    this.phase = 'extended';
    this.count = 0;
    this.candidate = null;
    this.candidateSince = 0;
  }

  /** Feed one frame. Returns true if a repetition completed on it. */
  update(angle, visibility, t) {
    if (!Number.isFinite(angle)) return false;
    if (visibility < this.minVisibility) return false;   // hold through an occlusion

    let wants = null;
    if (this.phase === 'extended' && angle < this.flexBelow) wants = 'flexed';
    else if (this.phase === 'flexed' && angle > this.extendAbove) wants = 'extended';

    // Confirmation, not lockout: the angle has to stay past the line.
    if (wants === null || wants !== this.candidate) {
      this.candidate = wants;
      this.candidateSince = t;
      return false;
    }
    if (t - this.candidateSince < this.confirmS) return false;
    this.candidate = null;

    if (wants === 'flexed') { this.phase = 'flexed'; return false; }
    this.phase = 'extended';
    this.count += 1;
    return true;                            // the only edge that counts a rep
  }
}

That is the shape a reader implements first, and it is what the widget above runs. The companion repository factors the same machine differently: the two phases and the conditions that move between them become a table of rules, and the guards sit outside it. Splitting it that way makes this transition counts a repetition a stated fact instead of something inferred from where the increment happens to sit, and adding a third phase becomes adding a row. The behaviour is identical, and a cross-language test replays both against the Python to keep it that way.

Four rules are doing the work, and they are not equally important.

Two thresholds instead of one. Entering the flexed phase needs the angle below 80 degrees; leaving it needs the angle above 150. Noise has to be larger than the gap to move the machine at all.

Both bounds, in that order. A repetition is counted on the flexed to extended transition, so a movement that never gets below the lower bound never enters the flexed phase and has no transition to complete. That is what rejects a half repetition, and it is a consequence of the first rule instead of an extra one.

A time requirement on a phase change. There are two of these in the wild and they do not do the same job. A lockout refuses transitions for a while after the last one; at the instant a spike arrives, the machine has been in its current phase for far longer than the lockout, so the spike flips the phase unopposed and the lockout only delays the return. A confirmation instead requires the angle to stay past the threshold before the phase changes at all, which is what a spike cannot do. The tests in the companion repository pin the difference, because an ablation table that conflates the two credits the wrong rule.

A confidence gate. Frames where the weakest landmark in the chain is below a visibility cut are dropped and the phase is held. An angle measured during an occlusion is not a measurement.

What the first rule is worth

A counter with a single threshold is not wrong everywhere, which is why it passes review and ships. On the push-up clip, with the threshold at 115 degrees, halfway down a 120 degree swing, it reports six out of six. So does every other configuration tested. Correct execution does not break a rep counter.

The clip contains the failing condition anyway, in its gaps. Between repetitions the subject holds a plank with the arm locked out. Taking that strictly, above 160 degrees, leaves 4.0 seconds of this clip, resting at a mean of 162.5 degrees with 3.48 degrees RMS around it (output/threshold_placement.json). The looser criterion used for the noise profile earlier keeps 40.5 seconds of the same clip and gives 1.57 degrees (output/noise_profile.json), so the figure here describes the top of the range instead of the clip as a whole.

Now walk a single threshold through that resting band.

threshold	crossings from noise	single threshold	hysteresis	truth
158°	39	20	3	6
160°	41	21	2	6
162°	21	11	2	6
164°	5	3	1	6
115°	12	6	6	6

output/threshold_placement.json. The hysteresis column pairs each threshold with a lower bound of 80°, so at 158° and above it is also badly placed and undercounts.

Twenty reported against a truth of six, from 39 crossings of a line the signal is sitting on.

The rule that follows: place the thresholds where the signal does not rest, and put as much distance between them as the movement allows. Tuning a threshold to “roughly where the arm is at the top” puts it in the worst possible place.

Step 5: before you ship

Five checks, each of which catches something this post measured.

1. Feed the landmarker one frozen frame and watch the output. If it moves, you have found your jitter floor, and no amount of lighting work goes below it.
2. Log the timestamps you pass to detectForVideo. A repeat or a step backwards means the tracker is degrading and nothing downstream will tell you.
3. Hold a limb still on the threshold for thirty seconds. The count must not move. This is the single most informative test in the list and it takes half a minute.
4. Do half repetitions on purpose. They must not count. If they do, the machine is triggering on one bound instead of two.
5. Turn side-on while holding a joint at a fixed angle. If the reported angle moves, you are reading image coordinates somewhere.

The whole pipeline

Ranked by size, on this clip: a badly placed threshold costs 14 repetitions, a projected angle costs up to 43.6 degrees of apparent movement, the filter choice costs 0.645 degrees of residual jitter at a fixed lag budget, and the landmarker’s own floor costs a third of a pixel. The two at the top are free to fix. The one that gets the attention is third.

Reproducibility

CPU	12th Gen Intel Core i7-12700H, 20 threads
GPU	None. delegate is left unset in the Python Tasks API and the runs log TensorFlow Lite’s XNNPACK CPU delegate. The browser widget asks for GPU, falls back to CPU, and reports which one it got
RAM and OS	31 GB, Ubuntu 22.04.5 LTS, kernel 6.8.0-136
Python	3.13.5
Libraries	mediapipe 1.0.1, numpy 2.5.2, pandas 3.0.5, opencv-python-headless 5.0.0, matplotlib 3.11.1
Model	MediaPipe Pose Landmarker, float16 bundles from Google’s published latest path. full unless stated; the three-way table names its own. Digests recorded per run in models/*.sha256
Landmarker settings	RunningMode.VIDEO, num_poses=1, detection, presence and tracking confidence all 0.5. Timestamps derived from the frame index and the declared frame rate, never the wall clock
Data	interval_pushups, Interval Push-ups by Taco Fleur, CC BY-SA 4.0, Wikimedia Commons: 1,543 frames, 640×480, 30 fps, 6 hand-counted push-ups. The pooled noise measurement adds kb_strict_press (Taco Fleur, CC BY-SA 4.0) and db_shoulder_press (Colossus Fitness, CC BY 3.0). Full attribution table in data/clips/SOURCES.md, generated from the Commons API
Runs and warmup	The self-jitter table is 60 repeats per condition over 9 conditions. Filter jitter discards the first second of each run as convergence. Everything else is a deterministic replay of a saved landmark track, so a configuration gives the same answer every time
Exclusions	The still segments feeding the noise and filter figures are selected on flatness, not on visibility, and frames with no detected pose are dropped throughout. The 0.7 visibility cut applies only to the 2D-versus-3D comparison, where a badly seen limb would otherwise be counted as a projection effect
Commands	poserep extract, then scripts/noise_profile.py, filter_sweep.py, threshold_placement.py, angle_2d_vs_3d.py, tracker_self_jitter.py
Artifacts	Every number above comes from a JSON or CSV in output/, named in the caption where it is used. The clips are not redistributed; scripts/fetch_clips.py downloads them
Notebook	notebooks/pose_rep_counting.ipynb has a cell for every Python snippet printed above

Limitations

One movement, one camera, a handful of clips. Push-ups in profile, plus two press clips for the pooled noise. Nothing here says how the numbers move for a squat at four metres, where the landmarks are worse to begin with.

The counting comparison has one labelled clip. Six repetitions, hand-counted twice. The threshold table is a walk across one clip’s resting band, and the shape of that band is a property of that clip. The direction of the effect follows from the mechanism; the size of it is one measurement.

Three of the conditions people complain about are missing. A subject deliberately holding still for half a minute, a set that mixes full and partial repetitions, and a limb held at a fixed angle while the torso rotates. These are apparatus, not exercise content, so found video does not contain them. The still condition is substituted by the plank holds and by the frozen-frame experiment. The partial-repetition case is argued from the state machine’s structure and demonstrated in the tests, and it is not measured on video here.

The rotation experiment is geometry plus a correlation. The rigid-body half is exact and proves what projection does. It does not prove that MediaPipe’s estimated world landmarks stay stable through a real turn. The clip-based correlation of +0.376 supports it and is not the same claim.

The pooled tilt bins cannot separate tilt from clip. The bin table rises from 6.8° to 50.9° of disagreement across 0 to 60 degrees of tilt and then drops (median_disagreement_by_arm_tilt in output/angle_2d_vs_3d.json). That drop is not reported as a reversal: the top bin is 89 percent one clip. The obvious explanation for it, an arm so foreshortened that both segments collapse together, was checked and is wrong, because none of those frames has an elbow past 150 degrees. No mechanism is claimed.

The Kalman result is one parameterisation family. A constant-velocity model with a swept process and measurement noise. A constant-acceleration model, or one tuned per phase of the movement, was not tried.

The widget runs a different model from the measurements. It loads the lite bundle so the download stays small, while every number in the post was measured on full. From the frozen-frame table that costs it 0.08 px more jitter, 0.418 against 0.340. It also pulls the landmarker from a CDN, so it needs a network connection the first time and will not start without one.

Takeaways

Read joint angles from world landmarks. The projected version moves by up to 43.6 degrees while the joint holds still, which is 62 percent of the hysteresis gap, and no filter removes it because it is not noise.

Use two thresholds, and place them where the signal does not rest. On this clip that single decision is the difference between 20 and 6, and it costs nothing.

If you filter, use the 1 euro filter below a 200 ms lag budget: 44 percent less jitter than the best EMA at 100 ms. By 400 ms it has closed to 5 percent, which is where a constant-velocity Kalman finally pulls ahead of both. All three converge in the region where the delay is long enough to notice.

Expect a jitter floor. The landmarker’s output moves 0.34 px RMS on a frozen frame with the full model, and its noise is correlated at 0.694 from frame to frame, so any filter comparison you read that used simulated white noise is reporting numbers 1.78 times better than what you will get.

Further reading

• Casiez, Roussel and Vogel, 1 € Filter [3]. Four pages, and the reference implementations at gery.casiez.net/1euro ship a groundTruth.csv for checking a port. The one in this project agrees with it to 4 × 10⁻⁶.
• Oppenheim and Schafer, Discrete-Time Signal Processing, ch. 5 [2], for why a causal filter cannot remove lag and what group delay means, which is the part of the argument that does not depend on any of these measurements.
• Pupil diameter from a webcam, for the other half of this pattern. That post’s finding is that a network asked for millimetres with no scale reference produces a per-person offset; this one is about a threshold with no reference for where the signal rests. Both are the same mistake made at different ends of a pipeline.
• Image formation and colour spaces, for the projection step that step 2 rests on.

References

[1] Google. MediaPipe Pose Landmarker. Google AI Edge documentation, mediapipe 1.0.1. developers.google.com/edge/mediapipe/solutions/vision/pose_landmarker

[2] Oppenheim, A. V. and Schafer, R. W. Discrete-Time Signal Processing. Pearson. Group delay is defined in ch. 5, Transform Analysis of Linear Time-Invariant Systems, under Phase Distortion and Group Delay. Section numbering differs between editions, so the section title is quoted instead of a number.

[3] Casiez, G., Roussel, N. and Vogel, D. 1 € Filter: A Simple Speed-Based Low-Pass Filter for Noisy Input in Interactive Systems. Proceedings of CHI ‘12, pp. 2527–2530, 2012. doi.org/10.1145/2207676.2208639

[4] Forsyth, D. A. and Ponce, J. Computer Vision: A Modern Approach, 2nd ed., ch. 11, Tracking. Pearson, 2011.

[5] Grishchenko, I., Bazarevsky, V., Zanfir, A., Bazavan, E. G., Zanfir, M., Yee, R., Raveendran, K., Zhdanovich, M., Grundmann, M. and Sminchisescu, C. BlazePose GHUM Holistic: Real-time 3D Human Landmarks and Pose Estimation. arXiv:2206.11678, 2022. arxiv.org/abs/2206.11678

[6] Bazarevsky, V., Grishchenko, I., Raveendran, K., Zhu, T., Zhang, F. and Grundmann, M. BlazePose: On-device Real-time Body Pose tracking. arXiv:2006.10204, 2020. arxiv.org/abs/2006.10204

[7] Fleur, T. Interval Push-ups. Wikimedia Commons, CC BY-SA 4.0. commons.wikimedia.org/wiki/File:Interval_Push-ups.webm