Computer VisionObject TrackingSegmentation
TL;DR. Starting from "threshold the dark pixels", I ended up with a pipeline of three parts:
Almost every component exists because of one concrete failure. The sections below follow those failures in order.
The camera looks straight down at the net. The puck is a black disc about 17 pixels across in a 1920×1080 frame. On empty ice it is easy to see. The trouble is everything else that is also small, dark and round:
On top of that, a shot moves the puck 100–200 pixels per frame and smears it into a faint, broken ring. The video is also a compilation of 8 clips with hard cuts, and the camera occasionally pans.
Everything is classical computer vision (OpenCV + NumPy), except the purple step: a pretrained COCO person-segmentation model (YOLO11-seg) used only to say "this pixel belongs to a player".
Motivation: the video is dim, and exposure changes between clips (ice brightness ranges from 122 to 147 on a 0–255 scale). My first fixed threshold rejected every real puck.
The ice covers most of the frame, so the median brightness is a robust reference: \[ \iota_t=\operatorname{median}_{\mathbf p}\,V_t(\mathbf p), \] and every photometric threshold is a multiple of \(\iota_t\).
Colour is tested with chroma \(C=\max_c I^c-\min_c I^c<60\) rather than HSV saturation. Saturation \(S=(\max-\min)/\max\) becomes unstable as \(\max\to 0\): near-black puck pixels came out with \(S=255\) from noise, which punched holes in the puck.
Motivation: a fast puck shows only a thin, broken dark rim. In frame 139, just 65 pixels pass the strict threshold. Simply lowering the threshold makes pucks merge with the stick blades they touch.
Fix: the same trick Canny uses for edges. Very dark pixels are seeds, and a blob grows into connected, moderately dark pixels:
\[ \mathcal S=\{V<0.45\,\iota\},\quad \mathcal W=\{V<0.70\,\iota\},\quad \mathcal M^{\text{hyst}}=\bigcup\big\{\mathcal K\in\mathrm{CC}_8(\mathcal W\cup\mathcal S):\mathcal K\cap\mathcal S\neq\varnothing\big\}. \]Both masks are then closed with a small disc, small holes are filled (the puck's bright logo makes it look like a ring), and both masks propose candidates.
| Test | Definition | Keeps | Why |
|---|---|---|---|
| circularity | \(\kappa=4\pi A/P^2\) | \(\ge0.6\) | blade fragments \(\approx0.55\) |
| elongation | rotated-rectangle sides \(a/b\) | \(\le2.3\) | a diagonal stick blade has a square axis-aligned box. Blades measured 2.5–3.2, pucks 1.0–1.9 |
| contrast ring | median of the ring \(\ge0.75\,\iota\) | bright surroundings | the puck sits alone on ice |
| clutter annulus | \(\xi=\langle \text{dark}\lor\text{coloured}\lor\text{very bright}\rangle_{12\le r<32}\) | \(\le0.35\) | jersey digits and goalie masks: 0.47–0.78. Real pucks: \(\le0.15\) |
| frame edge | blob touches the border | reject | the player it belongs to may be off-screen |
The candidate score combines these: \[ \sigma=\min\!\Big(1,\tfrac{\Delta}{\iota}\Big)\cdot\min(1,\kappa)\cdot(1-\rho)\cdot(1-\xi)\cdot\min\!\Big(1,\tfrac{A}{100}\Big), \] where \(\Delta\) is the contrast with the surrounding ring and \(\rho\) the fraction of dark pixels in that ring.
Motivation: the camera exposes for the ice, so a white jersey is about as bright as the ice. A black skate, a helmet opening or a dark sleeve stripe on a white player looks exactly like a puck on ice.
I tried seven single-frame cues: brightness, texture variance, edge density, angular enclosure, morphological "attachment to a player", colour and rotated shape. Every one of them overlapped with real pucks that sit next to players. The missing information was not about the blob; it was about where the players are.
Fix: segment people with a pretrained model and reject candidates on or near them. With player mask \(\mathcal P\), the distance transform is \[ D(\mathbf p)=\min_{\mathbf q\in\mathcal P}\lVert\mathbf p-\mathbf q\rVert_2, \] and a candidate blob \(\mathcal K\) is rejected if \(\min_{\mathbf p\in\mathcal K}D(\mathbf p)<25\) px.
Motivation: a puck-shaped fixture at the top of the net never moves, and it was reported as the puck for about 140 frames.
"Not moving" must mean not moving relative to the rink, because the camera sometimes pans. Phase correlation failed here: the ice is almost textureless and the moving players dominated it. Instead, I track corner features of the static scene (rink lines, crease, net) with pyramidal Lucas–Kanade, ignoring corners on the player masks, and take the median shift:
\[ \boldsymbol\delta_t=\operatorname{median}_k\,\mathbf u_k,\qquad \mathbf x^{\text{rink}}=\mathbf x-\textstyle\sum_{\tau\le t}\boldsymbol\delta_\tau . \]For each candidate, compare its neighbourhood with the same rink location 4 frames earlier:
\[ m(\mathbf c)=\Big\langle\big|G_t(\mathbf c+\mathbf u)-G_{t-4}\big(\mathbf c-(\boldsymbol\Delta_t-\boldsymbol\Delta_{t-4})+\mathbf u\big)\big|\Big\rangle_{\lVert\mathbf u\rVert\le12}. \]A track can only be the puck if its detections span at least 25 px on the rink within the last 20 frames, and at least one of its last 2 detections has \(m\ge8\). The second condition catches a track that jumps from a moving puck onto a static mark and would otherwise keep its "moving" history.
Most of my remaining "puck not detected" reports turned out to be tracker failures. The detector saw the puck; the tracker never assembled the sightings into a confirmed track. These were the fixes:
A brand-new track has no velocity, but a shot can move the puck 207 px in one frame, more than the original 145 px search radius. The search radius is now
\[ g=45+1.5\lVert\mathbf v\rVert+(10+0.8\lVert\mathbf v\rVert)\,\mu+220\cdot\mathbb 1[h<2], \]where \(h\) is the number of sightings and \(\mu\) the number of consecutive missed frames. After one sighting the search is wide. From the second sighting on, the window is centred on the constant-velocity prediction, so the third sighting must continue the same direction and spacing. A random pair of blobs rarely does.
On a track's second sighting the velocity was blended 60/40 with its initial value of zero, so a 100 px/frame pass was recorded as 60. After a one-frame gap the puck was out of range. Now:
\[ \mathbf v\leftarrow\alpha\,\frac{\mathbf c-\boldsymbol\ell}{\mu+1}+(1-\alpha)\,\mathbf v,\qquad \alpha=\begin{cases}1 & h=1\\ 0.6 & h\ge2\end{cases} \]Here \(\boldsymbol\ell\) is the last actual sighting, not the coasted prediction. While the puck is hidden, the predicted speed is reduced only gently (\(\times0.9\) per frame), so an accelerating shot isn't lost behind a player.
Some blurred rims break into separate fragments (circularity 0.35), and loosening the global thresholds would admit far too much. If the tracked puck is missing, the detector runs again on a small window around its prediction with relaxed size and shape limits, and keeps only candidates scoring at least 0.15 (the real puck: 0.20; a crease mark: 0.09).
Motivation: the goalie's moving stick blade (frames 610–620) beat a faint puck in the net. It is dark, moving and roundish, but 3–6 times larger than a puck.
Fix: derive the puck's size from the rink itself. The goal crease radius is 6 ft and spans about 420 px, so the scale is \(k\approx70\) px/ft. A 3-inch puck is therefore
\[ d_{\text{px}}=k\cdot 0.25\ \text{ft}\approx17.5\ \text{px},\qquad A_{\text{puck}}=\tfrac{\pi}{4}d_{\text{px}}^2\approx240\ \text{px}^2 . \]Allowing about 2.25× for motion blur and shadows gives \(A\le540\) px². Measured: blades 598–688 px², every real puck at most 475.
| Symptom | Root cause | Fix |
|---|---|---|
| nothing detected | absolute thresholds on a dim video | ice-relative thresholds |
| puck in the net missed | HSV saturation noisy near black | chroma |
| jersey digits, goalie mask | dark blobs on fabric | clutter annulus (all colours) |
| stick blade on pads | diagonal blade has a square box | rotated-rectangle elongation |
| skates, helmets, sleeves | white jersey ≈ ice brightness | person segmentation |
| static net fixture | no notion of motion | camera compensation + motion rule |
| track jumps onto a mark | motion judged per track only | per-detection motion evidence |
| blurred fast puck | broken thin rim | hysteresis mask + re-detection near the prediction |
| shots never confirmed | 145 px gate, heavy damping | two-point start-up, speed-aware gate |
| pass lost after a 1-frame gap | velocity blended with zero | \(\alpha=1\) on the 2nd sighting |
| first frames missing | 3-sighting confirmation | backfill |
| one-frame puck | no track possible | strong single-frame rule |
| stick blade chosen over the puck | loose area limit | geometric size prior |
| tracks across clip cuts | compilation video | frame-difference cut detector |
| Version | Frames with a detection | What changed |
|---|---|---|
| classical detector + tracker | 35.5% | many false positives |
| + player masks | 29.2% | skates and helmets removed |
| + motion rule | 13.0% | ~140 frames of a static fixture removed |
| + hysteresis mask, re-detection | 25.7% | fast pucks recovered |
| + fast-puck tracking, backfill | 33.1% | shots confirmed, gaps filled |
| + size prior | 32.7% (+18.6% predicted) | stick blade replaced by the real puck |
The share of detected frames went down before it went up. The early numbers were inflated by false positives, and each filter first removed those before the tracking work recovered real pucks. Much of the remaining "lost" time is genuine: the puck is under a goalie's pad, in a glove, or off-screen.
Lessons.