Pose detection under partial failure: what to ask the vendor before you sign

A pose detection stack rarely fails loudly. When the camera pipeline stalls or the inference service drops, most systems keep drawing the last skeleton they had, and the scoring layer keeps grading it. For a buyer with a shortlist and little lab time, every vendor question should be read as a probe into the silent gap between 'no pose' and 'old pose'.

Where does inference run when the link goes down?

Ask whether landmark extraction happens on the device, on a server, or on both with a switch between them. A hybrid design sounds resilient, but it hides a parity risk. The lightweight on-device model and the heavier server model rarely produce identical joint positions, so a dancer's score can shift the moment the connection degrades, and nobody can see why. Demand to know which model produced each frame and whether the scoring thresholds are calibrated separately for each one. If the vendor calls the fallback 'transparent', treat that as a red flag: when the switch is invisible in the interface, the drift is invisible in the results too.

What does the system do with a frozen frame?

This is the failure mode that matters most. A camera stall, a decoder hiccup or a thermal slowdown leaves the tracker holding or extrapolating the previous pose, and the timeline fills with plausible but fabricated movement. The architecture survives this only if every landmark set carries a capture timestamp and per-keypoint confidence, and if the scoring service rejects frames older than an agreed budget, for instance 120 ms from sensor capture to scoring. Ask the vendor to demonstrate that rejection path rather than describe it. On mid-range phones, long sessions can roughly halve the inference frame rate once thermal throttling sets in (an estimate, strongly device dependent), so the staleness check has to hold on a hot device, not only on a cold bench.

Who holds the buffer during an outage?

Ask also where raw video and landmark buffers sit while the cloud side is unreachable, because biometric privacy regimes such as GDPR draw no line between a deliberate archive and an upload queue that quietly grew on the device during an outage.

A decision rule for the shortlist

Keep only vendors whose pipeline can tell the scoring layer, frame by frame, that it has nothing current to offer. That one capability is the difference between a system that degrades into honest gaps and one that degrades into confident errors, and you can test it without a lab: cover the lens, pull the network, and watch whether the score pauses or keeps moving. The cost lands on your users, since honest gaps mean interrupted feedback and the occasional session marked incomplete. Decide before the contract whether your product can live with that, because the vendor who hides those gaps will always look smoother in the demo.

Can this be assessed without a full lab campaign?

Largely, yes. Run a full-length session on the weakest device you plan to support, block the camera and cut connectivity at moments you log yourself, then compare the exported timestamps and confidence values against those moments. Any score recorded while the sensor was blind is a defect, however good the overlay looked.

Back to DanceSaga - Pose Detection · Domains

← Back to DanceSaga - Pose Detection
Engineering & Architecture Brief

Pose detection under partial failure: what to ask the vendor before you sign

A pose detection stack rarely fails loudly. When the camera pipeline stalls or the inference service drops, most systems keep drawing the last skeleton they had, and the scoring layer keeps grading it. For a buyer with a shortlist and little lab time, every vendor question should be read as a probe into the silent gap between 'no pose' and 'old pose'.

Where does inference run when the link goes down?

Ask whether landmark extraction happens on the device, on a server, or on both with a switch between them. A hybrid design sounds resilient, but it hides a parity risk. The lightweight on-device model and the heavier server model rarely produce identical joint positions, so a dancer's score can shift the moment the connection degrades, and nobody can see why. Demand to know which model produced each frame and whether the scoring thresholds are calibrated separately for each one. If the vendor calls the fallback 'transparent', treat that as a red flag: when the switch is invisible in the interface, the drift is invisible in the results too.

What does the system do with a frozen frame?

This is the failure mode that matters most. A camera stall, a decoder hiccup or a thermal slowdown leaves the tracker holding or extrapolating the previous pose, and the timeline fills with plausible but fabricated movement. The architecture survives this only if every landmark set carries a capture timestamp and per-keypoint confidence, and if the scoring service rejects frames older than an agreed budget, for instance 120 ms from sensor capture to scoring. Ask the vendor to demonstrate that rejection path rather than describe it. On mid-range phones, long sessions can roughly halve the inference frame rate once thermal throttling sets in (an estimate, strongly device dependent), so the staleness check has to hold on a hot device, not only on a cold bench.

Who holds the buffer during an outage?

Ask also where raw video and landmark buffers sit while the cloud side is unreachable, because biometric privacy regimes such as GDPR draw no line between a deliberate archive and an upload queue that quietly grew on the device during an outage.

A decision rule for the shortlist

Keep only vendors whose pipeline can tell the scoring layer, frame by frame, that it has nothing current to offer. That one capability is the difference between a system that degrades into honest gaps and one that degrades into confident errors, and you can test it without a lab: cover the lens, pull the network, and watch whether the score pauses or keeps moving. The cost lands on your users, since honest gaps mean interrupted feedback and the occasional session marked incomplete. Decide before the contract whether your product can live with that, because the vendor who hides those gaps will always look smoother in the demo.

Technical FAQ: Can this be assessed without a full lab campaign?

Largely, yes. Run a full-length session on the weakest device you plan to support, block the camera and cut connectivity at moments you log yourself, then compare the exported timestamps and confidence values against those moments. Any score recorded while the sensor was blind is a defect, however good the overlay looked.

Related Architecture Specifications (DANCEGOAL): gesturedance.com · rehabanalysis.com · workoutmove.com