What "validated against PSG" actually means.

Almost every sleep gadget claims it's "validated against PSG." PSG — polysomnography — is the clinical sleep study, the one with all the sensors, and it's the closest thing the field has to ground truth. So "validated against PSG" sounds like a stamp of approval. Sometimes it is. Often it's nearly meaningless. The phrase does a lot of hiding.

I want to walk through what makes a validation actually worth something — partly so you can judge ours, and partly so you can see through the ones that aren't.

The word that's doing all the work: "paired"

The only validation that tells you anything is a paired-night one: the device and the PSG recording the same person, on the same night, at the same time. Then you can ask the real question — when the sleep study counted a breathing event at 3:14 a.m., did the phone catch it too?

What you'll often find instead is much weaker. "We compared our average results to published PSG averages." That's not validation, that's two unrelated groups that happen to land near each other. Or worse: a study on a handful of nights, in a quiet lab, on young healthy people. If the word "paired" isn't there — or the number of paired nights isn't there — assume the comparison is loose.

The part that's easy to cheat: scoring

Here's a subtle one. Someone has to look at the PSG and mark where the real events were — that's "scoring," and it's done by trained technicians. The question is: did the scorer know what the app already said?

If they did, the test is circular. It's like grading your own exam with the answer key open. The honest version is blinded scoring: the technician marks the sleep study without ever seeing the app's output. Only then is the agreement meaningful, because nobody was nudged toward the answer. When a company doesn't mention blinding at all, that silence is information.

"Agreement" is a spectrum, not a checkbox

Even with paired nights and blinded scoring, "it agreed with PSG" isn't yes-or-no. A phone might catch individual events well but drift on the per-hour total, or vice versa. The careful way to report it is a method called Bland–Altman, which asks: across many nights, how far apart are the two numbers, and does the gap grow as the nights get more severe?

For SomniSense, the per-night number lands within ±5 events per hour of the sleep-study count on 87% of nights, in the range where it matters most. I'm giving you that figure with its boundary attached on purpose — "agreement," unqualified, is the kind of word that sounds better than it should.

How ours was actually run

Briefly, so you can hold it against the checklist above: 80 paired nights across 40 participants — some in a lab, most at home with a portable sleep-study rig — adults with and without known sleep-breathing issues, scored by AASM-trained technicians who were blinded to what SomniSense said. The full design, the cohort we didn't represent well, and the preprints behind it are on the research page; the number-by-number results are on the accuracy page.