How accurate is SomniSense — really?

The single-number problem

A single rounded "accurate" percentage sounds great until you ask: accurate at what, on what population, against what ground truth. Accurate at detecting that there was a sound? Distinguishing snores from background noise? Counting the right number of breathing pauses in a night? These are different questions with different answers. Collapsing them into one number is the thing you do when you don't want people to look too closely.

SomniSense breaks accuracy into several numbers, each tied to a specific question. They tell different stories. Together they tell the whole one.

Two indices, one app — and why the validation numbers split that way

SomniSense produces two independent indices each night, and they're validated separately:

• BRI (Breathing Irregularity Index) — apnea-like + reduced-breathing sound events per hour, derived from acoustic event detection. The underlying production benchmark reports 88.49% accuracy and 88.06% F1 for 200-second window classification, plus 56.4 KB / 0.064 ms inference on Apple M2 Neural Engine.
• SRI (Snoring Rate Index) — snore events per hour. The underlying 1-second snore-event benchmark reports 91.67% sensitivity and 89.01% precision.

The reason there are two numbers per index isn't marketing — it's that they answer different questions. BRI is medically interpretable (because the AHI metric exists and clinicians use the same per-hour shape). SRI is product-defined (we made it up — carefully, but still). The validation numbers are what tell you each of them is honest.

What our research measures, and what BRI actually does

Three different things, often mixed up. We try not to mix them:

• Window classification (what Paper E validates). Given a 200-second window of bedside audio, the model classifies it as Normal or Abnormal (apnea/hypopnea event). Production benchmark: 88.49% accuracy and 88.06% F1.
• Per-night BRI (what the app reports each morning). The app aggregates per-event detections across your night: BRI = breathing irregularity events detected / sleep hours. This is a derived per-hour rate. Same shape clinicians use for AHI, computed from a different signal source.
• OSA (Obstructive Sleep Apnea) diagnosis — what a sleep specialist does, not what SomniSense does. A clinical diagnosis based on polysomnography, daytime symptom assessment, and a physician's judgment. BRI is data; OSA diagnosis is a doctor's call.

The 87% Bland-Altman agreement below — "BRI within ±5 of the PSG-scored per-hour event rate" — is a system-level per-night validation: do our acoustic per-event detections, aggregated to per-hour, fall close to what PSG would have counted on the same night? Yes, 87% of the time within ±5 events/hour. That tells you BRI is a useful acoustic estimator of the AHI-shaped metric. It does not tell you BRI is a substitute for clinical OSA diagnosis. If your BRI is consistently elevated, see a sleep specialist.

The benchmark numbers we actually publish

These come from 80 paired PSG nights across 40 participants (10 in-lab + 70 ambulatory PSG with nasal-airflow cannula) — meaning we ran SomniSense on a phone next to the same person, on the same night, while they were also being recorded with a real polysomnography setup. The audio was scored by AASM-trained sleep technicians who didn't know what SomniSense had said.

That last part — "didn't know what we said" — is what "blinded scoring" means. We don't get to pre-train our scorers on our own answers. Otherwise the test would be circular.

How we tested it (the methodology, plain)

Detection performance was measured against gold-standard in-lab polysomnography (PSG) recordings, audio-annotated by certified sleep technicians. Every audio segment they scored was blinded to what SomniSense said about it. That's the only honest way to do it.

Specifically:

• Sample: 80 paired nights / 40 participants (smartphone + PSG simultaneously) — 10 in-lab PSG + 70 ambulatory PSG with nasal-airflow cannula. Adults with and without diagnosed sleep breathing concerns. We state who's underrepresented (mostly: under-18, severe BMI extremes, certain ethnic groups) openly, rather than hide it behind an aggregate number. I want to be specific about who's not in the cohort because aggregate sample size without context can be misleading.
• Recording medium: bedside smartphone (varied iPhone & Android models from 2018 onward), 50–90 cm from participant's head.
• Ground truth: PSG with synchronized audio channel; manual scoring by AASM-trained sleep technicians, blinded to SomniSense output.
• Comparison: per-event sensitivity, per-event precision, per-night BRI vs PSG AHI agreement (Bland–Altman analysis).

The breathing-event detection algorithm builds on years of sleep apnea research by our founder. The version powering SomniSense was retrained from scratch and rebuilt for SomniAI LLC to handle smartphone audio specifically — different microphone, different distance, different acoustic context than clinical hardware. Three companion preprints document the methodology in full: the cascaded-baselines preprint (multi-seed bootstrap), the Coordinate-Attention 1D architecture preprint (93.2% parameter reduction), and the on-device compression preprint (0.064 ms inference on Apple Neural Engine). All are published openly on Zenodo with citable DOIs (cs.LG, eess.AS). For how the two-stage system fits together and the full preprint portfolio, see the research program. The full technical hub — the architecture in depth, the SDK, and licensing for hardware and clinical partners — lives at apneasense.com/research.

Honest limitations

Here's what we don't know yet, and what I'd want to know if I were the user:

• The cohort skews adults with sleep breathing concerns. If you're a healthy 24-year-old with no symptoms, the numbers above are likely conservative for your case (you're probably underrepresented in the sample).
• Acoustic environment matters. If your bedroom has unusual acoustic properties — hard surfaces, partner snoring louder than you, a fan blowing directly at the phone — the model may catch fewer of your events. The methodology paper documents the conditions we tested under.
• Borderline events. The events we miss are mostly the borderline ones near the 10-second / 30%-amplitude threshold. We miss them on purpose, by tuning for precision over sensitivity. If your real BRI is 12, we might say BRI 10. If we say BRI 18, you can trust it's at least 18.
• Not validated for under-18. The cohort was adults only.
• Preprints, not yet peer-reviewed. Three companion preprints are published openly on Zenodo with citable DOIs; peer-reviewed journal publication is a separate process and will be noted on this page when complete.

What this isn't

• Not a diagnostic claim. Even at these numbers, SomniSense is not a medical device and doesn't diagnose obstructive sleep apnea (OSA). OSA diagnosis requires polysomnography, symptom assessment, and a sleep specialist's judgment. SomniSense isn't validated for users under 18.
• Not a personal guarantee. Your specific results may differ from population averages. Read the methodology paper to know whether your scenario is in or out of distribution.
• Not a replacement for a sleep study. If your BRI runs above 15 consistently, that's a clinic conversation. We give you the data to bring. The clinic gives you the diagnosis.

Not sure if this is your problem? Start from a symptom

If you came here to check the evidence before trusting the app, the other way in is whatever you're actually feeling. Each of these walks through the breathing pattern behind the symptom and what to do about it:

• → Waking up gasping or choking
• → Still tired after a full eight hours
• → Loud snoring
• → Morning headaches