LubDubDecoder

Open-source code for CHI'26 paper LubDubDecoder: Decoding Heart Sounds from In-Ear Phonocardiogram using Deep Learning

Repository contents

File	Description
train_withinuser.py	Train and evaluate a model on within-user data using k-fold cross-validation
train_crossuser.py	Train and evaluate cross-user (leave-one-subject-out) models with optional calibration
dataplot.py	Visualize SCG, GCG, and ear-based cardiac signal channels together
babyface_pro_live_stream.m	Stream live audio from an RME Babyface Pro and estimate heart rate in MATLAB
model_orig.py	Neural network architecture
dataset.py	Dataset loading and preprocessing
utils.py	Training utilities and metrics
plot_npy.py	Evaluation plotting helpers

Quickstart commands

pip install -r requirements.txt

# Within-user training (k-fold cross-validation)
python train_withinuser.py \
  --identifiers participant_a \
  --signal_type scg \
  --groups spk_earphones

# Cross-user training (leave-one-subject-out)
python train_crossuser.py \
  --identifiers participant_a participant_b participant_c \
  --identifier my_run \
  --signal_type scg \
  --groups spk_earphones

Data privacy

Participant recordings are not included in this repository. Because of data privacy and human-subjects considerations, we cannot share our users' raw or labeled data publicly. You will need to prepare your own dataset locally following the expected directory layout (see below).

Expected data layout

Place your private data locally:

data/
  <participant_id>/
    <participant_id>_spk_earphones.wav
    <participant_id>_acc.txt
    <participant_id>_gyro.txt
data_label/
  label_output/
  label_cycle/

Model checkpoints are written to pth/ and evaluation outputs to eval/.

Multi-channel visualization:

Edit the participant, folder, and filename variables at the bottom of dataplot.py, then run:

python dataplot.py

Live Babyface Pro streaming (MATLAB):

Open babyface_pro_live_stream.m, set deviceName to your audio device, and run the script.

Citation

@inproceedings{10.1145/3772318.3790445,
author = {Zhang, Siqi and Zhang, Xiyuxing and Nguyen Tien Vu, Duc and Qiang, Tao and Palacios, Clara and Zhu, Jiangyifei and Wang, Yuntao and Goel, Mayank and Chan, Justin},
title = {LubDubDecoder: Bringing Micro-Mechanical Cardiac Monitoring to Hearables},
year = {2026},
isbn = {9798400722783},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3772318.3790445},
doi = {10.1145/3772318.3790445},
booktitle = {Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems},
articleno = {145},
numpages = {25},
keywords = {Earables, Cardiovascular health, Wearable health monitoring, Well-being, Seismocardiography},
location = {
},
series = {CHI '26}
}