Assessing Robustness of EEG Representations under Data-shifts via Latent Space and Uncertainty AnalysisThe recent availability of large datasets in bio-medicine has inspired the
development of representation learning methods for multiple healthcare
applications. Despite advances in predictive performance, the clinical utility
of such methods is limited when exposed to real-world data. Here we develop
model diagnostic measures to detect potential pitfalls during deployment
without assuming access to external data. Specifically, we focus on modeling
realistic data shifts in electrophysiological signals (EEGs) via data
transforms, and extend the conventional task-based evaluations with analyses of
a) model's latent space and b) predictive uncertainty, under these transforms.
We conduct experiments on multiple EEG feature encoders and two clinically
relevant downstream tasks using publicly available large-scale clinical EEGs.
Within this experimental setting, our results suggest that measures of latent
space integrity and model uncertainty under the proposed data shifts may help
anticipate performance degradation during deployment.
arxiv.org