• Conférence
  • Ingénierie & Outils numériques

Toward Interpretable Multimodal Stress Detection Using Facial Geometry and Emotional Valence

Conférence : Communications avec actes dans un congrès international

Stress is a major public health concern associated with cardiovascular disease, burnout, and cognitive decline, highlighting the need for accurate and accessible stress monitoring. This paper proposes an interpretable multimodal framework for binary stress detection using only a standard RGB camera. The proposed approach integrates two sources of facial information: temporal geometric facial descriptors extracted from MediaPipe FaceMesh and emotional valence estimated using DeepFace. To the best of our knowledge, this is the first study to combine temporal facial geometry and emotional valence within an interpretable camera-only framework for binary stress detection on the UBFC-Phys dataset. The framework was evaluated on recordings from 56 participants. The machine learning pipeline, based on XGBoost, achieved a balanced accuracy of 93.05% under Leave-One-Subject-Out cross-validation, while model-agnostic SHAP analysis of the weighted late-fusion predictor identified lip-related descriptors as the most influential feature group, followed by emotional valence. A deep learning pipeline integrating CNN, ST-GCN, and BiLSTM with weighted late fusion achieved a balanced accuracy of 90.77%. These results demonstrate that the proposed framework provides accurate and interpretable stress detection without relying on wearable sensors or physiological signal acquisition.