MSAF-NeXt: A Unified Multi-Scale Feature Fusion Framework for Facial Emotion Recognition Across Benchmark and Autism-Related Datasets

Facial Emotion Recognition (FER) is fundamental in computer vision, with applications in human–computer interaction, healthcare, and assistive technologies. Despite significant progress, FER remains challenging due to pose, illumination, occlusions, and subtle expression dynamics. In this paper, we propose MSAF-NeXt, a unified framework that integrates spatial alignment, hierarchical feature fusion, and lightweight attention within a ConvNeXt backbone to improve robustness in facial emotion recognition. Specifically, it combines a Spatial Transformer Network (STN) for facial alignment, an Efficient Channel Attention (ECA) module for feature refinement, and a Multi-Scale Attention Fusion (MSAF) strategy to fuse complementary semantic representations from multiple feature scales. Evaluated on FER2013 and through transfer learning on FERAC, a challenging autism-related FER dataset with limited, noisy, and highly imbalanced data, MSAF-NeXt achieves competitive benchmark performance and consistently improves after transfer learning, demonstrating robustness and adaptability across both standard and domain-specific scenarios.