Deep Transfer Learning and Feature Fusion for Improving Facial Expression Recognition on JAFFE Dataset

Abstract

Facial Expression Recognition (FER) is extensively used in human-computer interaction, where machines can recognize people's emotions through facial expressions. In this study, a hybrid FER framework was proposed using MobileNetV2 deep learning features combined with traditional handcrafted descriptors, LBP and HOG, to enhance classification performance on small datasets. Additionally, we evaluate our approach using the Japanese Female Facial Expression (JAFFE) dataset, which consists of 213 grayscale images showing seven basic emotions. Data augmentation and transfer learning were applied to increase model generalization. The feature fusion scheme leveraged deep semantic features and local texture descriptors. The feature fusion scheme was used to leverage the benefits of deep semantic features and local texture descriptors, integrated via a dimensionality reduction and classification module with a CNN-based architecture. The hybrid method achieved 96.49% accuracy, outperforming MobileNetV2 alone (94.73%) and handcrafted features (95.17%). This demonstrates both the utility of feature fusion to enhance FER accuracy in constrained datasets and indicates the possibility for more reliable emotion recognition systems in live applications.