Facial expression recognition (FER) is the task of determining a person’s current emotion. It plays an important role in healthcare, marketing, and counselling. With the advancement in deep learning algorithms like Convolutional Neural Network (CNN), the system’s accuracy is improving. A hybrid CNN and k-Nearest Neighbour (KNN) model can improve FER’s accuracy. This paper presents a hybrid CNN-KNN model for FER on the Raspberry Pi 4, where we use CNN for feature extraction. Subsequently, the KNN performs expression recognition. We use the transfer learning technique to build our system with an Efficient Net-Lite model. The hybrid model we propose replaces the Softmax layer in the Efficient Net with the KNN. We train our model using the FER-2013 dataset and compare its performance with different architectures trained on the same dataset. We perform optimization on the Fully Connected layer, loss function, loss optimizer, optimizer learning rate, class weights, and KNN distance function with the k-value. Despite running on the Raspberry Pi hardware with very limited processing power, low memory capacity, and small storage capacity, our proposed model achieves a similar accuracy of 75.26% (with a slight improvement of 0.06%) to the state-of-the-art’s Ensemble of 8 CNN model.

The major building blocks of the project are:

• Raspberry pi.
• Pi camera.
• SD card.
• LCD display.

Software’s used:

1. Embedded Linux programming.
2. Express SCH for Circuit design.
3. Face recognition using CNN.

Block diagram of the project:

video: