Most "emotion recognition" repos are really two projects stapled together: a face detector, and a classifier that only works on a perfectly cropped face. The interesting engineering question isn't "can a model tell happy from sad" - that part is a solved, importable problem - it's how little code it takes to wire the detector and the classifier together into something that runs live off a webcam.
Architecture
flowchart TD subgraph group_runtime["Desktop Runtime"] node_entry["emotion.py
Python entry point
[emotion.py]"] node_deps["requirements.txt
dependencies
[requirements.txt]"] node_webcam{{"Default webcam
camera input"}} node_display["OpenCV display window
local ephemeral UI
[emotion.py]"] node_cleanup["Camera and window cleanup
OpenCV shutdown
[emotion.py]"] end subgraph group_vision["Vision Pipeline"] node_capture["VideoCapture loop
OpenCV acquisition
[emotion.py]"] node_frame["Camera frame
image frame
[emotion.py]"] node_grayscale["Grayscale conversion
OpenCV image operation
[emotion.py]"] node_boxes["Face boxes
detection output
[emotion.py]"] node_roi["Face ROI crop
per-face region
[emotion.py]"] node_emotion["Emotion result
scores or dominant label
[emotion.py]"] node_annotate["Frame annotation
OpenCV drawing
[emotion.py]"] end subgraph group_models["Model Boundaries"] node_cascade{{"Haar face detector
OpenCV's bundled cascade, not the repo's xml copy"}} node_deepface{{"DeepFace analysis
emotion inference API"}} node_tensorflow{{"TensorFlow/Keras model
pretrained neural network"}} end node_deps -.->|"runtime dependencies"| node_entry node_entry -->|"creates"| node_capture node_webcam -->|"camera stream"| node_capture node_capture -->|"reads"| node_frame node_frame -->|"converts"| node_grayscale node_grayscale -->|"detects faces"| node_cascade node_cascade -->|"emits"| node_boxes node_frame -->|"crops by box"| node_roi node_boxes -->|"selects"| node_roi node_roi -->|"analyzes"| node_deepface node_deepface -->|"uses"| node_tensorflow node_deepface -->|"returns"| node_emotion node_frame -->|"base frame"| node_annotate node_boxes -->|"rectangles"| node_annotate node_emotion -->|"label text"| node_annotate node_annotate -->|"renders"| node_display node_display -.->|"continues until q"| node_capture node_display -->|"q pressed"| node_cleanup click node_entry "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/emotion.py" click node_deps "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/requirements.txt" click node_capture "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/emotion.py" click node_frame "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/emotion.py" click node_grayscale "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/emotion.py" click node_cascade "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/emotion.py" click node_boxes "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/emotion.py" click node_roi "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/emotion.py" click node_emotion "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/emotion.py" click node_annotate "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/emotion.py" click node_display "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/emotion.py" click node_cleanup "https://github.com/manish-9245/facial-emotion-recognition-using-opencv-and-deepface/blob/main/emotion.py" classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554 classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337 classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81 classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a class node_entry,node_deps,node_webcam,node_display,node_cleanup toneBlue class node_capture,node_frame,node_grayscale,node_boxes,node_roi,node_emotion,node_annotate toneAmber class node_cascade,node_deepface,node_tensorflow toneMint
Boxes are clickable and jump straight to the real source file on GitHub.
The whole program, in outline
emotion.py is a single file. The loop is:
- Open the default webcam with
cv2.VideoCapture(0). - Convert each frame to grayscale, then find faces with OpenCV's built-in Haar cascade:
cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml'). - For every detected face box, crop it back out of a version of the frame converted to RGB (DeepFace expects three channels, so grayscale has to be expanded back out even though the actual detection ran on a single channel).
- Hand that crop to
DeepFace.analyze(...)and draw whatever emotion comes back on top of the live video.
result = DeepFace.analyze(face_roi, actions=['emotion'], enforce_detection=False)
emotion = result[0]['dominant_emotion']
cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.putText(frame, emotion, (x, y - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (0, 0, 255), 2)The one flag that makes this work
enforce_detection=False is doing more work than its size suggests. DeepFace ships its own, heavier face detector as part of analyze(), and by default it raises an exception if it can't confirm a face in the image you hand it. Since the Haar cascade has already found and cropped a face by the time DeepFace sees it, re-running DeepFace's own detector on that crop would be redundant work - so the flag tells DeepFace "trust me, just classify this pixel patch." That's the actual design decision behind "the shortest code": use a cheap, fast classical detector to find where faces are, and skip straight to the expensive deep model for what expression they're making, rather than paying for detection twice.
It's a real accuracy trade, not a free lunch. Haar cascades are a 2001-era technique - much less robust to side angles, partial occlusion, and uneven lighting than the detector DeepFace would have used on its own. Bypassing it is what keeps this fast enough to run live on a laptop webcam, at the cost of missing faces the cascade itself can't find in the first place.
Where the frame budget actually goes
The obvious performance question for anything running per-frame is: what's the expensive part? Here it's unambiguous - DeepFace.analyze() is a full model inference, called synchronously, once per detected face, inside the main video loop. With one face in frame that's tolerable; with three or four people on camera, the loop has to run that inference three or four times before it can draw the next frame, and there's no batching, threading, or frame-skipping to soften it. For a demo script that's an acceptable ceiling. For anything meant to hold a steady frame rate with multiple people in view, the fix is the obvious one - batch all detected faces into a single DeepFace.analyze() call, or move inference to a background thread and let the video loop draw the last-known label while it catches up.
Running it
There's no training step and no dataset to download - pip install -r requirements.txt pulls opencv-python, deepface, and tf_keras (DeepFace's actual backend is TensorFlow, pulled in transitively). The one manual step is grabbing haarcascade_frontalface_default.xml from OpenCV's own GitHub, even though the code technically loads that same cascade from OpenCV's bundled data path (cv2.data.haarcascades) rather than the copy sitting in the repo - a small mismatch between what the README asks you to do and what the script actually reads, worth knowing if you go looking for where that XML file matters. From there it's python emotion.py, and q to quit.