Real-Time Face Blur

Installation & Quickstart

The project is packaged as a simple Python tool using Conda for dependency management. From the repository root:

1. Install conda or mamba.
2. Create the environment:
   conda env create --name face-blur-rt --file face-blur-rt.yml
3. Activate the environment:
   conda activate face-blur-rt
4. Run the demo:
   python main.py

You can point the demo to a webcam or to a video file; configuration is handled through simple command-line flags and config files.

Initial Goal

The goal of this project is to build a practical, privacy-preserving face blurring system that works in real time on everyday hardware.

• Blur faces in live streams and recorded footage without manual editing.
• Keep registered faces unblurred using a lightweight recognition model.
• Offer different trade-offs between frame rate and detection accuracy.
• Expose multiple blur modes (e.g., bounding-box blur vs. tighter facial segmentation).

Typical use cases include IRL streaming (e.g., Twitch), redacting security footage, and any setting where video must be monitored live without exposing bystanders' identities.

Implementation & Project Architecture

RealTimeFaceBlurrerByFrame orchestrates the entire pipeline, taking in video frames and passing them through detection, tracking, recognition, and blurring. Each stage is implemented as its own class and can be swapped out without changing the rest of the system.

• Video source: webcam or video file.
• FaceDetector: YuNet or SCRFD (both subclass a common interface).
• FaceTracker: SORT-based tracker, tuned for high recall.
• FaceRecognizer: SFace-based recognizer that keeps known faces unblurred.
• Blurrer: Multiple blur shapes and methods (e.g., box vs. Gaussian).
• PerformanceSettings: simple knobs to trade off speed vs. accuracy.

The pipeline is designed to be modular: add a new detector, tracker, or blur method by subclassing the corresponding abstract base class.

Key Components

Detection & Tracking

• YuNet: lightweight face detector built with depthwise separable convolutions and a feature pyramid, balancing speed and accuracy on edge devices.
• SCRFD: efficient face detector that redistributes samples and computation to difficult detection stages, providing a stronger alternative to YuNet when needed.
• SORT Tracker: online multi-object tracker using Kalman filters + Hungarian matching. We tune it for high recall so faces stay covered even when detection briefly fails.

Recognition

• SFace: privacy-friendly recognizer trained on synthetic faces. With only a few reference images per person, it can keep specific faces unblurred while anonymizing everyone else.

Core Libraries

• OpenCV + NumPy: frame capture, drawing bounding boxes, and applying blurs.
• PyTorch: loading and running pretrained models and timing GPU/CPU inference.

Benchmarking & Performance

We benchmark our system on a challenging 6-minute IRL Twitch stream clip with crowds, occlusions, and rapid motion. TinaFace serves as a strong GPU baseline; our system runs entirely on CPU.

• Average ~6.8 ms per frame on a laptop CPU (Intel i7-12700H) ⇒ ~147 FPS.
• Robust to heavy occlusions and many faces on screen.
• Tracker configuration keeps faces covered when the detector briefly drops them.
• Multiple blur methods (box, black, Gaussian) with different speed/appearance trade-offs.

While TinaFace achieves strong detection metrics, it fails to run reliably in this real-time, many-face setting. Our detector + tracker combination stays stable and faster than real time.

The excess-boxes analysis shows how our tracker deliberately errs on the side of drawing more boxes than strictly necessary. This high-recall configuration means we almost never miss a face, even if it leads to a small number of false-positive boxes on background regions. For a privacy-focused application like face blurring, this trade-off is desirable: it is better to blur a few extra patches than to expose a single face.

Discussion & Future Work

The system meets its original goal: real-time face blurring on live and recorded video with configurable speed/accuracy trade-offs and a simple interface for whitelisting faces.

Some current limitations and next steps:

• Handling extreme motion, low-light conditions, and very low frame rates more robustly.
• Reducing false-positive boxes introduced by the aggressive tracker configuration.
• Improving recognition for visually similar faces.
• Building a richer user interface for interactively choosing which faces to unblur.
• Further optimizing the pipeline for long offline videos and multi-hour streams.