Realtime Human Face Presence Detection using Face Api

[reboottime]

Overview

This article tracks my solution path on human presence in front of camera using Face Api in React app

• what is Face Api and what problems it solves
• Solution Code in React

[reboottime]

About FaceApi

Major Concepts for new User

1. Face Detection

• Detects face locations in images/video
• Multiple detector options: SSD MobileNet, Tiny Face Detector, MTCNN
• Returns bounding boxes with confidence scores

2. Face Landmark Detection

• Identifies 68 facial landmark points
• Maps features like eyes, nose, mouth, jawline
• Supports regular and tiny models

3. Face Recognition

• Generates face descriptors (128-value feature vectors)
• Enables face matching and identification
• Uses ResNet-34 architecture

4. Face Expression Analysis

• Detects emotions (neutral, happy, sad, angry, etc.)
• Provides confidence scores for expressions
• Lightweight model optimized for browser

What problems does FaceApi solves

1. Browser-based Face Processing

• Runs entirely in browser using TensorFlow.js
• No server-side processing required
• Real-time face analysis capabilities

2. Cross-platform Compatibility

• Works in browsers and Node.js
• Supports various input types (images, video, canvas)
• Flexible deployment options

3. Face Recognition System Development

• Face matching and verification
• Identity management
• Access control systems

What models are inside of Faceapi

• Face Detection Models

  • SSD Mobilenet V1 (5.4MB) - High accuracy, slower
  • Tiny Face Detector (190KB) - Fast, mobile-friendly
  • MTCNN (2MB) - Configurable, includes landmarks

• Face Landmark Model

  • Default (350KB) - 68 facial points
  • Tiny (80KB) - Lightweight version

• Face Recognition Model

  • Based on ResNet-34 (6.2MB)
  • Generates 128-value face descriptors
  • 99.38% accuracy on LFW benchmark

• Face Expression Model

  • Lightweight (310KB)
  • Detects basic emotions
  • Uses depthwise separable convolutions

In Human (Realtime) Presence Detection

For basic presence detection, you should use either:

• SSD MobileNet V1
• Tiny Face Detector (Recommended)

[reboottime]

Solution

• where to download models in solution code
• Integrate FaceApi with React

import { useEffect, useRef, useState } from "react";

declare global {
  interface Window {
    faceapi: any;
  }
}

const FaceDetection = () => {
  const videoRef = useRef<HTMLVideoElement>(null);
  const canvasRef = useRef<HTMLCanvasElement>(null);
  const intervalRef = useRef<ReturnType<typeof setInterval>>();

  const [isModelLoaded, setIsModelLoaded] = useState(false);
  const [isLoadingCamera, setIsLoadingCamera] = useState(true);

  const [error, setError] = useState<string>("");

  const loadFaceAPI = async () => {
    try {
      const script = document.createElement("script");
      script.src = "/faceapi/faceapi.min.js";
      await new Promise<void>((resolve, reject) => {
        script.onload = () => resolve();
        script.onerror = () => reject();
        document.head.appendChild(script);
      });

      await Promise.all([
        window.faceapi.nets.tinyFaceDetector.loadFromUri("/faceapi/models"),
        window.faceapi.nets.faceLandmark68Net.loadFromUri("/faceapi/models"),
        window.faceapi.nets.faceRecognitionNet.loadFromUri("/faceapi/models"),
        window.faceapi.nets.faceExpressionNet.loadFromUri("/faceapi/models"),
      ]);

      setIsModelLoaded(true);
    } catch (err) {
      setError("Failed to load face detection models");
      console.error("Face API load error:", err);
    }
  };

  const startVideo = async () => {
    if (!videoRef.current) return;

    try {
      const stream = await navigator.mediaDevices.getUserMedia({
        video: {
          width: { ideal: 1280 },
          height: { ideal: 720 },
          facingMode: "user",
          frameRate: { ideal: 30 },
          aspectRatio: { ideal: 1.7777777778 },
        },
        audio: false,
      });
      
      videoRef.current.srcObject = stream;
      videoRef.current.onloadedmetadata = () => {
        videoRef.current?.play();
        setIsLoadingCamera(false);
      };
    } catch (err) {
      setError("Failed to access camera");
      setIsLoadingCamera(false);
      console.error("Camera access error:", err);
    }
  };

  const onVideoPlay = () => {
    if (!videoRef.current || !canvasRef.current || !window.faceapi) return;

    const canvas = canvasRef.current;
    const displaySize = {
      width: videoRef.current.clientWidth,
      height: videoRef.current.clientHeight,
    };

    window.faceapi.matchDimensions(canvas, displaySize);

    intervalRef.current = setInterval(async () => {
      if (!videoRef.current || !canvas) return;

      const detections = await window.faceapi
        .detectAllFaces(
          videoRef.current,
          new window.faceapi.TinyFaceDetectorOptions({
            inputSize: 512,
            scoreThreshold: 0.8,
          })
        )
        .withFaceLandmarks();

      const context = canvas.getContext("2d");
      if (!context) return;

      context.clearRect(0, 0, canvas.width, canvas.height);

      if (detections.length === 1) {
        console.log("Face detected", detections[0].detection.score);
        // Emit detection event or handle detection logic here
      }

      const resizedDetections = window.faceapi.resizeResults(detections, displaySize);
      window.faceapi.draw.drawDetections(canvas, resizedDetections);
      window.faceapi.draw.drawFaceLandmarks(canvas, resizedDetections);
    }, 100);
  };

  useEffect(() => {
    loadFaceAPI();
    return () => clearInterval(intervalRef.current);
  }, []);

  useEffect(() => {
    if (isModelLoaded) {
      startVideo();
    }
  }, [isModelLoaded]);

  return (
    <div className="h-full pt-8">
      <div className="w-full max-w-2xl mx-auto px-6">
        <div className="my-6">
          <h2 className="text-xl font-medium text-center mb-4">Face Detection</h2>
          <p className="text-lg text-gray-700 text-center">
            Please position your face within the camera frame.
          </p>
        </div>
        {error && <p className="p-4 text-red-500 text-center">{error}</p>}
        <div className="relative">
          {isLoadingCamera && (
            <div className="absolute inset-0 bg-gray-100 animate-pulse flex items-center justify-center">
              <p className="text-gray-700 text-xl">Loading camera...</p>
            </div>
          )}
          <video
            ref={videoRef}
            className="w-full h-auto border-2 rounded-sm border-blue-500"
            onPlay={onVideoPlay}
            playsInline
            autoPlay
            muted
          />
          <canvas
            ref={canvasRef}
            className="absolute top-0 left-0 w-full h-full"
          />
        </div>
      </div>
    </div>
  );
};

export default FaceDetection;

[reboottime]

Reference

• Build Real Time Face Detection With JavaScript
• FaceApi Doc