Mukh

Mukh (मुख, meaning "face" in Sanskrit) is a comprehensive face analysis library that provides unified APIs for various face-related tasks. It simplifies the process of working with multiple face analysis models through a consistent interface.

Features

• 🥸 DeepFake Detector: First python package featuring an Ensemble of multiple models
• 🎯 Unified API: Single, consistent API for multiple face analysis tasks like face detection and reenactment
• 🔄 Model Flexibility: Support for multiple models per task
• 🛠️ Custom Pipelines: Optimized preprocessing and model combinations

Documentation

The library is documented in detail, click here to view the documentation.

Currently Supported Tasks

• Face Detection
• Face Reenactment with Source Image and Driving Video
• Deepfake Detection for Image and Video
• Deepfake Detection Pipeline - Ensemble of multiple models

Installation

bash conda create -n mukh-dev python=3.10 -y conda activate mukh-dev pip install mukh==0.1.14

Usage

Face Detection

```python from mukh.face_detection import FaceDetector

Initialize detector

detectionmodel = "mediapipe" # Other models: "blazeface", "ultralight" detector = FaceDetector.create(detectionmodel)

Detect faces

detections = detector.detect( imagepath="assets/images/img1.jpg", # Path to the image to detect faces in savejson=True, # Save the detections to a JSON file jsonpath=f"output/{detectionmodel}/detections.json", # Path to save the JSON file saveannotated=True, # Save the annotated image outputfolder=f"output/{detection_model}", # Path to save the annotated image ) ```

Example

python python examples/face_detection/basic_detection_single_image.py --detection_model mediapipe

Output Annotated Image

python image_name | x1 | y1 | x2 | y2 | confidence img1.jpg | 62 | 228 | 453 | 619 | 0.9381868243217468

Face Reenactment

```python from mukh.reenactment import FaceReenactor

Initialize reenactor

reenactormodel = "tps" # Available models: "tps" reenactor = FaceReenactor.create(reenactormodel)

Reenact face

resultpath = reenactor.reenactfromvideo( sourcepath="assets/images/img1.jpg", # Path to the source image drivingvideopath="assets/videos/video.mp4", # Path to the driving video outputpath=f"output/{reenactormodel}", # Path to save the reenacted video savecomparison=True, # Save the comparison video resizetoimageresolution=False, # Resize the reenacted video to the image resolution ) ```

Example

python python examples/reenactment/basic_reenactment.py \ --reenactor_model tps \ --source_path assets/images/img1.jpg \ --driving_video_path assets/videos/video_1sec.mp4 \ --output_folder output

Output

https://github.com/user-attachments/assets/875ba692-ea78-42e3-9e03-d1f4703930be

Deepfake Detection

Images

```python import torch from mukh.deepfake_detection import DeepfakeDetector

Initialize detector

detectionmodel = "efficientnet" # Other models "resnetinception"

detector = DeepfakeDetector( modelname=detectionmodel, confidencethreshold=0.5, device=torch.device("cuda" if torch.cuda.isavailable() else "cpu"), )

Detect deepfakes in Images

media_path = "assets/images/img1.jpg"

detections, finalresult = detector.detect( mediapath=mediapath, # Path to the media file (image/video) savecsv=True, # Save the detections to a CSV file csvpath=f"output/{detectionmodel}/deepfakedetections.csv", # Path to save the CSV file saveannotated=True, # Save the annotated media outputfolder=f"output/{detectionmodel}", # Path to save the annotated media ) ```

Example

python python examples/deepfake_detection/detection.py \ --detection_model resnet_inception \ --media_path assets/images/img1.jpg \ --confidence_threshold 0.5

Output

python media_name | frame_number | is_deepfake | confidence | model_name img1.jpg | 0 | False | 0.99 | efficientnet

Videos

```python import torch from mukh.deepfake_detection import DeepfakeDetector

Initialize detector

detectionmodel = "efficientnet" # Other models "resnetinception"

detector = DeepfakeDetector( modelname=detectionmodel, confidencethreshold=0.5, device=torch.device("cuda" if torch.cuda.isavailable() else "cpu"), )

Detect deepfakes in Videos

mediapath = "assets/videos/deepfakeelon_musk.mp4"

detections, finalresult = detector.detect( mediapath=mediapath, # Path to the media file (image/video) savecsv=True, # Save the detections to a CSV file csvpath=f"output/{detectionmodel}/deepfakedetections.csv", # Path to save the CSV file saveannotated=True, # Save the annotated media outputfolder=f"output/{detectionmodel}", # Path to save the annotated media num_frames=11, # Number of equally spaced frames for video analysis )

```

Example

python python examples/deepfake_detection/detection.py \ --detection_model resnet_inception \ --media_path assets/videos/deepfake_elon_musk.mp4 \ --confidence_threshold 0.5 \ --num_frames 11

Output

python media_name | frame_number |is_deepfake|confidence| model_name deepfake_elon_musk.mp4 | 0 | True | 0.99 |EfficientNetAutoAttB4 deepfake_elon_musk.mp4 | 43 | True | 0.69 |EfficientNetAutoAttB4 deepfake_elon_musk.mp4 | 86 | False | 0.73 |EfficientNetAutoAttB4 deepfake_elon_musk.mp4 | 172 | True | 0.95 |EfficientNetAutoAttB4 deepfake_elon_musk.mp4 | 215 | True | 0.98 |EfficientNetAutoAttB4 deepfake_elon_musk.mp4 | 129 | True | 0.96 |EfficientNetAutoAttB4 deepfake_elon_musk.mp4 | 258 | True | 0.53 |EfficientNetAutoAttB4 deepfake_elon_musk.mp4 | 301 | True | 0.77 |EfficientNetAutoAttB4 deepfake_elon_musk.mp4 | 344 | False | 0.83 |EfficientNetAutoAttB4 deepfake_elon_musk.mp4 | 387 | True | 0.62 |EfficientNetAutoAttB4 deepfake_elon_musk.mp4 | 431 | False | 0.79 |EfficientNetAutoAttB4 python | deepfake_elon_musk.mp4 | EfficientNetAutoAttB4 | 8/11 deepfake frames | Final: DEEPFAKE

Deepfake Detection Pipeline

```python from mukh.pipelines.deepfake_detection import PipelineDeepfakeDetection

Define model configurations with weights

modelconfigs = { "resnetinception": 0.5, "efficientnet": 0.5 }

Create ensemble detector

pipeline = PipelineDeepfakeDetection(model_configs)

mediapath = "assets/videos/deepfakeelon_musk.mp4" # Or pass an image path

Detect deepfakes

result = pipeline.detect( mediapath=mediapath, outputfolder="output/deepfakedetectionpipeline", savecsv=True, num_frames=11, # Number of equally spaced video frames for analysis ) ```

Example

python python examples/pipelines/deepfake_detection.py \ --media_path assets/videos/deepfake_elon_musk.mp4 \ --output_folder output/deepfake_detection_pipeline

Output

Ensemble confidence score python frame_number|is_deepfake|confidence 0 | True | 0.5 43 | True | 0.84 86 | True | 0.635 129 | True | 0.98 172 | True | 0.975 215 | True | 0.99 258 | True | 0.765 301 | True | 0.885 344 | True | 0.585 387 | True | 0.81 431 | True | 0.605

Result from the respective models python | deepfake_elon_musk.mp4 | ResNetInception | 10/11 deepfake frames | Final: DEEPFAKE | deepfake_elon_musk.mp4 | EfficientNetAutoAttB4 | 8/11 deepfake frames | Final: DEEPFAKE

Final Pipeline Output python Final Ensemble Result: DEEPFAKE Deepfake frames: 11/11 Average confidence: 0.7791 Model configurations: { 'resnet_inception': 0.5, 'efficientnet': 0.5 }

Citations

For a detailed list of models, libraries, and datasets used, please refer to CITATIONS.md.

Contact

For questions and feedback, please open an issue on GitHub.