Object detection

Giskard is an open-source framework for testing all ML models, from LLMs to tabular models. Don’t hesitate to give the project a star on GitHub ⭐️ if you find it useful!

In this notebook, you’ll learn how to create comprehensive test suites for your model in a few lines of code, thanks to Giskard’s open-source Python library.

Use-case:

• Object detection

• Model:

  • HF Face detection model (https://huggingface.co/goshiv/detr-finetuned-face)

• Dataset:

  • 300W dataset (https://ibug.doc.ic.ac.uk/resources/300-W/)

Outline:

• Detect vulnerabilities automatically with Giskard’s scan

Warning Running this notebook will require downloading data from Hugging Face (3GB), and training a model.

Install dependencies

To run the scan on a vision model, you would need to install both the giskard-vision and the giskard library.

%pip install giskard giskard-vision

Needed imports

from giskard_vision.object_detection.dataloaders.loaders import DataLoader300WFaceDetection
from giskard_vision.object_detection.models.wrappers import DetrFinetunedHFModel

from giskard_vision.core.scanner import scan

Load model and dataset

In this example, we load the demo wrapper for a HF Face detection model and the 300W dataset that comes with the library.

dataloader = DataLoader300WFaceDetection(
    dir_path="path-to-giskard-vision/examples/landmark_detection/datasets/300W/sample"
)
model = DetrFinetunedHFModel()

Detect vulnerabilities in the model

Giskard’s scan allows you to detect vulnerabilities in your model automatically. On object detection, these include performance biases and unrobustness.

results = scan(model, dataloader)

If you are running in a notebook, you can display the scan report directly in the notebook using display(...), otherwise you can export the report to an HTML file. Check the API Reference for more details on the export methods available on the ScanReport class.

display(results)

# Save it to file
results.to_html("scan_report.html")