This repository contains code for scoring heatmap based ViT explanations (plausibility, fidelity, consistency) using Grad-CAM–style and related explainers.
src/evaluator/— Three evaluator metrics: plausibility, fidelity (insertion/deletion/faithfulness) and consistency (RIS/ROS/SSIM)src/xai_methods/(gradcam_vit.py,rollout_vit.py,chefer_vit.py) — Heatmap based explainability methods for Vision Transformerssrc/utils/— helper functionsevaluator_workflow_demo.ipynb— demo notebook showing a minimal ViT + heatmap + evaluator flow.
Create a Python environment (recommended python 3.10 or higher) and install dependencies:
pip install -r requirements.txtYou may need to adjust versions for your platform. This code was tested on python 3.10.12.
| File | Purpose |
|---|---|
README.md (this file) |
Release overview and setup |
src/evaluator/README.md |
Evaluator imports and usage (plausibility, fidelity, consistency) |
src/evaluator/plausibility/README.md |
What plausibility metrics measure (IoU, relevance mass/rank) |
src/evaluator/fidelity/README.md |
What fidelity metrics measure (insertion, deletion, faithfulness correlation) |
src/evaluator/consistency/README.md |
What consistency metrics measure (RIS, ROS, SSIM) |
About the Catalog of Regulatory Science Tools
The enclosed tool is part of the Catalog of Regulatory Science Tools, which provides a peer-reviewed resource for stakeholders to use where standards and qualified Medical Device Development Tools (MDDTs) do not yet exist. These tools do not replace FDA-recognized standards or MDDTs. This catalog collates a variety of regulatory science tools that the FDA’s Center for Devices and Radiological Health’s (CDRH) Office of Science and Engineering Labs (OSEL) developed. These tools use the most innovative science to support medical device development and patient access to safe and effective medical devices. If you are considering using a tool from this catalog in your marketing submissions, note that these tools have not been qualified as Medical Device Development Tools and the FDA has not evaluated the suitability of these tools within any specific context of use. You may request feedback or meetings for medical device submissions as part of the Q-Submission Program. For more information about the Catalog of Regulatory Science Tools, email RST_CDRH@fda.hhs.gov.