Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

🐢 Open-Source Evaluation & Testing for ML & LLM systems

Deliver safe & effective language models

[ICML 2024] TrustLLM: Trustworthiness in Large Language Models

The open-sourced Python toolbox for backdoor attacks and defenses.

Moonshot - A simple and modular tool to evaluate and red-team any LLM application.

🚀 A fast safe reinforcement learning library in PyTorch

[NeurIPS-2023] Annual Conference on Neural Information Processing Systems

A comprehensive toolbox for model inversion attacks and defenses, which is easy to get started.

AI Verify

A toolbox for benchmarking trustworthiness of multimodal large language models (MultiTrust, NeurIPS 2024 Track Datasets and Benchmarks)

[USENIX Security 2025] PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models

A toolkit for tools and techniques related to the privacy and compliance of AI models.

A project to add scalable state-of-the-art out-of-distribution detection (open set recognition) support by changing two lines of code! Perform efficient inferences (i.e., do not increase inference time) and detection without classification accuracy drop, hyperparameter tuning, or collecting additional data.

The official implementation for ICLR23 paper "GNNSafe: Energy-based Out-of-Distribution Detection for Graph Neural Networks"

[ICCV2021 Oral] Fooling LiDAR by Attacking GPS Trajectory

[NeurIPS'24 & ICMLW'24] CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models

[ACM MM22] Towards Robust Video Object Segmentation with Adaptive Object Calibration, ACM Multimedia 2022

Principal Image Sections Mapping. Convolutional Neural Network Visualisation and Explanation Framework

A project to improve out-of-distribution detection (open set recognition) and uncertainty estimation by changing a few lines of code in your project! Perform efficient inferences (i.e., do not increase inference time) without repetitive model training, hyperparameter tuning, or collecting additional data.