🐢 Open-Source Evaluation & Testing library for LLM Agents

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

A Python package to assess and improve fairness of machine learning models.

LangFair is a Python library for conducting use-case level LLM bias and fairness assessments

WEFE: The Word Embeddings Fairness Evaluation Framework. WEFE is a framework that standardizes the bias measurement and mitigation in Word Embeddings models. Please feel welcome to open an issue in case you have any questions or a pull request if you want to contribute to the project!

Toolkit for Auditing and Mitigating Bias and Fairness of Machine Learning Systems 🔎🤖🧰

Safety Score for Pre-Trained Language Models

Fairness Aware Machine Learning. Bias detection and mitigation for datasets and models.

PyTorch package to train and audit ML models for Individual Fairness

👋 Influenciae is a Tensorflow Toolbox for Influence Functions

[ACL 2020] Towards Debiasing Sentence Representations

[ICML 2021] Towards Understanding and Mitigating Social Biases in Language Models

[ACM 2024] Jurity: Fairness & Evaluation Library

A tool for gender bias identification in text. Part of Microsoft's Responsible AI toolbox.

Credo AI Lens is a comprehensive assessment framework for AI systems. Lens standardizes model and data assessment, and acts as a central gateway to assessments created in the open source community.

A fairness library in PyTorch.

Counterfactual Local Explanations of AI systems

SIREN: A Simulation Framework for Understanding the Effects of Recommender Systems in Online News Environments

PyTorch reimplementation of computing Shapley values via Truncated Monte Carlo sampling from "What is your data worth? Equitable Valuation of Data" by Amirata Ghorbani and James Zou [ICML 2019]

Package for evaluating the performance of methods which aim to increase fairness, accountability and/or transparency