fairmetrics: Fairness evaluation metrics with confidence intervals

A collection of functions for computing fairness metrics for machine learning and statistical models, including confidence intervals for each metric. The package supports the evaluation of group-level fairness criterion commonly used in fairness research, particularly in healthcare. It is based on the overview of fairness in machine learning written by Gao et al (2024) (https://arxiv.org/abs/2406.09307).

• Link to online tutorial.
• Link to preprint.

Installation

To install the latest CRAN release run:

install.packages("fairmetrics")

To install the package from the Github repository run:

devtools::install_github("jianhuig/fairmetrics")

Citation

To cite package ‘fairmetrics’ in publications use:

Gao J, Smith B, Chou B, Gronsbell J (2025). fairmetrics: Fairness evaluation metrics with confidence intervals. R package version 1.0.0, https://jianhuig.github.io/fairmetrics/.

A BibTeX entry for LaTeX users is

@Manual{,
    title = {fairmetrics: Fairness evaluation metrics with confidence intervals},
    author = {Jianhui Gao and Benjamin Smith and Benson Chou and Jessica Gronsbell},
    year = {2025},
    note = {R package version 1.0.0},
    url = {https://jianhuig.github.io/fairmetrics/},
  }

Similar Works

• fairness R package
• fairmodels R package
• fairlearn Python package

References

1. Gao, J. et al. What is Fair? Defining Fairness in Machine Learning for Health. arXiv.org https://arxiv.org/abs/2406.09307 (2024).