Examine Equalized Odds of a Predictive Model

This function examines the equalized odds of a predictive model by comparing both the False Negative Rates (FNR) and False Positive Rates (FPR) across different groups defined by a sensitive attribute. It assesses if a model performs unbiasedly for binary outcomes across these groups, adhering to the equalized odds fairness criterion.

Usage

eval_eq_odds(
  data,
  outcome,
  group,
  probs,
  cutoff = 0.5,
  bootstraps = 2500,
  alpha = 0.05,
  digits = 2,
  message = TRUE
)

Arguments

data

A dataframe containing the actual outcomes, predicted outcomes, and sensitive attributes necessary for evaluating model fairness.

outcome

The name of the outcome variable in the data; it must be binary.

group

The name of the sensitive attribute variable used to define groups for comparison in the fairness evaluation.

probs

The name of the variable containing predicted probabilities or scores.

cutoff

The threshold for converting predicted probabilities into binary predictions; defaults to 0.5.

bootstraps

The number of bootstrap samples used for estimating the uncertainty in the fairness metrics; defaults to 1000.

alpha

The 1 - significance level for the confidence interval; defaults to 0.05.

digits

The number of decimal places to which numerical results are rounded; defaults to 2.

message

Logical; whether to print summary results to the console; defaults to TRUE.

Value

Returns a dataframe with the following columns:

• Metric: Describes the metric being reported (FNR and FPR for each group, difference).

• Group1: Rate for the first group.

• Group2: Rate for the second group.

• Difference: The difference in rates between the two groups.

• 95% CI: The 95% confidence interval for the rate difference。

Examples

# Example usage:
eval_eq_odds(
  data = your_data, outcome = "actual_outcome",
  group = "sensitive_attribute", probs = "predicted_probs"
)
#> Error in eval(expr, envir, enclos): object 'your_data' not found