Performance Metric
What is a performance metric?
Performance Metrics are a measure of the model's predictive power, or the quality of the model's predictions. Performance metrics are user determined and defined trust factors that may be added in Certifai to track specific use case concerns.
In order for Certifai to calculate values for identified performance metrics the dataset used must contain an "outcome value" feature.
In addition to the main Certifai reports, you can optionally identify one or more additional performance metrics.
For the Banking:Loan Approval use case the optional performance metric configured is Accuracy.
If you have identified more than one performance metric you must identify which one to use in the ATX (AI Trust Index) calculation.
Supported Performance Metric
Certifai can calculate the following metrics as part of the Performance Evaluation:
- Accuracy
- Recall
- Precision
- F1
- R-Squared
Info
Certifai supports micro
, macro
, and binary
variants of Recall, Precision, and F1. The default variant for each of
these metrics is binary
. To specify a variant of the listed metrics, you should place the variant within parenthesis.
For example, Recall(micro)
specifies the micro variant of Recall, while Recall
alone is equivalent to the the binary
variant of Recall.
Other performance metrics can be included as part of the Performance Evaluation, but each model in the evaluation must specify the corresponding metric value in the scan definition.