Calculate various statistics from a confusion matrix

Given a vector of predictions and target values, calculate numerous statistics of interest.

confusion_matrix(
  prediction,
  target,
  positive = NULL,
  prevalence = NULL,
  return_table = FALSE,
  dnn = c("Predicted", "Target"),
  longer = FALSE,
  ...
)

Arguments

prediction	A vector of predictions
target	A vector of target values
positive	The positive class for a 2-class setting. Default is NULL, which will result in using the first level of target.
prevalence	Prevalence rate. Default is NULL.
return_table	Logical. Whether to return the table of prediction vs. target. Default is FALSE. Cannot have both return_table and longer TRUE.
dnn	The row and column headers for the table returned by return_table. Default is 'Predicted' for rows and 'Target' for columns.
longer	Transpose the output to long form. Default is FALSE (requires tidyr 1.0). Cannot have both return_table and longer TRUE.
...	Other parameters, not currently used.

Value

A list of tibble(s) with the associated statistics and possibly the frequency table as list column of the first element.

Details

This returns accuracy, agreement, and other statistics. See the functions below to find out more. Originally inspired by the confusionMatrix function from the caret package.

References

Kuhn, M., & Johnson, K. (2013). Applied predictive modeling.

See also

confusionMatrix calc_accuracy calc_agreement calc_stats

Examples

library(confusionMatrix)

p = c(0,1,1,0,0,1,0,1,1,1)
o = c(0,1,1,1,0,1,0,1,0,1)

confusion_matrix(p, o, return_table = TRUE, positive = '1')
#> $Accuracy
#> # A tibble: 1 x 6
#>   Accuracy `Accuracy LL` `Accuracy UL` `Accuracy Guess… `Accuracy P-val…
#>      <dbl>         <dbl>         <dbl>            <dbl>            <dbl>
#> 1      0.8         0.444         0.975              0.6            0.167
#> # … with 1 more variable: `Frequency Table` <list>
#> 
#> $Other
#> # A tibble: 1 x 19
#>   Positive     N `N Positive` `N Negative` `Sensitivity/Re… `Specificity/TN…
#>   <chr>    <int>        <int>        <int>            <dbl>            <dbl>
#> 1 1           10            6            4            0.833             0.75
#> # … with 13 more variables: `PPV/Precision` <dbl>, NPV <dbl>, `F1/Dice` <dbl>,
#> #   Prevalence <dbl>, `Detection Rate` <dbl>, `Detection Prevalence` <dbl>,
#> #   `Balanced Accuracy` <dbl>, FDR <dbl>, FOR <dbl>, `FPR/Fallout` <dbl>,
#> #   FNR <dbl>, `D Prime` <dbl>, AUC <dbl>
#> 
#> $`Association and Agreement`
#> # A tibble: 1 x 6
#>   Kappa `Adjusted Rand`  Yule   Phi Peirce Jaccard
#>   <dbl>           <dbl> <dbl> <dbl>  <dbl>   <dbl>
#> 1 0.583           0.286 0.875 0.583  0.583   0.714
#> 

p = sample(letters[1:4], 250, replace = TRUE, prob = 1:4)
o = sample(letters[1:4], 250, replace = TRUE, prob = 1:4)

confusion_matrix(p, o, return_table = TRUE)
#> Warning: Some association metrics may not be
#> calculated due to lack of 2x2 table
#> $Accuracy
#> # A tibble: 1 x 6
#>   Accuracy `Accuracy LL` `Accuracy UL` `Accuracy Guess… `Accuracy P-val…
#>      <dbl>         <dbl>         <dbl>            <dbl>            <dbl>
#> 1    0.324         0.266         0.386             0.42            0.999
#> # … with 1 more variable: `Frequency Table` <list>
#> 
#> $Other
#> # A tibble: 5 x 17
#>   Class     N `Sensitivity/Re… `Specificity/TN… `PPV/Precision`   NPV `F1/Dice`
#>   <chr> <dbl>            <dbl>            <dbl>           <dbl> <dbl>     <dbl>
#> 1 a      27             0.0741            0.897           0.08  0.889    0.0769
#> 2 b      45             0.289             0.780           0.224 0.833    0.252 
#> 3 c      73             0.274             0.706           0.278 0.702    0.276 
#> 4 d     105             0.438             0.662           0.484 0.619    0.46  
#> 5 Aver…  62.5           0.269             0.761           0.267 0.761    0.266 
#> # … with 10 more variables: Prevalence <dbl>, `Detection Rate` <dbl>,
#> #   `Detection Prevalence` <dbl>, `Balanced Accuracy` <dbl>, FDR <dbl>,
#> #   FOR <dbl>, `FPR/Fallout` <dbl>, FNR <dbl>, `D Prime` <dbl>, AUC <dbl>
#> 
#> $`Association and Agreement`
#> # A tibble: 1 x 6
#>    Kappa `Adjusted Rand` Yule  Phi   Peirce Jaccard
#>    <dbl>           <dbl> <lgl> <lgl> <lgl>  <lgl>  
#> 1 0.0394         0.00300 NA    NA    NA     NA     
#>