Precision vs Recall

Sources:

• Precision vs Recall (Shurti Saxena)

• Identification of Similar and Complementary Subparts in B-Rep Mechanical Models

• Beyond Accuracy: Precision and Recall (Will Koehrsen)

Overview

• Accuracy expresses the percentage of results correctly classified.

• Precision means the percentage of your results that are relevant.

• Recall refers to the percentage of total relevant results correctly classified by your algorithm.

The trade-off

We can see when our precision is 1.0 (no FP), our recall remains very low because we still have many FN. If we go to the other extreme and classify all inputs as negatives, we will have a recall of 1.0 but our precision will be very low and we’ll detain many innocent individuals. In other words, as we increase precision we decrease recall and vice-versa.

Depending on the situation, we may maximize either precision (i.e. spam detection) or recall (i.e. disease detection).

The confusion matrix is useful for quickly calculating precision and recall given the predicted labels from a model. The other main visualization technique for showing the performance of a classification model is the Receiver Operating Characteristic (ROC) curve.

Combining Precision and Recall

If we want to create a balanced classification model with the optimal balance of recall and precision, then we try to maximize the F1 score.

In cases where we want to find an optimal blend of precision and recall we can combine the two metrics using what is called the F1 score. It is the harmonic mean of precision and recall taking both metrics into account.

Metrics

We use the harmonic mean instead of a simple average because it punishes extreme values. A classifier with a precision of 1.0 and a recall of 0.0 has a simple average of 0.5 but an F1 score of 0.

How does a ROC curve work?