How to compute A’ or AUC without ROC curve?

A’ is the probability that if you give a model an example from each category, it will accurately identify which is which. It is the percentage of time your model can differentiate between the negative and positive.

Given here is a small example of how to calculate AUC of a model without actually drawing the curve. The example given here has been done manually and is only for illustration purpose. AUC can be automatically computed using most software packages.

Let us say we have an output from a model as depicted here:

Sample data from Predictive Model

Sample data from Predictive Model

On summarizing the positive and negatives in Column A (Data) we get the following :

Y = 12; N = 13

Where Y = Total number of positives and N = Total Number of negatives.

AUC is given by the following formula. Here S j is the rank of the jth negative  element in sequence and Sj – j is the number of positives before that jth negative element.

eqn

Steps for calculation of A’ are given below:

Step 1: Sort the dataset in decreasing order of magnitude on column B.
Step 2: For every negative entry in the sorted data set computer the total number of positive entries above it as shown below:
Step 3: calculate the sum of all the items calculated in step above. It comes out to be 144 in this example.
Step 4:  Total Number of Positives, Y = 12 and Total Number of negatives, N = 13 as shown in the data summary.
Step 5: Compute AUC as follows: AUC = (Sum of all items in Step 3) / Y * N = 144/ 12* 13 = 0.92

A' computation

A’ computation

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s