Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Confusion matrix #1524

Closed
beckernick opened this issue Dec 30, 2019 · 1 comment · Fixed by #1817
Closed

[FEA] Confusion matrix #1524

beckernick opened this issue Dec 30, 2019 · 1 comment · Fixed by #1817
Labels
? - Needs Triage Need team to review and classify feature request New feature or request

Comments

@beckernick
Copy link
Member

It would be great to be able to calculate the confusion matrix of the predictions/ground truth of a given estimator. Calculating the confusion matrix from preds/ground truth of size ~3 million can take at least 2.5 seconds on the CPU with sklearn.

Following mention in the text of #242 and #608 , filing an issue to make tracking explicit.

@beckernick beckernick added feature request New feature or request ? - Needs Triage Need team to review and classify labels Dec 30, 2019
@beckernick
Copy link
Member Author

Related to #242 , the following would be a basic 2-class confusion matrix with CuPy:

def cupy_conf_mat(y, y_pred):
    """
    Simple, fast confusion matrix for two class models designed to match sklearn.
    Assumes the classes are one of [0, 1]. It will fail edge cases, which are fairly
    numerous.
    
    Could be expanded to multi-class by following similar logic to the precision implementation,
    but no need for now.
    
    This is about 300x faster than sklearn for 3M float64 predictions.
    """
    nclasses = len(cp.unique(y))
    assert nclasses == 2
    res = cp.zeros((2, 2))
    
    pos_pred_ix = cp.where(y_pred == 1)
    neg_pred_ix = cp.where(y_pred != 1)
    tn_sum = (y[neg_pred_ix] == 0).sum()
    fn_sum = (y[neg_pred_ix] == 1).sum()
    tp_sum = (y[pos_pred_ix] == 1).sum()
    fp_sum = (y[pos_pred_ix] == 0).sum()
    
    res[0, 0] = tn_sum
    res[1, 0] = fn_sum
    res[0, 1] = fp_sum
    res[1, 1] = tp_sum
    return res

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
? - Needs Triage Need team to review and classify feature request New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant