{
    "info": {
        "author": "Taylor Denouden",
        "author_email": "taylordenouden@gmail.com",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "# OOD Detection Metrics\n\nFunctions for computing metrics commonly used in the field of out-of-distribution (OOD) detection.\n\n## Installation\n\n`pip install ood-metrics`\n\n## Metrics functions\n\n### AUROC\n\nCalculate and return the area under the ROC curve using unthresholded predictions on the data and a binary true label.\n\n```python\nfrom ood_metrics import auroc\n\nlabels = [0, 0, 0, 1, 0]\nscores = [0.1, 0.3, 0.6, 0.9, 1.3]\n\nprint(auroc(scores, labels))\n# 0.75\n```\n\n### AUPR\n\nCalculate and return the area under the Precision Recall curve using unthresholded predictions on the data and a binary true label.\n\n```python\nfrom ood_metrics import aupr\n\nlabels = [0, 0, 0, 1, 0]\nscores = [0.1, 0.3, 0.6, 0.9, 1.3]\n\nprint(aupr(scores, labels))\n# 0.25\n```\n\n### FPR @ 95% TPR\n\nReturn the FPR when TPR is at least 95%.\n\n```python\nfrom ood_metrics import fpr_at_95_tpr\n\nlabels = [0, 0, 0, 1, 0]\nscores = [0.1, 0.3, 0.6, 0.9, 1.3]\n\nprint(fpr_at_95_tpr(scores, labels))\n# 0.25\n```\n\n### Detection Error\n\nReturn the misclassification probability when TPR is 95%.\n\n```python\nfrom ood_metrics import detection_error\n\nlabels = [0, 0, 0, 1, 0]\nscores = [0.1, 0.3, 0.6, 0.9, 1.3]\n\nprint(detection_error(scores, labels))\n# 0.125\n```\n\n### Calculate all stats\n\nUsing predictions and labels, return a dictionary containing all novelty detection performance statistics.\n\n```python\nfrom ood_metrics import calc_metrics\n\nlabels = [0, 0, 0, 1, 0]\nscores = [0.1, 0.3, 0.6, 0.9, 1.3]\n\nprint(calc_metrics(scores, labels))\n# {\n#     'fpr_at_95_tpr': 0.25,\n#     'detection_error': 0.125,\n#     'auroc': 0.75,\n#     'aupr_in': 0.25,\n#     'aupr_out': 0.94375\n# }\n```\n\n## Plotting functions\n\n### Plot ROC\n\nPlot an ROC curve based on unthresholded predictions and true binary labels.\n\n```python\nfrom ood_metrics import plot_roc\n\nlabels = [0, 0, 0, 1, 0]\nscores = [0.1, 0.3, 0.6, 0.9, 1.3]\n\nplot_roc(scores, labels)\n# Generate Matplotlib AUROC plot\n```\n\n### Plot PR\n\nPlot an Precision-Recall curve based on unthresholded predictions and true binary labels.\n\n```python\nfrom ood_metrics import plot_pr\n\nlabels = [0, 0, 0, 1, 0]\nscores = [0.1, 0.3, 0.6, 0.9, 1.3]\n\nplot_pr(scores, labels)\n# Generate Matplotlib Precision-Recall plot\n```\n\n### Plot Barcode\n\nPlot a visualization showing inliers and outliers sorted by their prediction of novelty.\n\n```python\nfrom ood_metrics import plot_barcode\n\nlabels = [0, 0, 0, 1, 0]\nscores = [0.1, 0.3, 0.6, 0.9, 1.3]\n\nplot_barcode(scores, labels)\n# Shows visualization of sort order of labels occording to the scores.\n```",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/tayden/ood-metrics",
        "keywords": "",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "ood-metrics",
        "package_url": "https://pypi.org/project/ood-metrics/",
        "platform": "",
        "project_url": "https://pypi.org/project/ood-metrics/",
        "project_urls": {
            "Homepage": "https://github.com/tayden/ood-metrics"
        },
        "release_url": "https://pypi.org/project/ood-metrics/0.2.8/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Calculate common OOD detection metrics",
        "version": "0.2.8"
    },
    "last_serial": 5482728,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c6330187837f413fe04250dfc5863a09",
                    "sha256": "999c02bc0b1ec35de1046a765f068e04af8301101b8908b83a00da4e2b09fa4e"
                },
                "downloads": -1,
                "filename": "ood-metrics-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "c6330187837f413fe04250dfc5863a09",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2868,
                "upload_time": "2019-05-20T23:05:01",
                "url": "https://files.pythonhosted.org/packages/b6/50/238f7db260b9421369787b955e1372090f6a7586de706d542c8b5620bd9b/ood-metrics-0.1.0.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7a4eb5209cd0a7252ed20f9d162fe9ed",
                    "sha256": "9b71f32bbf5a3a92884f4b95323c6ef5c4d8549d7ed78cb0fc3eef83d403daf1"
                },
                "downloads": -1,
                "filename": "ood-metrics-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "7a4eb5209cd0a7252ed20f9d162fe9ed",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3588,
                "upload_time": "2019-05-20T23:39:53",
                "url": "https://files.pythonhosted.org/packages/73/bb/28b7d48f5d64b3a99a83ad9f88b4c20e23a8184be3e03336a3d2d288edd8/ood-metrics-0.2.0.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2f1eddb5c6572319f5b620cf7a65cd2c",
                    "sha256": "d5901e0b3c608c01b9a33f4306c8417ae890f21c49bf182dd2e962544612ad90"
                },
                "downloads": -1,
                "filename": "ood-metrics-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "2f1eddb5c6572319f5b620cf7a65cd2c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3645,
                "upload_time": "2019-05-22T02:53:50",
                "url": "https://files.pythonhosted.org/packages/b3/da/e57b29d479d009961c86ce909273ecae258a1c1f32f155dc5b766681cdb5/ood-metrics-0.2.1.tar.gz"
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "24d813f05cd9c1e8826cf21abfc4eb37",
                    "sha256": "97e570475cd30554097e68bc86ad72842f17908cd881a83f8ee0827b7893313c"
                },
                "downloads": -1,
                "filename": "ood-metrics-0.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "24d813f05cd9c1e8826cf21abfc4eb37",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3643,
                "upload_time": "2019-05-22T21:10:15",
                "url": "https://files.pythonhosted.org/packages/c7/40/4d442dac2d8582a4b1dfc646787752c32f937d7ef9b70b265fce9749503b/ood-metrics-0.2.2.tar.gz"
            }
        ],
        "0.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "091a0fc97321c59af30e60e07e281e92",
                    "sha256": "0f355f93b3642cd041909f28a89d0d374aaa791301388fb1516b670407011620"
                },
                "downloads": -1,
                "filename": "ood-metrics-0.2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "091a0fc97321c59af30e60e07e281e92",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3624,
                "upload_time": "2019-05-22T21:14:48",
                "url": "https://files.pythonhosted.org/packages/eb/ba/5d1781a8834a2071496f8accbf82dc8713576b0f462936d88bfc77acb2bc/ood-metrics-0.2.3.tar.gz"
            }
        ],
        "0.2.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3e4d3e8d1b067e5ac88fd7ab8d688caa",
                    "sha256": "1d79ab0aa779bb67286135598291771e66282de74f6659032e8f38de400065c9"
                },
                "downloads": -1,
                "filename": "ood-metrics-0.2.4.tar.gz",
                "has_sig": false,
                "md5_digest": "3e4d3e8d1b067e5ac88fd7ab8d688caa",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3490,
                "upload_time": "2019-07-03T16:39:32",
                "url": "https://files.pythonhosted.org/packages/2a/eb/4c6ae21abcaf98d902a1c7e309603f6ca922b228c413273d53c1b199afa2/ood-metrics-0.2.4.tar.gz"
            }
        ],
        "0.2.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "cdfba2479900d35c8a2a9ba67bbd96ad",
                    "sha256": "a60a7db86a8d7a1efbe567354647332bb35affe28d1b48307b2e8a8cba95c2c7"
                },
                "downloads": -1,
                "filename": "ood-metrics-0.2.5.tar.gz",
                "has_sig": false,
                "md5_digest": "cdfba2479900d35c8a2a9ba67bbd96ad",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3658,
                "upload_time": "2019-07-03T16:53:12",
                "url": "https://files.pythonhosted.org/packages/c8/a3/bfc2ef4c4c7d84753b64c418334beda82e6f317fba029f30d441842ae712/ood-metrics-0.2.5.tar.gz"
            }
        ],
        "0.2.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f2aab006854d01e8ceb2ba0df2beca54",
                    "sha256": "16db41d4357dd4bb36cbbc5dd6b6ccf5ca81f9afc64fd6e1244bd4bd1cf06f44"
                },
                "downloads": -1,
                "filename": "ood-metrics-0.2.6.tar.gz",
                "has_sig": false,
                "md5_digest": "f2aab006854d01e8ceb2ba0df2beca54",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3664,
                "upload_time": "2019-07-03T16:55:09",
                "url": "https://files.pythonhosted.org/packages/52/5a/d6ba9c5ec4687df9f6f82ce115ce6169414f86a6682808ef99816d332cf5/ood-metrics-0.2.6.tar.gz"
            }
        ],
        "0.2.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "93d46261c24f464b90992b8713834244",
                    "sha256": "95c5ad6f01ac6c1dde45cf2ddfbac863088152b3d2188f602e8f7d8e196e5370"
                },
                "downloads": -1,
                "filename": "ood-metrics-0.2.7.tar.gz",
                "has_sig": false,
                "md5_digest": "93d46261c24f464b90992b8713834244",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4452,
                "upload_time": "2019-07-03T17:03:15",
                "url": "https://files.pythonhosted.org/packages/75/98/90b995db1306c52df9bf7a2afa51a2d9d91fb9cd1dba3b63e6de3414dee3/ood-metrics-0.2.7.tar.gz"
            }
        ],
        "0.2.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ba16da1250d3f354c761a66ecb2195cd",
                    "sha256": "25fc8a91c18d48d31a956035253b7a3589ef8f15242cb242b985d952e1aa706e"
                },
                "downloads": -1,
                "filename": "ood-metrics-0.2.8.tar.gz",
                "has_sig": false,
                "md5_digest": "ba16da1250d3f354c761a66ecb2195cd",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4430,
                "upload_time": "2019-07-03T17:10:51",
                "url": "https://files.pythonhosted.org/packages/15/75/fa7cf9b2f00cbd5b3b2b6e1a6134adf8e4d2dafda6a5a0b8acfd13169dcf/ood-metrics-0.2.8.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "ba16da1250d3f354c761a66ecb2195cd",
                "sha256": "25fc8a91c18d48d31a956035253b7a3589ef8f15242cb242b985d952e1aa706e"
            },
            "downloads": -1,
            "filename": "ood-metrics-0.2.8.tar.gz",
            "has_sig": false,
            "md5_digest": "ba16da1250d3f354c761a66ecb2195cd",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 4430,
            "upload_time": "2019-07-03T17:10:51",
            "url": "https://files.pythonhosted.org/packages/15/75/fa7cf9b2f00cbd5b3b2b6e1a6134adf8e4d2dafda6a5a0b8acfd13169dcf/ood-metrics-0.2.8.tar.gz"
        }
    ]
}