{
    "info": {
        "author": "A Ghosh",
        "author_email": "",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "## Permutation Importance Physics\n\nThis package helps calculate permutation importance of features for a given model based on a given metric to measure performance. The idea is to assess the performance of a _trained model_ based on a given performance metric if one input feature were taking away (values of the feature are shuffled between samples).\n\nResults are usually more meaningful than _feature importance_ provided by Boosted Decision Trees, can be avalauted for other models such as Neural Networks, provide an uncertainty measure, and allow more flexibility to choose evaluation metric as well as test dataset. It is faster to compute compared to _iterative removal_ method because there is no re-training.\n\nThis package handles 'sample_weights', custom performance metrics, and provides some predefined High Energy Physics based metrics.  It can be used to evaluate feature importance on a given dataset that is not necessarily from the same distribution as the training set, so the feature importances of a trained model can be recalculated for a new test dataset, and/or evaluation metric. This might be useful to test sensitivity to systematic shifts (domain adaptation), or just the impact of features for particular subsets of the dataset (samples with 1 jet, 2 jets, samples with a score > 0.6, signal at mass 700 GeV, 800 GeV, etc). It can also handle multiple input matrices as long as they are combined in the form of a list `X_eval` (if your evaluation metric requires systematics up, systematics down datasets for example).\n\n>WARNING: Choosing the right metric is essential to get meaningful results. Make sure to check if the value of PI for your given features makes sense. If 'discovery significance' is your metric (which usually ranges between 0 and 6), a permutation importance of 112 for a particular feature should worry you.\n\nWhen in doubt, use 'AUC' as a reasonable metric for a classification problem, rather than 'accuracy'.\n\n\n>WARNING: With random forrests or DNN with dropouts, the PI for 2 correlated features might be 0 because dropping any one individually does not hamper the performance of the model, however dropping both might decrease performance. In this package the PI is calculated by dropping only 1 feature at a time for now. Future versions might provide an option to calculate PI taking into account correlations, if there is demonstrated interest.\n\n### Quick tutorial\n    pip install PermutationImportancePhysics\n\nIn `Python3`\n\n    from permutationimportancephysics.PermutationImportance import PermulationImportance\n    pi = PermulationImportance(model=bdt, X=X_test,y=y_test,weights=weights_test,n_iterations=3,usePredict_poba=True,\n                          scoreFunction=\"AUC\")\n    pi.dislayResults()\n\nOr for discovery significance\n\n    pi = PermulationImportance(model=bdt, X=X_test,y=y_test,weights=weights_test,n_iterations=3,usePredict_poba=True,\n                           scoreFunction=\"amsasimov\")\n    pi.dislayResults()\n\nPlot feature importances with error bars\n\n    plt = pi.plotBars()\n    plt.show()\n\n`n_iterations(default=3)`:  number of times the permutation importance of a feature is calculated after a new shuffle. Higher => smaller uncertainty, more computation time.\n`usePredict_poba(default=False)`: use `model.predict_proba()` instead of `model.predict()`, useful for SKLearn models.\n`scoreFunction(default='AUC')`: evaluation metric used to calculate permutation importance over the entire evaluation dataset. User defined function possible of the form: `func (X_eval, y_true, weights)`.\n\n### Dependencies:\n- numpy\n- matplotlib\n- sklearn\n\n### ToDo:\n\n- Multiprocessing\n- AUC with negative weight handling\n- Add more physics metrics (significance with systematics, interference)\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/aghoshpub/permutationImportancePhysics",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "PermutationImportancePhysics",
        "package_url": "https://pypi.org/project/PermutationImportancePhysics/",
        "platform": "",
        "project_url": "https://pypi.org/project/PermutationImportancePhysics/",
        "project_urls": {
            "Homepage": "https://github.com/aghoshpub/permutationImportancePhysics"
        },
        "release_url": "https://pypi.org/project/PermutationImportancePhysics/0.111/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Permutation Importance for Physics",
        "version": "0.111"
    },
    "last_serial": 5672045,
    "releases": {
        "0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f5f59e303b0bb1809a64b89ca8729072",
                    "sha256": "225fa341278df23796c219466234eca6181068e25e0d44820738391bbee7b1b9"
                },
                "downloads": -1,
                "filename": "PermutationImportancePhysics-0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f5f59e303b0bb1809a64b89ca8729072",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 4037,
                "upload_time": "2019-07-25T18:04:32",
                "url": "https://files.pythonhosted.org/packages/28/85/0ae5545743d975004b79f6e5806f33ed85e733cc0507fad8379f2f68ba89/PermutationImportancePhysics-0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "21ed0c15bcb8321bfaee1cea41eb741d",
                    "sha256": "63a0543900eea75bf7b23c3964175c768bcd76349bf967ea8695acb13809830b"
                },
                "downloads": -1,
                "filename": "PermutationImportancePhysics-0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "21ed0c15bcb8321bfaee1cea41eb741d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2273,
                "upload_time": "2019-07-25T17:37:23",
                "url": "https://files.pythonhosted.org/packages/b5/36/e260899fbd09af5389159d481f349a3482ac2b0be68c1a10121876eb674e/PermutationImportancePhysics-0.1.tar.gz"
            }
        ],
        "0.11": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a4dc702274de4c9c3070e8942a321a79",
                    "sha256": "6625ad36f10e4bac7c63612e88199656ca00f6ae9941be4bb943bae4e97471a9"
                },
                "downloads": -1,
                "filename": "PermutationImportancePhysics-0.11-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a4dc702274de4c9c3070e8942a321a79",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 5470,
                "upload_time": "2019-08-13T13:54:04",
                "url": "https://files.pythonhosted.org/packages/7c/1d/96b56ff85093155d9bcc3dbeb9dff5ce22a0b0c28e9ba68f3fb5cc92b55a/PermutationImportancePhysics-0.11-py3-none-any.whl"
            }
        ],
        "0.111": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "14fe07a87f6d1c1b69784bc42be0c6c3",
                    "sha256": "4f6e9b8487107221e7b12e9e515a117ca0f2c8745168bbb054c66b0c9c66e390"
                },
                "downloads": -1,
                "filename": "PermutationImportancePhysics-0.111-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "14fe07a87f6d1c1b69784bc42be0c6c3",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 5625,
                "upload_time": "2019-08-13T14:08:50",
                "url": "https://files.pythonhosted.org/packages/28/02/eeb6f1429b4bd9f16721777f8a110293455bf76fdf6a91c666fd34b759d5/PermutationImportancePhysics-0.111-py3-none-any.whl"
            }
        ],
        "0.12": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e6080f1ea74d5c437f75bf36ea40951f",
                    "sha256": "6caa09e86fbddfcfdc063b5a26987c7c9e41a88810c05777a06dcd3af6cfb736"
                },
                "downloads": -1,
                "filename": "PermutationImportancePhysics-0.12-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "e6080f1ea74d5c437f75bf36ea40951f",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 5611,
                "upload_time": "2019-08-13T14:08:51",
                "url": "https://files.pythonhosted.org/packages/62/f8/fed29884ef35831a204ad3b3bee7c83a113569193d835f1858b6f39286b5/PermutationImportancePhysics-0.12-py3-none-any.whl"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "14fe07a87f6d1c1b69784bc42be0c6c3",
                "sha256": "4f6e9b8487107221e7b12e9e515a117ca0f2c8745168bbb054c66b0c9c66e390"
            },
            "downloads": -1,
            "filename": "PermutationImportancePhysics-0.111-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "14fe07a87f6d1c1b69784bc42be0c6c3",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 5625,
            "upload_time": "2019-08-13T14:08:50",
            "url": "https://files.pythonhosted.org/packages/28/02/eeb6f1429b4bd9f16721777f8a110293455bf76fdf6a91c666fd34b759d5/PermutationImportancePhysics-0.111-py3-none-any.whl"
        }
    ]
}