{
    "info": {
        "author": "Maxmilian Alber, Sebastian Lapuschkin, Miriam Haegele, Kristof Schuett, Philipp Seegerer, Pieter-Jan Kindermans, and others",
        "author_email": "workDoTalberDoTmaximilian@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Environment :: Console",
            "Intended Audience :: Science/Research",
            "License :: OSI Approved :: BSD License",
            "Operating System :: POSIX :: Linux",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3.6",
            "Topic :: Scientific/Engineering :: Artificial Intelligence"
        ],
        "description": "# [iNNvestigate neural networks!](https://github.com/albermax/innvestigate) [![Tweet](https://img.shields.io/twitter/url/http/shields.io.svg?style=social)](https://twitter.com/intent/tweet?text=iNNvestigate%20neural%20networks!&url=https://github.com/albermax/innvestigate&hashtags=iNNvestigate,artificialintelligence,machinelearning,deeplearning,datascience)\n\n\n[![GitHub package version](https://img.shields.io/badge/Version-v1.0.6-green.svg)](https://github.com/albermax/innvestigate)\n[![Keras package version](https://img.shields.io/badge/KerasVersion-v2.2.4-green.svg)](https://github.com/albermax/innvestigate)\n[![License: BSD-2](https://img.shields.io/badge/License-BSD--2-blue.svg)](https://github.com/albermax/innvestigate/blob/master/LICENSE)\n[![Build Status](https://travis-ci.org/albermax/innvestigate.svg?branch=master)](https://travis-ci.org/albermax/innvestigate)\n[![Dev-Build Status](https://travis-ci.org/albermax/innvestigate.svg?branch=develop)](https://travis-ci.org/albermax/innvestigate)))]]\n\n![Different explanation methods on ImageNet.](https://github.com/albermax/innvestigate/raw/master/examples/images/analysis_grid.png)\n\n## Table of contents\n\n* [Introduction](#introduction)\n* [Installation](#installation)\n* [Usage and Examples](#usage-and-examples)\n* [More documentation](#more-documentation)\n* [Contributing](#contributing)\n* [Releases](#releases)\n\n## Introduction\n\nIn the recent years neural networks furthered the state of the art in many domains like, e.g., object detection and speech recognition.\nDespite the success neural networks are typically still treated as black boxes. Their internal workings are not fully understood and the basis for their predictions is unclear.\nIn the attempt to understand neural networks better several methods were proposed, e.g., Saliency, Deconvnet, GuidedBackprop, SmoothGrad, IntergratedGradients, LRP, PatternNet\\&-Attribution.\nDue to the lack of a reference implementations comparing them is a major effort.\nThis library addresses this by providing a common interface and out-of-the-box implementation for many analysis methods.\nOur goal is to make analyzing neural networks' predictions easy!\n\n\n### If you use this code please star the repository and cite the following paper:\n\n**[\"iNNvestigate neural networks!\"](http://arxiv.org/abs/1808.04260)([http://arxiv.org/abs/1808.04260](http://arxiv.org/abs/1808.04260)) by Maximilian Alber, Sebastian Lapuschkin, Philipp Seegerer, Miriam Hägele, Kristof T. Schütt, Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller, Sven Dähne, Pieter-Jan Kindermans**\n\n## Installation\n\niNNvestigate can be installed with the following commands.\nThe library is based on Keras and therefore requires a supported [Keras-backend](https://keras.io/backend/)\no(Currently only the Tensorflow backend is supported. We test with Python 3.6, Tensorflow 1.12 and Cuda 9.x.):\n\n```bash\npip install innvestigate\n# Installing Keras backend\npip install [tensorflow | theano | cntk]\n```\n\nTo use the example scripts and notebooks one additionally needs to install the package matplotlib:\n\n```bash\npip install matplotlib\n```\n\nThe library's tests can be executed via:\n```bash\ngit clone https://github.com/albermax/innvestigate.git\ncd innvestigate\npython setup.py test\n```\n\n## Usage and Examples\n\nThe iNNvestigate library contains implementations for the following methods:\n\n* *function:*\n  * **gradient:** The gradient of the output neuron with respect to the input.\n  * **smoothgrad:** [SmoothGrad](https://arxiv.org/abs/1706.03825) averages the gradient over number of inputs with added noise.\n* *signal:*\n  * **deconvnet:** [DeConvNet](https://arxiv.org/abs/1311.2901) applies a ReLU in the gradient computation instead of the gradient of a ReLU.\n  * **guided:** [Guided BackProp](https://arxiv.org/abs/1412.6806) applies a ReLU in the gradient computation additionally to the gradient of a ReLU.\n  * **pattern.net:** [PatternNet](https://arxiv.org/abs/1705.05598) estimates the input signal of the output neuron.\n* *attribution:*\n  * **input_t_gradient:** Input \\* Gradient\n  * **deep_taylor[.bounded]:** [DeepTaylor](https://www.sciencedirect.com/science/article/pii/S0031320316303582?via%3Dihub) computes for each neuron a rootpoint, that is close to the input, but which's output value is 0, and uses this difference to estimate the attribution of each neuron recursively.\n  * **pattern.attribution:** [PatternAttribution](https://arxiv.org/abs/1705.05598) applies Deep Taylor by searching rootpoints along the singal direction of each neuron.\n  * **lrp.\\*:** [LRP](http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130140) attributes recursively to each neuron's input relevance proportional to its contribution of the neuron output.\n  * **integrated_gradients:** [IntegratedGradients](https://arxiv.org/abs/1703.01365) integrates the gradient along a path from the input to a reference.\n  * **deeplift.wrapper:** [DeepLIFT (wrapper around original code, slower)](http://proceedings.mlr.press/v70/shrikumar17a.html) computes a backpropagation based on \"finite\" gradients.\n* *miscellaneous:*\n  * **input:** Returns the input.\n  * **random:** Returns random Gaussian noise.\n\n**The intention behind iNNvestigate is to make it easy to use analysis methods, but it is not to explain the underlying concepts and assumptions. Please, read the according publication(s) when using a certain method and when publishing please cite the according paper(s) (as well as the [iNNvestigate paper](https://arxiv.org/abs/1808.04260)). Thank you!**\n\nAll the available methods have in common that they try to analyze the output of a specific neuron with respect to input to the neural network.\nTypically one analyses the neuron with the largest activation in the output layer.\nFor example, given a Keras model, one can create a 'gradient' analyzer:\n\n```python\nimport innvestigate\n\nmodel = create_keras_model()\n\nanalyzer = innvestigate.create_analyzer(\"gradient\", model)\n```\n\nand analyze the influence of the neural network's input on the output neuron by:\n\n```python\nanalysis = analyzer.analyze(inputs)\n```\n\nTo analyze a neuron with the index i, one can use the following scheme:\n\n```python\nanalyzer = innvestigate.create_analyzer(\"gradient\",\n                                        model,\n\t\t\t\t\tneuron_selection_mode=\"index\")\nanalysis = analyzer.analyze(inputs, i)\n```\n\nLet's look at an example ([code](https://github.com/albermax/innvestigate/blob/master/examples/readme_code_snippet.py)) with VGG16 and this image:\n\n![Input image.](https://github.com/albermax/innvestigate/raw/master/examples/images/readme_example_input.png)\n\n\n```python\nimport innvestigate\nimport innvestigate.utils\nimport keras.applications.vgg16 as vgg16\n\n# Get model\nmodel, preprocess = vgg16.VGG16(), vgg16.preprocess_input\n# Strip softmax layer\nmodel = innvestigate.utils.model_wo_softmax(model)\n\n# Create analyzer\nanalyzer = innvestigate.create_analyzer(\"deep_taylor\", model)\n\n# Add batch axis and preprocess\nx = preprocess(image[None])\n# Apply analyzer w.r.t. maximum activated output-neuron\na = analyzer.analyze(x)\n\n# Aggregate along color channels and normalize to [-1, 1]\na = a.sum(axis=np.argmax(np.asarray(a.shape) == 3))\na /= np.max(np.abs(a))\n# Plot\nplt.imshow(a[0], cmap=\"seismic\", clim=(-1, 1))\n```\n\n![Analysis image.](https://github.com/albermax/innvestigate/raw/master/examples/images/readme_example_analysis.png)\n\n\n#### Trainable methods\n\nSome methods like PatternNet and PatternAttribution are data-specific and need to be trained.\nGiven a data set with train and test data, this can be done in the following way:\n\n```python\nimport innvestigate\n\nanalyzer = innvestigate.create_analyzer(\"pattern.net\", model)\nanalyzer.fit(X_train)\nanalysis = analyzer.analyze(X_test)\n```\n\n### Tutorials\n\nIn the directory [examples](https://github.com/albermax/innvestigate/blob/master/examples/) one can find different examples as Python scripts and as Jupyter notebooks:\n\n* **[Introduction to iNNvestigate](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/introduction.ipynb):** shows how to use **iNNvestigate**.\n* **[Comparing methods on MNIST](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/mnist_compare_methods.ipynb):** shows how to train and compare analyzers on MNIST.\n* **[Comparing output neurons on MNIST](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/mnist_neuron_selection.ipynb):** shows how to analyze the prediction of different classes on MNIST.\n* **[Comparing methods on ImageNet](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/imagenet_compare_methods.ipynb):** shows how to compare analyzers on ImageNet.\n* **[Comparing networks on ImageNet](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/imagenet_compare_networks.ipynb):** shows how to compare analyzes for different networks on ImageNet.\n* **[Sentiment Analysis](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/sentiment_analysis.ipynb)**.\n* **[Development with iNNvestigate](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/introduction_development.ipynb):** shows how to develop with **iNNvestigate**.\n* **[Perturbation Analysis](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/mnist_perturbation.ipynb)**.\n---\n\n**To use the ImageNet examples please download the example images first ([script](https://github.com/albermax/innvestigate/blob/master/examples/images/wget_imagenet_2011_samples.sh)).**\n\n## More documentation\n\n... can be found here: [https://innvestigate.readthedocs.io/en/latest/](https://innvestigate.readthedocs.io/en/latest/)\n\n## Contributing\n\nIf you would like to contribute or add your analysis method please get in touch with us!\n\n## Releases\n\n[Can be found here.](https://github.com/albermax/innvestigate/blob/master/VERSION.md)\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/albermax/innvestigate",
        "keywords": "",
        "license": "BSD-2",
        "maintainer": "",
        "maintainer_email": "",
        "name": "innvestigate",
        "package_url": "https://pypi.org/project/innvestigate/",
        "platform": "",
        "project_url": "https://pypi.org/project/innvestigate/",
        "project_urls": {
            "Homepage": "https://github.com/albermax/innvestigate"
        },
        "release_url": "https://pypi.org/project/innvestigate/1.0.8/",
        "requires_dist": [
            "future",
            "h5py",
            "keras (==2.2.4)",
            "numpy",
            "pillow",
            "pytest",
            "scipy"
        ],
        "requires_python": "",
        "summary": "A toolbox to innvestigate neural networks' predictions.",
        "version": "1.0.8"
    },
    "last_serial": 4958991,
    "releases": {
        "1.0.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d3a9f44e1d9e16684cc9ec37c1199593",
                    "sha256": "8e078d2d6dddd3c28267cc873eb4b8ea7c908bc1e7901e4f63e14f4d140c1e0a"
                },
                "downloads": -1,
                "filename": "innvestigate-1.0.8-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "d3a9f44e1d9e16684cc9ec37c1199593",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 97372,
                "upload_time": "2019-03-19T14:31:31",
                "url": "https://files.pythonhosted.org/packages/d7/ce/2138b045dce599515eeecc16cfebc48ee89616c073cdec87503a482f9cd6/innvestigate-1.0.8-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "8d8e43e4707c00093442f3b5c7018df5",
                    "sha256": "e3884bcb2ff200c594c2f6cede48553c601d1b0f0699449281f137421eed565d"
                },
                "downloads": -1,
                "filename": "innvestigate-1.0.8.tar.gz",
                "has_sig": false,
                "md5_digest": "8d8e43e4707c00093442f3b5c7018df5",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 73350,
                "upload_time": "2019-03-19T14:31:34",
                "url": "https://files.pythonhosted.org/packages/11/46/90990351c300af9843d9c1a28addb0008bca72fc280960e9badd3af88188/innvestigate-1.0.8.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "d3a9f44e1d9e16684cc9ec37c1199593",
                "sha256": "8e078d2d6dddd3c28267cc873eb4b8ea7c908bc1e7901e4f63e14f4d140c1e0a"
            },
            "downloads": -1,
            "filename": "innvestigate-1.0.8-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "d3a9f44e1d9e16684cc9ec37c1199593",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 97372,
            "upload_time": "2019-03-19T14:31:31",
            "url": "https://files.pythonhosted.org/packages/d7/ce/2138b045dce599515eeecc16cfebc48ee89616c073cdec87503a482f9cd6/innvestigate-1.0.8-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "8d8e43e4707c00093442f3b5c7018df5",
                "sha256": "e3884bcb2ff200c594c2f6cede48553c601d1b0f0699449281f137421eed565d"
            },
            "downloads": -1,
            "filename": "innvestigate-1.0.8.tar.gz",
            "has_sig": false,
            "md5_digest": "8d8e43e4707c00093442f3b5c7018df5",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 73350,
            "upload_time": "2019-03-19T14:31:34",
            "url": "https://files.pythonhosted.org/packages/11/46/90990351c300af9843d9c1a28addb0008bca72fc280960e9badd3af88188/innvestigate-1.0.8.tar.gz"
        }
    ]
}