{
"info": {
"author": "Philippe Remy",
"author_email": "",
"bugtrack_url": null,
"classifiers": [],
"description": "# Keract: Keras Activations + Gradients\n\n\n[](https://pepy.tech/project/keract)\n[](https://pepy.tech/project/keract)\n\n```bash\npip install keract\n```\n\n*You have just found a way to get the activations (outputs) and gradients for each layer of your Keras model (LSTM, conv nets...).*\n\n
\n \n
\n\n## API\n\n- [get_activations](#get-activations-nodeslayers-outputs-as-numpy-arrays)\n- [display_activations](#display-the-activations-youve-obtained)\n- [display_heatmaps](#display-the-activations-as-a-heatmap-overlaid-on-an-image)\n- [get_gradients_of_trainable_weights](#get-gradients-of-weights)\n- [get_gradients_of_activations](#get-gradients-of-activations)\n- [persist_to_json_file](#persist-activations-to-json)\n- [load_activations_from_json_file](#load-activations-from-json)\n\n### Get activations (nodes/layers outputs as Numpy arrays)\n\n```python\nkeract.get_activations(model, x, layer_names=None, nodes_to_evaluate=None, output_format='simple', nested=False, auto_compile=True)\n```\n\nFetch activations (nodes/layers outputs as Numpy arrays) for a Keras model and an input X.\nBy default, all the activations for all the layers are returned.\n\n- `model`: Keras compiled model or one of ['vgg16', 'vgg19', 'inception_v3', 'inception_resnet_v2',\n 'mobilenet_v2', 'mobilenetv2', ...].\n- `x`: Numpy array to feed the model as input. In the case of multi-inputs, `x` should be of type List.\n- `layer_names`: (optional) Single name of a layer or list of layer names for which activations should be\n returned. It is useful in very big networks when it is computationally expensive to evaluate all the layers/nodes.\n- `nodes_to_evaluate`: (optional) List of Keras nodes to be evaluated.\n- `output_format`: Change the output dictionary key of the function.\n - `simple`: output key will match the names of the Keras layers. For example Dense(1, name='d1') will\n return {'d1': ...}.\n - `full`: output key will match the full name of the output layer name. In the example above, it will\n return {'d1/BiasAdd:0': ...}.\n - `numbered`: output key will be an index range, based on the order of definition of each layer within the model.\n- `nested`: If specified, will move recursively through the model definition to retrieve nested layers. Recursion ends at leaf layers of the model tree or at layers with their name specified in `layer_names`. For example a Sequential model in another Sequential model is considered nested.\n- `auto_compile`: If set to True, will auto-compile the model if needed.\n\nReturns: Dict {layer_name (specified by output_format) -> activation of the layer output/node (Numpy array)}.\n\n*Example*\n\n```python\nimport numpy as np\nfrom tensorflow.keras import Input, Model\nfrom tensorflow.keras.layers import Dense, concatenate\nfrom keract import get_activations\n\n# model definition\ni1 = Input(shape=(10,), name='i1')\ni2 = Input(shape=(10,), name='i2')\n\na = Dense(1, name='fc1')(i1)\nb = Dense(1, name='fc2')(i2)\n\nc = concatenate([a, b], name='concat')\nd = Dense(1, name='out')(c)\nmodel = Model(inputs=[i1, i2], outputs=[d])\n\n# inputs to the model\nx = [np.random.uniform(size=(32, 10)), np.random.uniform(size=(32, 10))]\n\n# call to fetch the activations of the model.\nactivations = get_activations(model, x, auto_compile=True)\n\n# print the activations shapes.\n[print(k, '->', v.shape, '- Numpy array') for (k, v) in activations.items()]\n\n# Print output:\n# i1 -> (32, 10) - Numpy array\n# i2 -> (32, 10) - Numpy array\n# fc1 -> (32, 1) - Numpy array\n# fc2 -> (32, 1) - Numpy array\n# concat -> (32, 2) - Numpy array\n# out -> (32, 1) - Numpy array\n```\n\n### Display the activations you've obtained\n\n```python\nkeract.display_activations(activations, cmap=None, save=False, directory='.', data_format='channels_last', fig_size=(24, 24), reshape_1d_layers=False)\n```\n\nPlot the activations for each layer using matplotlib\n\nInputs are:\n- `activations` dict - a dictionary mapping layers to their activations (the output of get_activations)\n- `cmap` (optional) string - a valid matplotlib colormap to be used\n- `save`(optional) a bool, if True the images of the activations are saved rather than being shown\n- `directory`: (optional) string - where to store the activations (if save is True)\n- `data_format`: (optional) tring - one of \"channels_last\" (default) or \"channels_first\".\n- `reshape_1d_layers`: (optional) bool - tries to reshape large 1d layers to a square/rectangle.\n- `fig_size`: (optional) (float, float) - width, height in inches.\n\nThe ordering of the dimensions in the inputs. \"channels_last\" corresponds to inputs with shape (batch, steps, channels) (default format for temporal data in Keras) while \"channels_first\" corresponds to inputs with shape (batch, channels, steps).\n\n### Display the activations as a heatmap overlaid on an image\n\n```python\nkeract.display_heatmaps(activations, input_image, save=False)\n```\n\nPlot heatmaps of activations for all filters overlayed on the input image for each layer\n\nInputs are:\n- `activations`: a dictionary mapping layers to their activations (the output of get_activations).\n- `input_image`: numpy array of the image you inputed to the get_activations.\n- `save`(optional) bool - if True the images of the activations are saved rather than being shown.\n- `fix`: (optional) bool - if automated checks and fixes for incorrect images should be run.\n- `directory`: string - where to store the activations (if save is True).\n\n### Get gradients of weights\n\n```python\nkeract.get_gradients_of_trainable_weights(model, x, y)\n```\n\n- `model` is a `keras.models.Model` object.\n- `x`: Numpy array to feed the model as input. In the case of multi-inputs, `x` should be of type List.\n- `y`: Labels (numpy array). Keras convention.\n\nThe output is a dictionary mapping each trainable weight to the values of its gradients (regarding x and y).\n\n### Get gradients of activations\n\n```python\nkeract.get_gradients_of_activations(model, x, y, layer_name=None, output_format='simple')\n```\n\n- `model` is a `keras.models.Model` object.\n- `x`: Numpy array to feed the model as input. In the case of multi-inputs, `x` should be of type List.\n- `y`: Labels (numpy array). Keras convention.\n- `layer_name`: (optional) Name of a layer for which activations should be returned.\n- `output_format`: Change the output dictionary key of the function.\n - `simple`: output key will match the names of the Keras layers. For example Dense(1, name='d1') will\n return {'d1': ...}.\n - `full`: output key will match the full name of the output layer name. In the example above, it will\n return {'d1/BiasAdd:0': ...}.\n - `numbered`: output key will be an index range, based on the order of definition of each layer within the model.\n\nReturns: Dict {layer_name (specified by output_format) -> grad activation of the layer output/node (Numpy array)}.\n\nThe output is a dictionary mapping each layer to the values of its gradients (regarding x and y).\n\n### Persist activations to JSON\n\n```python\nkeract.persist_to_json_file(activations, filename)\n```\n\n- `activations`: activations (dict mapping layers)\n- `filename`: output filename (JSON format)\n\n### Load activations from JSON\n\n```python\nkeract.load_activations_from_json_file(filename)\n```\n\n- `filename`: filename to read the activations from (JSON format)\n\nIt returns the activations.\n\n## Examples\n\nExamples are provided for:\n- `keras.models.Sequential` - mnist.py\n- `keras.models.Model` - multi_inputs.py\n- Recurrent networks - recurrent.py\n\nIn the case of MNIST with LeNet, we are able to fetch the activations for a batch of size 128:\n\n```\nconv2d_1/Relu:0\n(128, 26, 26, 32)\n\nconv2d_2/Relu:0\n(128, 24, 24, 64)\n\nmax_pooling2d_1/MaxPool:0\n(128, 12, 12, 64)\n\ndropout_1/cond/Merge:0\n(128, 12, 12, 64)\n\nflatten_1/Reshape:0\n(128, 9216)\n\ndense_1/Relu:0\n(128, 128)\n\ndropout_2/cond/Merge:0\n(128, 128)\n\ndense_2/Softmax:0\n(128, 10)\n```\n\nWe can visualise the activations. Here's another example using VGG16:\n\n```bash\ncd examples\npip install -r examples-requirements.txt\npython vgg16.py\n```\n\n
\n \n A cat.\n
\n\n\n
\n \n Outputs of the first convolutional layer of VGG16.\n
\n\nAlso, we can visualise the heatmaps of the activations:\n\n```bash\ncd examples\npip install -r examples-requirements.txt\npython heat_map.py\n```\n\n
![](\"assets/intro.png\")
\n![](\"assets/cat.jpg\")
\n ![](\"assets/cat_activations.png\")
\n ![](\"assets/heatmap.png\")
\n