{
"info": {
"author": "Arno Onken",
"author_email": "asnelt@asnelt.org",
"bugtrack_url": null,
"classifiers": [
"Development Status :: 4 - Beta",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Topic :: Scientific/Engineering"
],
"description": "======================\nmmae Package for Keras\n======================\n\nPackage for multimodal autoencoders with Bregman divergences.\n\n\nDescription\n-----------\n\nThis package contains an implementation of a flexible autoencoder that can\ntake into account the noise distributions of multiple modalities. The\nautoencoder can be used to find a low-dimensional representation of\nmultimodal data, taking advantage of the information that one modality\nprovides about another.\n\nNoise distributions are taken into account by means of Bregman divergences\nwhich correspond to particular exponential families such as Gaussian, Poisson\nor gamma distributions. Each modality can have its own Bregman divergence as\nloss function, thereby assuming a particular output noise distribution.\n\nBy default, the autoencoder network fusing multiple modalities consists of a\nvariable number of ReLU layers that are densely connected. The number of\nlayers and number of units per layer of the encoder and decoder networks are\nsymmetric. Other network architectures can be easily implemented by overriding\nconvenience methods.\n\n\nRequirements\n------------\n\nThe package is compatible with Python 2.7 and 3.x and additionally requires\nNumPy, Six and Keras. It was tested with Python 2.7.15, Python 3.6.6,\nNumPy 1.15.2, Six 1.11.0 and Keras 2.2.4.\n\n\nInstallation\n------------\n\nTo install the mmae package, run::\n\n pip install mmae\n\n\nUsage\n-----\n\nThe main class of this package is ``MultimodalAutoencoder`` which is\nimplemented in the module ``mmae.multimodal_autoencoder``. This class can be\nused to easily construct a multimodal autoencoder for dimensionality reduction.\nThe main arguments for instantiation are ``input_shapes`` which is a list or\ndictionary of shapes for each modality, ``hidden_dims`` which is a list of the\nnumber of units per hidden layer of the encoder, and ``output_activations``\nwhich is a list or dictionary of output activations for each modality.\n\nThe last element of ``hidden_dims`` is the dimensionality of the latent space\nrepresentation. The other elements are mirrored for the decoder construction.\nFor instance, if ``hidden_dims = [128, 64, 8]`` then the encoder will have\nhidden layers with 128 and 64 units and produce an 8 dimensional representation\nwhereas the decoder will take the 8 dimensional representation, feed it into\nhidden layers with 64 and 128 units and produce multimodal outputs with shapes\nfollowing ``input_shapes``.\n\nThe method ``compile`` is used to set the model configuration for training.\nThe main arguments are ``optimizer`` which is a Keras optimizer and ``loss``\nwhich is a list or dictionary of loss functions for each modality. In addition\nto the standard Keras loss functions, regular Bregman divergences can be used.\nCurrent options are ``gaussian_divergence``, ``gamma_divergence``,\n``bernoulli_divergence`` and ``poisson_divergence``, corresponding to Gaussian,\ngamma, Bernoulli and Poisson noise models, respectively. Divergences\ncorresponding to binomial and negative binomial distributions can be used by\ninstantiating ``BinomialDivergence`` and ``NegativeBinomialDivergence``,\nrespectively, and passing the instance in ``loss``. To implement other\ndivergences, additional classes can be derived from ``BregmanDivergence``\nwhere the abstract methods ``_phi`` and ``_phi_gradient`` need to be\noverridden. ``BregmanDivergence`` is implemented in the\n``mmae.bregman_divergences`` module.\n\nThe following code fits a multimodal autoencoder to MNIST, where the images are\ntreated as one modality and the number label is treated as another modality:\n\n.. code-block:: python\n\n from keras.datasets import mnist\n from mmae.multimodal_autoencoder import MultimodalAutoencoder\n # Load example data\n (x_train, y_train), (x_validation, y_validation) = mnist.load_data()\n # Scale pixel values to range [0, 1]\n x_train = x_train.astype('float32') / 255.0\n x_validation = x_validation.astype('float32') / 255.0\n # Multimodal training data\n data = [x_train, y_train]\n # Multimodal validation data\n validation_data = [x_validation, y_validation]\n # Set network parameters\n input_shapes = [x_train.shape[1:], (1,)]\n # Number of units of each layer of encoder network\n hidden_dims = [128, 64, 8]\n # Output activation functions for each modality\n output_activations = ['sigmoid', 'relu']\n # Name of Keras optimizer\n optimizer = 'adam'\n # Loss functions corresponding to a noise model for each modality\n loss = ['bernoulli_divergence', 'poisson_divergence']\n # Construct autoencoder network\n autoencoder = MultimodalAutoencoder(input_shapes, hidden_dims,\n output_activations)\n autoencoder.compile(optimizer, loss)\n # Train model where input and output are the same\n autoencoder.fit(data, epochs=100, batch_size=256,\n validation_data=validation_data)\n\nTo obtain a latent representation of the training data:\n\n.. code-block:: python\n\n latent_data = autoencoder.encode(data)\n\nTo decode the latent representation:\n\n.. code-block:: python\n\n reconstructed_data = autoencoder.decode(latent_data)\n\nEncoding and decoding can also be merged into the following single statement:\n\n.. code-block:: python\n\n reconstructed_data = autoencoder.predict(data)\n\nBy default, the different modalities are fed directly into a densely connected\nfusion network. In order to pre- and post-process each modality, for instance\nusing a convolutional neural network for the image data, the\n``MultimodalAutoencoder`` methods ``_construct_unimodal_encoders`` and\n``_construct_unimodal_decoders`` can be overridden. These methods add networks\nbetween the input and the fusion encoder and between the fusion decoder and the\noutput, respectively.\n\n\nSource code\n-----------\n\nThe source code of the mmae package is hosted on\n`GitHub\n`_.\n\n\nLicense\n-------\n\nCopyright (C) 2018 Arno Onken\n\nThis file is part of the mmae package.\n\nThe mmae package is free software; you can redistribute it and/or modify it\nunder the terms of the GNU General Public License as published by the Free\nSoftware Foundation; either version 3 of the License, or (at your option) any\nlater version.\n\nThe mmae package is distributed in the hope that it will be useful, but WITHOUT\nANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS\nFOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License along with\nthis program; if not, see .\n\n\n",
"description_content_type": "",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/asnelt/mmae/",
"keywords": "autoencoder multimodal deep learning bregman",
"license": "GPLv3+",
"maintainer": "",
"maintainer_email": "",
"name": "mmae",
"package_url": "https://pypi.org/project/mmae/",
"platform": "",
"project_url": "https://pypi.org/project/mmae/",
"project_urls": {
"Homepage": "https://github.com/asnelt/mmae/"
},
"release_url": "https://pypi.org/project/mmae/0.2.0/",
"requires_dist": [
"numpy",
"six",
"keras"
],
"requires_python": "",
"summary": "Package for multimodal autoencoders with Bregman divergences.",
"version": "0.2.0"
},
"last_serial": 4477496,
"releases": {
"0.1.0": [
{
"comment_text": "",
"digests": {
"md5": "5324c2a6011ba28559e1a0cac8a96f88",
"sha256": "975cc7b0701d0955c3dd790f98091fddca5b1d583147fcf5cfd0a445d030eb32"
},
"downloads": -1,
"filename": "mmae-0.1.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "5324c2a6011ba28559e1a0cac8a96f88",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 21439,
"upload_time": "2018-10-05T00:15:21",
"url": "https://files.pythonhosted.org/packages/60/c4/8ed5a539f6bd5c8eae33e2f3caa78d4da9cc5ea7475d7bcab63352791135/mmae-0.1.0-py2.py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "e2322388647c34aa1c7947201c416fdc",
"sha256": "fe27ba0116c1242aa50de821c6a37c55305d7f9e1c5c5f83931ae7485620ce1a"
},
"downloads": -1,
"filename": "mmae-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "e2322388647c34aa1c7947201c416fdc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 21865,
"upload_time": "2018-10-05T00:15:23",
"url": "https://files.pythonhosted.org/packages/ea/7b/d7dc6f9fd6742858a913762a4da853335580e82b0a18abf83f049853fbe8/mmae-0.1.0.tar.gz"
}
],
"0.2.0": [
{
"comment_text": "",
"digests": {
"md5": "4bd78c5d2d4b84a18fa84867312e9805",
"sha256": "7ca08666d56aa944d46cf8f6e1a52e5320321de7ea8ec69c423b8817497e01d9"
},
"downloads": -1,
"filename": "mmae-0.2.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "4bd78c5d2d4b84a18fa84867312e9805",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 24269,
"upload_time": "2018-11-12T14:08:10",
"url": "https://files.pythonhosted.org/packages/fc/b5/7f5bd6c5fedf5ec57ce40ae8f86f37b94ed7f2b571fd959931070e628936/mmae-0.2.0-py2.py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "6c0f8f286963e225d679592642198691",
"sha256": "b6e8856201443ffa0db3b30e4ed3e94436b231374aaa3773c3159614822be75a"
},
"downloads": -1,
"filename": "mmae-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "6c0f8f286963e225d679592642198691",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 25494,
"upload_time": "2018-11-12T14:08:11",
"url": "https://files.pythonhosted.org/packages/f1/92/af416b38529d8e104fd41b69c9cba43456bd1416a5b4c9b3a5287acde47d/mmae-0.2.0.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "4bd78c5d2d4b84a18fa84867312e9805",
"sha256": "7ca08666d56aa944d46cf8f6e1a52e5320321de7ea8ec69c423b8817497e01d9"
},
"downloads": -1,
"filename": "mmae-0.2.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "4bd78c5d2d4b84a18fa84867312e9805",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 24269,
"upload_time": "2018-11-12T14:08:10",
"url": "https://files.pythonhosted.org/packages/fc/b5/7f5bd6c5fedf5ec57ce40ae8f86f37b94ed7f2b571fd959931070e628936/mmae-0.2.0-py2.py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "6c0f8f286963e225d679592642198691",
"sha256": "b6e8856201443ffa0db3b30e4ed3e94436b231374aaa3773c3159614822be75a"
},
"downloads": -1,
"filename": "mmae-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "6c0f8f286963e225d679592642198691",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 25494,
"upload_time": "2018-11-12T14:08:11",
"url": "https://files.pythonhosted.org/packages/f1/92/af416b38529d8e104fd41b69c9cba43456bd1416a5b4c9b3a5287acde47d/mmae-0.2.0.tar.gz"
}
]
}