{ "info": { "author": "Russi Chatterjee", "author_email": "root@ixaxaar.in", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Scientific/Engineering :: Artificial Intelligence" ], "description": "# Decoupled Neural Interfaces Using Synthetic Gradients\n\n[![Build Status](https://travis-ci.org/ixaxaar/pytorch-dni.svg?branch=master)](https://travis-ci.org/ixaxaar/pytorch-dni) [![PyPI version](https://badge.fury.io/py/dni.svg)](https://badge.fury.io/py/pytorch-dni)\n\n\n\n\n\n- [Install](#install)\n - [From source](#from-source)\n- [Architecure](#architecure)\n- [Usage](#usage)\n- [Tasks](#tasks)\n\n\n\nThis is an implementation of [Decoupled Neural Interfaces using Synthetic Gradients, Jaderberg et al.](https://arxiv.org/abs/1608.05343).\n\n## Install\n\n```bash\npip install pytorch-dni\n```\n\n### From source\n\n```\ngit clone https://github.com/ixaxaar/pytorch-dni\ncd pytorch-dni\npip install -r ./requirements.txt\npip install -e .\n```\n\n## Architecure\n\n\n\n## Usage\n\n```python\nfrom dni import DNI\n\n# Custom network, can be anything extending nn.Module\nnet = WhateverNetwork(**kwargs)\nopt = optim.Adam(net.parameters(), lr=0.001)\n\n# use DNI to optimize this network\nnet = DNI(net, optim=opt)\n\n# after that we go about our business as usual\nfor e in range(epoch):\n opt.zero_grad()\n output = net(input, *args)\n loss = criterion(output, target_output)\n loss.backward()\n opt.step()\n\n...\n```\n\n## DNI Networks\n\nThis package ships with 3 types of DNI networks:\n\n- RNN_DNI: stacked `LSTM`s, `GRU`s or `RNN`s\n- Linear_DNI: 2-layer `Linear` modules\n- Linear_Sigmoid_DNI: 2-layer `Linear` followed by `Sigmoid`\n\n## Custom DNI Networks\n\nCustom DNI nets can be created using the `DNI_Network` interface:\n\n```python\nclass MyDNI(DNI_Network):\n def __init__(self, input_size, hidden_size, output_size, **kwargs):\n super(MyDNI, self).__init__(input_size, hidden_size, output_size)\n\n self.net = { ... your custom net }\n ...\n\n def forward(self, input, hidden):\n return self.net(input), None # return (output, hidden), hidden can be None\n\n```\n\n## Tasks\n\nThe tasks included in this project are the same as those in [pytorch-dnc](https://github.com/ixaxaar/pytorch-dnc#tasks), except that they're trained here using DNI.\n\n## Notable stuff\n\n- Using a linear SG module makes the implicit assumption that loss is a quadratic function of the activations\n- For best performance one should adapt the SG module architecture to the loss function used. For MSE linear SG is a reasonable choice, however for log loss one should use architectures including a sigmoid applied pointwise to a linear SG\n\n\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/pypa/pytorch-dni", "keywords": "rnn pytorch", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "pytorch-dni", "package_url": "https://pypi.org/project/pytorch-dni/", "platform": "", "project_url": "https://pypi.org/project/pytorch-dni/", "project_urls": { "Homepage": "https://github.com/pypa/pytorch-dni" }, "release_url": "https://pypi.org/project/pytorch-dni/0.0.1/", "requires_dist": [ "torch", "numpy", "check-manifest; extra == 'dev'", "coverage; extra == 'test'" ], "requires_python": ">=3", "summary": "DNI, for Pytorch", "version": "0.0.1" }, "last_serial": 3500824, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "7643cfd56faa266da5348fe0e2070d23", "sha256": "03105e8d1929c3765bbce61e0da872a4930b29781324ce5042730c9f1cd8cb55" }, "downloads": -1, "filename": "pytorch_dni-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "7643cfd56faa266da5348fe0e2070d23", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 10200, "upload_time": "2018-01-18T13:15:09", "url": "https://files.pythonhosted.org/packages/3f/ef/0e764a8ffeb82930fab400e3a5ad86a0c6d1ade66a51d41a3a9b3471b516/pytorch_dni-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "27226a76a3330b7fd72c229e5ceee049", "sha256": "b19a952cfa036e18bd5f5551811af3f9ed1b9e721c2093f1f0ae325ab23fdf42" }, "downloads": -1, "filename": "pytorch-dni-0.0.1.tar.gz", "has_sig": false, "md5_digest": "27226a76a3330b7fd72c229e5ceee049", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 7442, "upload_time": "2018-01-18T13:15:10", "url": "https://files.pythonhosted.org/packages/58/fa/02761dd4b490724223ba2d5d0b9ee585703108444eeec973f8fc7be3f424/pytorch-dni-0.0.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "7643cfd56faa266da5348fe0e2070d23", "sha256": "03105e8d1929c3765bbce61e0da872a4930b29781324ce5042730c9f1cd8cb55" }, "downloads": -1, "filename": "pytorch_dni-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "7643cfd56faa266da5348fe0e2070d23", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 10200, "upload_time": "2018-01-18T13:15:09", "url": "https://files.pythonhosted.org/packages/3f/ef/0e764a8ffeb82930fab400e3a5ad86a0c6d1ade66a51d41a3a9b3471b516/pytorch_dni-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "27226a76a3330b7fd72c229e5ceee049", "sha256": "b19a952cfa036e18bd5f5551811af3f9ed1b9e721c2093f1f0ae325ab23fdf42" }, "downloads": -1, "filename": "pytorch-dni-0.0.1.tar.gz", "has_sig": false, "md5_digest": "27226a76a3330b7fd72c229e5ceee049", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 7442, "upload_time": "2018-01-18T13:15:10", "url": "https://files.pythonhosted.org/packages/58/fa/02761dd4b490724223ba2d5d0b9ee585703108444eeec973f8fc7be3f424/pytorch-dni-0.0.1.tar.gz" } ] }