{
    "info": {
        "author": "Jesse Cai",
        "author_email": "jcjessecai@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3"
        ],
        "description": "# distbelief\nImplementing Google's DistBelief paper.\n\n## Installation/Development instructions\n\nYou'll want to create a python3 virtualenv first by running `make setup`, after which, you should run `make install`. \n\nYou'll then be able to use distbelief by importing distbelief\n```python \n\nfrom distbelief.optim import DownpourSGD\n\noptimizer = DownpourSGD(net.parameters(), lr=0.1, n_push=5, n_pull=5, model=net)\n\n```\n\nAs an example, you can see our implementation running by using the script provided in `example/main.py`.\n\nTo run a 2-training node setup locally, open up three terminal windows, source the `venv` and then run `make first`, `make second`, and `make server`.\nThis will begin training AlexNet on CIFAR10 locally with all default params.\n\n## Benchmarking\n\n**NOTE:** we graph the train/test accuracy of each node, hence node1, node2, node3. A better comparison would be to evaluate the parameter server's params and use that value.\nHowever we can see that the accuracy between the three nodes is fairly consistent, and adding an evaluator might put too much stress on our server. \n\nWe scale the learning rate of the nodes to be learning_rate/freq (.03) .\n\n![train](/docs/train_time.png)\n\n![test](/docs/test_time.png)\n\nWe used AWS c4.xlarge instances to compare the CPU runs, and a GTX 1060 for the GPU run.\n\n## DownpourSGD for PyTorch\n\n### Diagram\n\n<img src=\"./docs/diagram.jpg\" width=\"500\">\n\nHere **2** and **3** happen concurrently. \n\nYou can read more about our implementation [here](https://jcaip.github.io/Distbelief/).\n\n### References\n- [Pytorch distributed tutorial](http://pytorch.org/tutorials/intermediate/dist_tuto.html)\n- [Akka implementation of distbelief](http://alexminnaar.com/implementing-the-distbelief-deep-neural-network-training-framework-with-akka.html)\n- [gevent actor tutorial](http://sdiehl.github.io/gevent-tutorial/#actors)\n- [DistBelief paper](https://static.googleusercontent.com/media/research.google.com/en//archive/large_deep_networks_nips2012.pdf)\n- [Analysis of delayed grad problem](https://openreview.net/pdf?id=BJLSGcywG)\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/ucla-labx/distbelief",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "pytorch-distbelief",
        "package_url": "https://pypi.org/project/pytorch-distbelief/",
        "platform": "",
        "project_url": "https://pypi.org/project/pytorch-distbelief/",
        "project_urls": {
            "Homepage": "https://github.com/ucla-labx/distbelief"
        },
        "release_url": "https://pypi.org/project/pytorch-distbelief/0.1.0/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Distributed training for pytorch",
        "version": "0.1.0"
    },
    "last_serial": 4198108,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "edc999b550dd420d807b2cd556eecf78",
                    "sha256": "e478e7ddbe68d014bc4baaca06bcccceca0c4d592f447ada0ca0b52c00834702"
                },
                "downloads": -1,
                "filename": "pytorch_distbelief-0.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "edc999b550dd420d807b2cd556eecf78",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 6278,
                "upload_time": "2018-08-22T23:19:21",
                "url": "https://files.pythonhosted.org/packages/0e/0c/110aa501aa32573bc2f9a485da7c6ca7eba2b4cf1871b2d70e897723d2ff/pytorch_distbelief-0.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "0684c0733c179a2d5c2e2eb689dc01be",
                    "sha256": "4aeb894824d758181b32539d09ca19af698af7e8ce51e4421ac7fcc970f4f0d9"
                },
                "downloads": -1,
                "filename": "pytorch-distbelief-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "0684c0733c179a2d5c2e2eb689dc01be",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4580,
                "upload_time": "2018-08-22T23:19:22",
                "url": "https://files.pythonhosted.org/packages/e2/ff/dabfd30c3cc70c3c6fd51b19fd095aad5eaa13fe55ca08f370f65842443b/pytorch-distbelief-0.1.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "edc999b550dd420d807b2cd556eecf78",
                "sha256": "e478e7ddbe68d014bc4baaca06bcccceca0c4d592f447ada0ca0b52c00834702"
            },
            "downloads": -1,
            "filename": "pytorch_distbelief-0.1.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "edc999b550dd420d807b2cd556eecf78",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 6278,
            "upload_time": "2018-08-22T23:19:21",
            "url": "https://files.pythonhosted.org/packages/0e/0c/110aa501aa32573bc2f9a485da7c6ca7eba2b4cf1871b2d70e897723d2ff/pytorch_distbelief-0.1.0-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "0684c0733c179a2d5c2e2eb689dc01be",
                "sha256": "4aeb894824d758181b32539d09ca19af698af7e8ce51e4421ac7fcc970f4f0d9"
            },
            "downloads": -1,
            "filename": "pytorch-distbelief-0.1.0.tar.gz",
            "has_sig": false,
            "md5_digest": "0684c0733c179a2d5c2e2eb689dc01be",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 4580,
            "upload_time": "2018-08-22T23:19:22",
            "url": "https://files.pythonhosted.org/packages/e2/ff/dabfd30c3cc70c3c6fd51b19fd095aad5eaa13fe55ca08f370f65842443b/pytorch-distbelief-0.1.0.tar.gz"
        }
    ]
}