{
    "info": {
        "author": "Patrik Purgai",
        "author_email": "purgai.patrik@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3"
        ],
        "description": "# Switching from Adam to SGD\n\n*[Wilson et al. (2018)](https://arxiv.org/pdf/1705.08292.pdf)* shows that \"*the solutions found by adaptive methods generalize worse (often significantly worse) than SGD, even when these solutions have better training performance. These results suggest that practitioners should reconsider the use of adaptive methods to train neural networks.*\"\n\n\"*SWATS from [Keskar & Socher (2017)](https://arxiv.org/pdf/1712.07628.pdf) a high-scoring paper by ICLR in 2018, a method proposed to automatically switch from Adam to SGD for better generalization performance. The idea of the algorithm itself is very simple. It uses Adam, which works well despite minimal tuning, but after learning until a certain stage, it is taken over by SGD.*\"\n\n## Usage\n\nInstalling the package is straightforward with pip directly from this git repository or from pypi with either of the following commands.\n\n```bash\npip install git+https://github.com/Mrpatekful/swats\n```\n\n```bash\npip install pytorch-swats\n```\n\nAfter installation *SWATS* can be used as any other `torch.optim.Optimizer`. The following code snippet serves as a simple overview of how to use the algorithm.\n\n```python\nimport swats\n\noptimizer = swats.SWATS(model.parameters())\ndata_loader = torch.utils.data.DataLoader(...)\n\nfor epoch in range(10):\n    for inputs, targets in data_loader:\n        # deleting the stored grad values\n        optimizer.zero_grad()\n\n        outputs = model(inputs)\n        loss = loss_fn(outputs, targets)\n        loss.backward()\n\n        # performing parameter update\n        optimizer.step()\n```\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/Mrpatekful/swats",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "pytorch-swats",
        "package_url": "https://pypi.org/project/pytorch-swats/",
        "platform": "",
        "project_url": "https://pypi.org/project/pytorch-swats/",
        "project_urls": {
            "Homepage": "https://github.com/Mrpatekful/swats"
        },
        "release_url": "https://pypi.org/project/pytorch-swats/0.1.0/",
        "requires_dist": [
            "torch"
        ],
        "requires_python": "",
        "summary": "PyTorch implementation of SWATS algorithm.",
        "version": "0.1.0"
    },
    "last_serial": 5349532,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e48589788a343376fee977158e9d4d10",
                    "sha256": "11e84ac4eef22e92776f8fd151845cc33fb36ffab903fd2c9514375a1b4bb149"
                },
                "downloads": -1,
                "filename": "pytorch_swats-0.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "e48589788a343376fee977158e9d4d10",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 5263,
                "upload_time": "2019-06-02T18:26:45",
                "url": "https://files.pythonhosted.org/packages/12/64/a759b912da86176447520a39ca06825c599da75716276d92846ea9a6ac99/pytorch_swats-0.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "d8d96136213ecbfd7b2f8bdb73539c02",
                    "sha256": "63a8c0b61f78b96aa57624878f954372d21eea8b0bcea77f1efd2024b1076787"
                },
                "downloads": -1,
                "filename": "pytorch-swats-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "d8d96136213ecbfd7b2f8bdb73539c02",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3895,
                "upload_time": "2019-06-02T18:26:47",
                "url": "https://files.pythonhosted.org/packages/68/31/99090940509e24966dfa34545a93d9e078a5a440e01c1dc2cb06c55e9c88/pytorch-swats-0.1.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "e48589788a343376fee977158e9d4d10",
                "sha256": "11e84ac4eef22e92776f8fd151845cc33fb36ffab903fd2c9514375a1b4bb149"
            },
            "downloads": -1,
            "filename": "pytorch_swats-0.1.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "e48589788a343376fee977158e9d4d10",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 5263,
            "upload_time": "2019-06-02T18:26:45",
            "url": "https://files.pythonhosted.org/packages/12/64/a759b912da86176447520a39ca06825c599da75716276d92846ea9a6ac99/pytorch_swats-0.1.0-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "d8d96136213ecbfd7b2f8bdb73539c02",
                "sha256": "63a8c0b61f78b96aa57624878f954372d21eea8b0bcea77f1efd2024b1076787"
            },
            "downloads": -1,
            "filename": "pytorch-swats-0.1.0.tar.gz",
            "has_sig": false,
            "md5_digest": "d8d96136213ecbfd7b2f8bdb73539c02",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 3895,
            "upload_time": "2019-06-02T18:26:47",
            "url": "https://files.pythonhosted.org/packages/68/31/99090940509e24966dfa34545a93d9e078a5a440e01c1dc2cb06c55e9c88/pytorch-swats-0.1.0.tar.gz"
        }
    ]
}