{ "info": { "author": "Liangchen Luo", "author_email": "luolc.witty@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Science/Research", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Scientific/Engineering :: Artificial Intelligence" ], "description": "# AdaBound\n[![PyPI - Version](https://img.shields.io/pypi/v/adabound.svg?style=flat)](https://pypi.org/project/adabound/)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/adabound.svg)](https://pypi.org/project/adabound/)\n[![PyPI - Wheel](https://img.shields.io/pypi/wheel/adabound.svg?style=flat)](https://pypi.org/project/adabound/)\n[![GitHub - LICENSE](https://img.shields.io/github/license/Luolc/AdaBound.svg?style=flat)](./LICENSE)\n\nAn optimizer that trains as fast as Adam and as good as SGD, for developing state-of-the-art \ndeep learning models on a wide variety of pupolar tasks in the field of CV, NLP, and etc.\n\nBased on Luo et al. (2019). \n[Adaptive Gradient Methods with Dynamic Bound of Learning Rate](https://openreview.net/forum?id=Bkg3g2R9FX).\nIn *Proc. of ICLR 2019*.\n\n

\n \n

\n\n## Quick Links\n\n- [Website](https://www.luolc.com/publications/adabound/)\n- [Demos](./demos)\n\n## Installation\n\nAdaBound requires Python 3.6.0 or later.\nWe currently provide PyTorch version and AdaBound for TensorFlow is coming soon.\n\n### Installing via pip\n\nThe preferred way to install AdaBound is via `pip` with a virtual environment.\nJust run \n```bash\npip install adabound\n```\nin your Python environment and you are ready to go!\n\n### Using source code\n\nAs AdaBound is a Python class with only 100+ lines, an alternative way is directly downloading\n[adabound.py](./adabound/adabound.py) and copying it to your project.\n\n## Usage\n\nYou can use AdaBound just like any other PyTorch optimizers.\n\n```python3\noptimizer = adabound.AdaBound(model.parameters(), lr=1e-3, final_lr=0.1)\n```\n\nAs described in the paper, AdaBound is an optimizer that behaves like Adam at the beginning of\ntraining, and gradually transforms to SGD at the end.\nThe `final_lr` parameter indicates AdaBound would transforms to an SGD with this learning rate.\nIn common cases, a default final learning rate of `0.1` can achieve relatively good and stable\nresults on unseen data.\nIt is not very sensitive to its hyperparameters.\nSee Appendix G of the paper for more details.\n\nDespite of its robust performance, we still have to state that, **there is no silver bullet**.\nIt does not mean that you will be free from tuning hyperparameters once using AdaBound.\nThe performance of a model depends on so many things including the task, the model structure,\nthe distribution of data, and etc.\n**You still need to decide what hyperparameters to use based on your specific situation,\nbut you may probably use much less time than before!**\n\n## Demos\n\nThanks to the awesome work by the GitHub team and the Jupyter team, the Jupyter notebook (`.ipynb`)\nfiles can render directly on GitHub.\nWe provide several notebooks (like [this one](./demos/cifar10/visualization.ipynb)) for better\nvisualization.\nWe hope to illustrate the robust performance of AdaBound through these examples.\n\nFor the full list of demos, please refer to [this page](./demos).\n\n## Citing\nIf you use AdaBound in your research, please cite [Adaptive Gradient Methods with Dynamic Bound of Learning Rate](https://openreview.net/forum?id=Bkg3g2R9FX).\n```text\n@inproceedings{Luo2019AdaBound,\n author = {Luo, Liangchen and Xiong, Yuanhao and Liu, Yan and Sun, Xu},\n title = {Adaptive Gradient Methods with Dynamic Bound of Learning Rate},\n booktitle = {Proceedings of the 7th International Conference on Learning Representations},\n month = {May},\n year = {2019},\n address = {New Orleans, Louisiana}\n}\n```\n\n## License\n[Apache 2.0](./LICENSE)\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/Luolc/AdaBound", "keywords": "machine learning,deep learning", "license": "Apache", "maintainer": "", "maintainer_email": "", "name": "adabound", "package_url": "https://pypi.org/project/adabound/", "platform": "", "project_url": "https://pypi.org/project/adabound/", "project_urls": { "Homepage": "https://github.com/Luolc/AdaBound" }, "release_url": "https://pypi.org/project/adabound/0.0.5/", "requires_dist": [ "torch (>=0.4.0)" ], "requires_python": ">=3.6.0", "summary": "AdaBound optimization algorithm, build on PyTorch.", "version": "0.0.5" }, "last_serial": 4868030, "releases": { "0.0.2": [ { "comment_text": "", "digests": { "md5": "29fe38ea89df2aa1102ddb7126013880", "sha256": "d9301f7d1e70baf393ddf9252604fc04f35f0f264f0b901c463c46230a9cc772" }, "downloads": -1, "filename": "adabound-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "29fe38ea89df2aa1102ddb7126013880", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6.0", "size": 3273, "upload_time": "2019-02-19T05:59:36", "url": "https://files.pythonhosted.org/packages/01/a4/0a98e4a635a96b32118e8bd16b29e2b6a5a597f0945c2d9af64d7a41c169/adabound-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "041588d67280062ff6bc0235cb73ae50", "sha256": "2baed515e810531c88d5fa34647d1afe7c977a0005257b2f0606ca7039d5921a" }, "downloads": -1, "filename": "adabound-0.0.2.tar.gz", "has_sig": false, "md5_digest": "041588d67280062ff6bc0235cb73ae50", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 2727, "upload_time": "2019-02-19T05:59:38", "url": "https://files.pythonhosted.org/packages/d6/f3/3787975fca281dd981f1f7f13e760ba278c261a45a86cf38ed7968c140af/adabound-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "5237120dc2a82bcabd5453ef076a4ec1", "sha256": "4580ecb6113b5e3af575a7bb31e8a5d7df9b14bccc02d5f935b86031b9601c01" }, "downloads": -1, "filename": "adabound-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "5237120dc2a82bcabd5453ef076a4ec1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6.0", "size": 4407, "upload_time": "2019-02-19T07:22:41", "url": "https://files.pythonhosted.org/packages/a4/e0/44cdbf008b8b723f0bb0fc2df6c5da10a486fcfa478a76f093bde12dcf3d/adabound-0.0.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8af200ca52ffa421298d20a1192d1e61", "sha256": "10c32713e33989272d7a8b5da976b1dcd56dab2f6e2a3b494aa8e209db22d83b" }, "downloads": -1, "filename": "adabound-0.0.3.tar.gz", "has_sig": false, "md5_digest": "8af200ca52ffa421298d20a1192d1e61", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 4126, "upload_time": "2019-02-19T07:22:43", "url": "https://files.pythonhosted.org/packages/b7/03/ad3af3492a82ee7af73c61ac25c24cce7ca915e7c2e3e77687acb71c9f17/adabound-0.0.3.tar.gz" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "365e0797d15a5f6663d7cf170147be91", "sha256": "725d74ed7dc158517e8bd258353ab19761203a2bbf51b41210df13e101d918eb" }, "downloads": -1, "filename": "adabound-0.0.4-py3-none-any.whl", "has_sig": false, "md5_digest": "365e0797d15a5f6663d7cf170147be91", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6.0", "size": 4579, "upload_time": "2019-02-20T03:41:25", "url": "https://files.pythonhosted.org/packages/3a/36/4063039c386bceebccad8b7d2201e94fdc975eb79c9eae8853d73fcbbb05/adabound-0.0.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "cf3887ac51b34da8e947b71fd46fff3e", "sha256": "ed9b78ec4ec695561ad307d2b6667b5912a1f391cdc94552edf390a9fbb9f302" }, "downloads": -1, "filename": "adabound-0.0.4.tar.gz", "has_sig": false, "md5_digest": "cf3887ac51b34da8e947b71fd46fff3e", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 4340, "upload_time": "2019-02-20T03:41:28", "url": "https://files.pythonhosted.org/packages/96/d0/1b0d5ef6592fca14c5fd3a0a29c12bd598a4909d826acec11c1c2d00bfbc/adabound-0.0.4.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "916abffde01bdbbff6de6f475baac7c1", "sha256": "0e66e4ef3c913ce4f938527d7391cf30ab5e31678e932d5ebaa80f41311f8b1c" }, "downloads": -1, "filename": "adabound-0.0.5-py3-none-any.whl", "has_sig": false, "md5_digest": "916abffde01bdbbff6de6f475baac7c1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6.0", "size": 5066, "upload_time": "2019-02-26T04:23:45", "url": "https://files.pythonhosted.org/packages/cd/44/0c2c414effb3d9750d780b230dbb67ea48ddc5d9a6d7a9b7e6fcc6bdcff9/adabound-0.0.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2a03eeff67337fcb116e263336a5138c", "sha256": "61c6e7564d64ac4929497fde2190b7919e40415e24f9718997189b102f766e3f" }, "downloads": -1, "filename": "adabound-0.0.5.tar.gz", "has_sig": false, "md5_digest": "2a03eeff67337fcb116e263336a5138c", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 4960, "upload_time": "2019-02-26T04:23:46", "url": "https://files.pythonhosted.org/packages/8f/42/32963579f313c66c0c014a3810cf7215600ca79fa11a424f20d9fe68844b/adabound-0.0.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "916abffde01bdbbff6de6f475baac7c1", "sha256": "0e66e4ef3c913ce4f938527d7391cf30ab5e31678e932d5ebaa80f41311f8b1c" }, "downloads": -1, "filename": "adabound-0.0.5-py3-none-any.whl", "has_sig": false, "md5_digest": "916abffde01bdbbff6de6f475baac7c1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6.0", "size": 5066, "upload_time": "2019-02-26T04:23:45", "url": "https://files.pythonhosted.org/packages/cd/44/0c2c414effb3d9750d780b230dbb67ea48ddc5d9a6d7a9b7e6fcc6bdcff9/adabound-0.0.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2a03eeff67337fcb116e263336a5138c", "sha256": "61c6e7564d64ac4929497fde2190b7919e40415e24f9718997189b102f766e3f" }, "downloads": -1, "filename": "adabound-0.0.5.tar.gz", "has_sig": false, "md5_digest": "2a03eeff67337fcb116e263336a5138c", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 4960, "upload_time": "2019-02-26T04:23:46", "url": "https://files.pythonhosted.org/packages/8f/42/32963579f313c66c0c014a3810cf7215600ca79fa11a424f20d9fe68844b/adabound-0.0.5.tar.gz" } ] }