{
"info": {
"author": "Yuji Kanagawa",
"author_email": "yuji.kngw.80s.revive@gmail.com",
"bugtrack_url": null,
"classifiers": [
"Development Status :: 3 - Alpha",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: Apache Software License",
"Natural Language :: English",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: Implementation :: CPython"
],
"description": "\n# Rainy\n[](https://travis-ci.org/kngwyu/Rainy)\n[](https://pypi.org/project/rainy/)\n\nReinforcement learning utilities and algrithm implementations using PyTorch.\n\n## API documentation\nCOMING SOON\n\n## Supported python version\nPython >= 3.6.1\n\n## Run examples\nThough this project is still WIP, all examples are verified to work.\n\nFirst, install [pipenv](https://pipenv.readthedocs.io/en/latest/).\nE.g. you can install it via\n``` bash\npip3 install pipenv --user\n```\n\nThen, clone this repository and create a virtual environment in it.\n```bash\ngit clone https://github.com/kngwyu/Rainy.git\ncd Rainy\npipenv --site-packages --three install\n```\n\nNow you are ready to start!\n\n```bash\npipenv run python examples/acktr_cart_pole.py train\n```\n\nAfter training, you can run learned agents.\n\nPlease replace `(log-directory)` in the below command with your real log directory.\nIt should be named like `acktr_cart_pole-190219-134651-35a14601`.\n``` bash\npipenv run python acktr_cart_pole.py eval (log-directory) --render\n```\n\nYou can also plot training results in your log directory.\nThis command opens an ipython shell with your log file.\n``` bash\npipenv run python -m rainy.ipython\n```\nThen you can plot training rewards via\n```python\nlog = open_log('log-directory')\nlog.plot_reward(12 * 20, max_steps=int(4e5), title='ACKTR cart pole')\n```\n\n\n## Run examples with MPI or NCCL\nDistributed training is supported via [horovod](https://horovod.readthedocs.io/en/latest/).\nI recommend installing it in your `site-packages`(= not in the virtualenv).\nE.g., if you want to use it with NCLL, you can install it via\n```bash\nsudo env HOROVOD_GPU_ALLREDUCE=NCCL pip3 install --no-cache-dir horovod\n```\n\nOnce you install it, you can run the training script using `horovodrun` command.\n\nE.g., if you want to use two hosts(localhost and anotherhost) and run `ppo_atari.py`, use\n```bash\nhorovodrun -np 2 -H localhost:1,anotherhost:1 pipenv run python examples/ppo_atari.py train\n```\n\n## Override configuration from CLI\nCurrently, Rainy provides an easy-to-use CLI via [click](https://palletsprojects.com/p/click/).\nYou can view its usages by, say,\n```bash\npipenv run python examples/a2c_cart_pole.py --help\n```\n\nThis CLI has a simple data-driven interface.\nI.e., once you fill a config object, then all commands(train, eval, retarain, and etc.) work.\nSo you can start experiments easily without copying and pasting, say, argument parser codes.\n\nHowever, it has a limitation that you cannot add new options.\n\nSo Rainy-CLI provides an option named `override`, which executes the given string as a Python code\nwith the config object set as `config`.\n\nExample usage:\n```bash\npipenv run python examples/a2c_cart_pole.py --override='config.grad_clip=0.5; config.nsteps=10' train\n```\n\nIf this feature still doesn't satisfy your requirement, you can\n[override subcommands by `ctx.invoke`](https://click.palletsprojects.com/en/7.x/advanced/#invoking-other-commands).\n\n## Implementation Status\n\n|**Algorithm** |**Multi Worker(Sync)**|**Recurrent** |**Discrete Action** |**Continuous Action**|**MPI** |\n| ------------ | -------------------- | ------------------------------ | ------------------ | ------------------- | ---------------- |\n|DQN/Double DQN|:x: |:x: |:heavy_check_mark: |:x: |:x: |\n|DDPG |:x: |:x: |:x: |:heavy_check_mark: |:x: |\n|TD3 |:x: |:x: |:x: |:heavy_check_mark: |:x: |\n|PPO |:heavy_check_mark: |:heavy_check_mark:(1)|:heavy_check_mark: |:heavy_check_mark: |:heavy_check_mark:|\n|A2C |:heavy_check_mark: |:heavy_check_mark: |:heavy_check_mark: |:heavy_check_mark: |:x: |\n|ACKTR |:heavy_check_mark: |:x:(2) |:heavy_check_mark: |:heavy_check_mark: |:x: |\n|AOC |:heavy_check_mark: |:x: |:heavy_check_mark: |:heavy_check_mark: |:x: |\n\n(1): It's very unstable
\n(2): Needs https://openreview.net/forum?id=HyMTkQZAb implemented
\n\n## Sub packages\n\n- [intrinsic-rewards](https://github.com/kngwyu/intrinsic-rewards)\n - Contains an implementation of RND(Random Network Distillation)\n\n## References\n\n### DQN (Deep Q Network)\n- https://www.nature.com/articles/nature14236/\n\n### DDQN (Double DQN)\n- https://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/viewPaper/12389\n\n### DDPQ(Deep Deterministic Policy Gradient)\n- https://arxiv.org/abs/1509.02971\n\n### TD3(Twin Delayed Deep Deterministic Policy Gradient)\n- https://arxiv.org/abs/1802.09477\n\n### A2C (Advantage Actor Critic)\n- http://proceedings.mlr.press/v48/mniha16.pdf , https://arxiv.org/abs/1602.01783 (A3C, original version)\n- https://blog.openai.com/baselines-acktr-a2c/ (A2C, synchronized version)\n\n### ACKTR (Actor Critic using Kronecker-Factored Trust Region)\n- https://papers.nips.cc/paper/7112-scalable-trust-region-method-for-deep-reinforcement-learning-using-kronecker-factored-approximation\n\n### PPO (Proximal Policy Optimization)\n- https://arxiv.org/abs/1707.06347\n\n### AOC (Advantage Option Critic)\n- https://arxiv.org/abs/1609.05140 (DQN-like option critic)\n- https://arxiv.org/abs/1709.04571 (A3C-like option critic called A2OC)\n\n## Implementaions I referenced\nI referenced mainly openai baselines, but all these pacakages were useful.\n\nThanks!\n\nhttps://github.com/openai/baselines\n\nhttps://github.com/ikostrikov/pytorch-a2c-ppo-acktr\n\nhttps://github.com/ShangtongZhang/DeepRL\n\nhttps://github.com/chainer/chainerrl\n\nhttps://github.com/Thrandis/EKFAC-pytorch (for ACKTR)\n\nhttps://github.com/jeanharb/a2oc_delib (for AOC)\n\nhttps://github.com/sfujim/TD3 (for DDPG and TD3)\n\n## License\nThis project is licensed under Apache License, Version 2.0\n([LICENSE-APACHE](LICENSE) or http://www.apache.org/licenses/LICENSE-2.0).\n\n\n\n\n",
"description_content_type": "text/markdown",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/kngwyu/Rainy",
"keywords": "",
"license": "Apache2",
"maintainer": "",
"maintainer_email": "",
"name": "rainy",
"package_url": "https://pypi.org/project/rainy/",
"platform": "",
"project_url": "https://pypi.org/project/rainy/",
"project_urls": {
"Code": "https://github.com/kngwyu/Rainy",
"Homepage": "https://github.com/kngwyu/Rainy",
"Issue tracker": "https://github.com/kngwyu/Rainy/issues"
},
"release_url": "https://pypi.org/project/rainy/0.4.0/",
"requires_dist": [
"click (>=7.0)",
"GitPython (>=2.0)",
"gym[atari] (>=0.11.0)",
"numpy (>=1.15.0)",
"opencv-python (>=3.4)",
"Pillow (>=5.0)",
"torch (>=1.0)",
"pybullet (>=2.4) ; extra == 'bullet'",
"pybullet (>=0.16) ; extra == 'horovod'",
"ipython (>=7.0) ; extra == 'ipython'",
"matplotlib (>=3.0) ; extra == 'ipython'",
"ptpython (>=2.0) ; extra == 'ipython'"
],
"requires_python": ">=3.6.0",
"summary": "Algorithm and utilities for deep reinforcement learning",
"version": "0.4.0"
},
"last_serial": 5798126,
"releases": {
"0.1.0": [
{
"comment_text": "",
"digests": {
"md5": "aae5029c96d3b41d01fffdc50ab13cf0",
"sha256": "42db732ea85cdb206d3becbd318848c18b2c1bfed0d98b0e1b5578cb8dfadfd0"
},
"downloads": -1,
"filename": "rainy-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "aae5029c96d3b41d01fffdc50ab13cf0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 75890,
"upload_time": "2019-04-19T05:34:25",
"url": "https://files.pythonhosted.org/packages/41/52/42b1e0597a0705ebec98903c87c120398296e9f695c7492726be6b06ad71/rainy-0.1.0-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "714f831f5e16fe86ffb31b0a9241dae6",
"sha256": "347408e0306b0994659c5571dcb4b2f517a887457874fee36533c00b0edbc437"
},
"downloads": -1,
"filename": "rainy-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "714f831f5e16fe86ffb31b0a9241dae6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 34715,
"upload_time": "2019-04-19T05:34:28",
"url": "https://files.pythonhosted.org/packages/9b/04/5c5c6ae7ba13d5a49d331efae7f9413f66153c9cf5d1e0d16439e67dc7f0/rainy-0.1.0.tar.gz"
}
],
"0.2.0": [
{
"comment_text": "",
"digests": {
"md5": "ede8a5c8d49fa7b0931792a91c7070d1",
"sha256": "1e0b468ca96bcfbc8371ea9b9601c132e764832b5fdf32e32bf811fb2cd0c6fa"
},
"downloads": -1,
"filename": "rainy-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "ede8a5c8d49fa7b0931792a91c7070d1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6.0",
"size": 37943,
"upload_time": "2019-06-20T09:56:46",
"url": "https://files.pythonhosted.org/packages/c8/b6/ff352407c472f40df5804139ed4e70377a8a4986f0fd7aab197b228ec55e/rainy-0.2.0.tar.gz"
}
],
"0.3.0": [
{
"comment_text": "",
"digests": {
"md5": "b5408f383d8a95a2304f3404020e2cc5",
"sha256": "685ff9c89accd1ac3c2e3b63f61ae69d85be2e496672d85b3c23a6941638a6ab"
},
"downloads": -1,
"filename": "rainy-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b5408f383d8a95a2304f3404020e2cc5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6.0",
"size": 60136,
"upload_time": "2019-07-09T03:52:22",
"url": "https://files.pythonhosted.org/packages/f9/4a/dfba465ac0659cae226902979365c57cf623a974603e567d4ccb2f7ab438/rainy-0.3.0-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "e12e0b4343c4934cd0303773173b9b9f",
"sha256": "b07c22d76773a95ec6ae0676135ae137de50cf4797fb94350603cca8e15a54f2"
},
"downloads": -1,
"filename": "rainy-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "e12e0b4343c4934cd0303773173b9b9f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6.0",
"size": 43035,
"upload_time": "2019-07-09T03:52:24",
"url": "https://files.pythonhosted.org/packages/22/ac/f4cbe51164fa80a392d282e71e7734f604eb65c6e1e8fbb2f13051a71468/rainy-0.3.0.tar.gz"
}
],
"0.3.1": [
{
"comment_text": "",
"digests": {
"md5": "17e674a7d8014dc5f2ce5d6db737237d",
"sha256": "093f0ad09f0d30b62a190cd37d041db583beac181fc01cd5b2d02d97bd9d33dd"
},
"downloads": -1,
"filename": "rainy-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "17e674a7d8014dc5f2ce5d6db737237d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6.0",
"size": 77299,
"upload_time": "2019-08-15T16:00:49",
"url": "https://files.pythonhosted.org/packages/dc/b5/450ad397b895369e329289574f39c6851580a0ea8ec2f0564dab020d9e95/rainy-0.3.1-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "1ece2094c38d73687c69f12715db555f",
"sha256": "afdbb3eb0a30fc824a52becff2382e77934c8b2d6de1ab818a352ea56f96afda"
},
"downloads": -1,
"filename": "rainy-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "1ece2094c38d73687c69f12715db555f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6.0",
"size": 46269,
"upload_time": "2019-08-15T16:00:52",
"url": "https://files.pythonhosted.org/packages/6c/db/7e3a87079151b93a65c6ac53bc512dabb2ee31f8c9f6efd4f2ecfacae8b2/rainy-0.3.1.tar.gz"
}
],
"0.4.0": [
{
"comment_text": "",
"digests": {
"md5": "90af222cd09864ff18d07bef54ba6467",
"sha256": "90fe69b284463ac23a8ae611fe1a5ad80c1404daab9ba090d5cdac0465ded962"
},
"downloads": -1,
"filename": "rainy-0.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "90af222cd09864ff18d07bef54ba6467",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6.0",
"size": 81547,
"upload_time": "2019-09-08T03:05:32",
"url": "https://files.pythonhosted.org/packages/65/56/28ca6c930556931be8cbcc99bce576e967dab0e4aa8cc1d3b94052538170/rainy-0.4.0-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "002b4987582c77d9da5d921f063dbe99",
"sha256": "2ed888e568dad96712211f0648bd8966923f7527731fc420c766e2f92bcbaec8"
},
"downloads": -1,
"filename": "rainy-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "002b4987582c77d9da5d921f063dbe99",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6.0",
"size": 48559,
"upload_time": "2019-09-08T03:05:34",
"url": "https://files.pythonhosted.org/packages/53/22/c19999575577687fca74000a92b68cf80b66ca7f309db75430a274949cdc/rainy-0.4.0.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "90af222cd09864ff18d07bef54ba6467",
"sha256": "90fe69b284463ac23a8ae611fe1a5ad80c1404daab9ba090d5cdac0465ded962"
},
"downloads": -1,
"filename": "rainy-0.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "90af222cd09864ff18d07bef54ba6467",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6.0",
"size": 81547,
"upload_time": "2019-09-08T03:05:32",
"url": "https://files.pythonhosted.org/packages/65/56/28ca6c930556931be8cbcc99bce576e967dab0e4aa8cc1d3b94052538170/rainy-0.4.0-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "002b4987582c77d9da5d921f063dbe99",
"sha256": "2ed888e568dad96712211f0648bd8966923f7527731fc420c766e2f92bcbaec8"
},
"downloads": -1,
"filename": "rainy-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "002b4987582c77d9da5d921f063dbe99",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6.0",
"size": 48559,
"upload_time": "2019-09-08T03:05:34",
"url": "https://files.pythonhosted.org/packages/53/22/c19999575577687fca74000a92b68cf80b66ca7f309db75430a274949cdc/rainy-0.4.0.tar.gz"
}
]
}