{ "info": { "author": "Rotinov Egor", "author_email": "rotinov-github@pm.me", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python :: 3", "Topic :: Scientific/Engineering :: Artificial Intelligence" ], "description": "# AGNES - Flexible Reinforcement Learning Framework with PyTorch\n\n**Status:** This framework is under active development and bugs may occur.\n\n[![Build status](https://travis-ci.org/rotinov/AGNES.svg?branch=master)](https://travis-ci.org/rotinov/AGNES)\n![Upload Python Package](https://github.com/rotinov/AGNES/workflows/Upload%20Python%20Package/badge.svg?branch=master)\n\n## Results\n#### MuJoCo\n**(Current results)**\n[![The ending average is 5326.2](https://raw.githubusercontent.com/rotinov/AGNES/master/results/MuJoCo/reward_per_timestep.svg?raw=true&sanitize=true)](results/MuJoCo/Ant-v2_MLP)\n*MuJoCo \"Ant-v2\" training with 1M steps. **Single** runner with **PPO** algorithm, **MLP** NN and 32 number of envs. The curve is an average of 3 runs.*\n\n*You can get the Tensorboard log file by clicking the image above(You will be redirected to the destination GitHub folder). The default config for the MuJoCo environment was used. Plotted by **examples/plot.py***\n\n#### Atari\n**(Old results)**\n\n[![Peaking at 861.8 at the end. The ending average is 854.8.](https://raw.githubusercontent.com/rotinov/AGNES/master/results/Atari-BreakoutNoFrameskip-v4-PPO-10M/reward_per_update.svg?raw=true&sanitize=true)](results/Atari-BreakoutNoFrameskip-v4-PPO-10M)\n*Atari \"BreakoutNoFrameskip-v4\" with frame stack training with 10M steps. **DistributedMPI** runner with **PPO** algorithm, **LSTMCNN** and 16 number of envs.*\n\n*You can get the Tensorboard log file by clicking the image above(You will be redirected to the destination GitHub folder). The default config for the Atari environment was used.*\n\n![LSTMCNN agent plays Breakout](https://raw.githubusercontent.com/rotinov/AGNES/master/results/Atari-BreakoutNoFrameskip-v4-PPO-10M/Breakout-LSTMCNN.gif)\n\n*Grad-cam technique was used for sampled action chosen by trained LSTMCNN(previous point).*\n\n![LSTMCNN agent plays Breakout](https://github.com/rotinov/AGNES/raw/master/results/Atari-BreakoutNoFrameskip-v4-PPO-10M/Breakout-LSTMCNN-Grad-Cam.gif)\n\n## Runners\n#### Single\n\nOne worker and trainer. **agnes.make_vec_env** can also be used here.\n\n```python\nimport agnes\nimport time\n\n\nif __name__ == '__main__':\n env = agnes.make_env(\"InvertedDoublePendulum-v2\")\n runner = agnes.Single(env, agnes.PPO, agnes.MLP)\n runner.log(agnes.log, agnes.TensorboardLogger(\".logs/\"), agnes.CsvLogger(\".logs/\"))\n runner.run()\n\n```\n\n**agnes.log** - object of **StandardLogger** class that outputs parameters to console.\n**agnes.TensorboardLogger** - class for writing logs in Tensorboard file.\n**agnes.CsvLogger** - class for writing logs in csv file. (required for plotting)\n\n\n#### DistributedMPI\n\nUnlike in **Single** runner, in **DistributedMPI** runner due to async executing, weights are delayed by one rollout but this has no effect on learning because weights are delayed only by one update as it is in **Single** runner. So all parameters like probabilities ratio stay the same.\n\nRuns with\n\n```bash\nmpiexec -n 3 python -m mpi4py script_name.py\nor\nmpirun -n 3 python -m mpi4py script_name.py\n```\nThis command will run 2 workers and 1 trainer.\n```python\n# script_name.py\nimport agnes\n\n\nif __name__ == '__main__':\n env = agnes.make_vec_env(\"BreakoutNoFrameskip-v4\")\n runner = agnes.DistributedMPI(env, agnes.PPO, agnes.CNN)\n runner.run()\n\n```\n\n## Algorithms\n#### A2C\nSync version of Advantage Actor Critic is implemented in this framework and can be used simply:\n```python\nimport agnes\n\n\nif __name__ == '__main__':\n runner = agnes.Single(env, agnes.A2C, agnes.MLP)\n runner.run()\n\n```\n\n#### PPO\nProximal Policy Optimization is implemented in this framework and can be used simply:\n```python\nimport agnes\n\n\nif __name__ == '__main__':\n runner = agnes.Single(env, agnes.PPO, agnes.MLP)\n runner.run()\n\n```\n\n## Neural Network Architectures\n\n#### Multi Layer Perceptron\n\nCan be used with both continuous and discrete action spaces.\n```python\n...\nrunner = agnes.Single(env, agnes.PPO, agnes.MLP)\n...\n```\n\n#### Convolutional Neural Network\n\nCan be used only with discrete action spaces.\n```python\n...\nrunner = agnes.Single(env, agnes.PPO, agnes.CNN)\n...\n```\n\n#### Recurrent Neural Network\n\nCan be used with both continuous and discrete action spaces.\n```python\n...\nrunner = agnes.Single(env, agnes.PPO, agnes.RNN)\n...\n```\n\n#### Convolutional Recurrent Neural Network\n\nCan be used only with discrete action spaces.\n```python\n...\nrunner = agnes.Single(env, agnes.PPO, agnes.RNNCNN)\n...\n```\n\n#### Convolutional Neural Network with last LSTM layer\nCan be used only with discrete action spaces.\n```python\n...\nrunner = agnes.Single(env, agnes.PPO, agnes.LSTMCNN)\n...\n```\n\n## Make environment\n* **make_vec_env(env, envs_num**=ncpu, **config**=None)**\n\n Parameters:\n * **env**(str or function) is id of gym environment or function, that returns initialized environment\n * **envs_num**(int) is a number of environments to initialize, by default is a number of logical cores on the CPU\n * **config**(dict) is a dictionary with parameters for **Monitor** and for initializing environment, by default is None(uses default config)\n\n Returns:\n * dict of\n 1. **\"env\"**(**VecEnv** object)\n 2. **\"env_type\"**(str)\n 3. **\"env_num\"**(int) is a number of envs in **VecEnv** object\n 4. **\"env_name\"**(str) is the name of envs in **VecEnv** object(Id in gym or class name)\n\n The whole tuple should be put in a **runner**.\n\n* **make_env(env, config=None)** is an alias of **make_vec_env** without **envs_num** argument that will be setted to 1.\n\n**Notice:** Some plot functions and environment wrappers were taken from [OpenAI Baselines(2017)](https://github.com/openai/baselines).\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/rotinov/AGNES", "keywords": "PyTorch reinforcement learning ppo a2c framework random network distillation", "license": "MIT LICENSE", "maintainer": "", "maintainer_email": "", "name": "agnes", "package_url": "https://pypi.org/project/agnes/", "platform": "", "project_url": "https://pypi.org/project/agnes/", "project_urls": { "Homepage": "https://github.com/rotinov/AGNES" }, "release_url": "https://pypi.org/project/agnes/0.0.7.4/", "requires_dist": [ "cloudpickle", "gym[atari]", "psutil", "atari-py", "numpy", "torch (>=1.2)", "opencv-python", "mpi4py ; extra == 'all'", "Tensorboard ; extra == 'all'", "mpi4py ; extra == 'distributed'", "Tensorboard ; extra == 'tensorboard'" ], "requires_python": "", "summary": "AGNES - Flexible Reinforcement Learning Framework with PyTorch", "version": "0.0.7.4", "yanked": false, "yanked_reason": null }, "last_serial": 7354480, "releases": { "0.0.5": [ { "comment_text": "", "digests": { "md5": "8686594d08193b882092ee0f32451733", "sha256": "6ceed1a7ea8d815d11e4dc87315553467f7499741e0a7d6f3a7de2cc649fee26" }, "downloads": -1, "filename": "agnes-0.0.5-py3-none-any.whl", "has_sig": false, "md5_digest": "8686594d08193b882092ee0f32451733", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 47187, "upload_time": "2019-10-16T10:28:49", "upload_time_iso_8601": "2019-10-16T10:28:49.677192Z", "url": "https://files.pythonhosted.org/packages/78/47/6094ac6064be7d11c34ade2556cee8ed5d6aeb19a3a96107b422211ff865/agnes-0.0.5-py3-none-any.whl", "yanked": false, "yanked_reason": null } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "fe7c91f1285d974ccd8eca102481b8f0", "sha256": "8935eacf5c496c9fc5f3957f3d71df3a878879671d7afd0b34e420c4a2723b8c" }, "downloads": -1, "filename": "agnes-0.0.6-py3-none-any.whl", "has_sig": false, "md5_digest": "fe7c91f1285d974ccd8eca102481b8f0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 47539, "upload_time": "2019-10-23T20:09:04", "upload_time_iso_8601": "2019-10-23T20:09:04.398790Z", "url": "https://files.pythonhosted.org/packages/54/95/8bcce1d599d969b13d352d02395b274034fa9fcc23d5bc9f6d967d9b3c8c/agnes-0.0.6-py3-none-any.whl", "yanked": false, "yanked_reason": null } ], "0.0.7": [ { "comment_text": "", "digests": { "md5": "b2442c52304e779ddc0e45550e31c940", "sha256": "29b7251cdc3a471a8f9b982ec411ccf6a45c37579a1d04aaed4bdd931a7cbfb3" }, "downloads": -1, "filename": "agnes-0.0.7-py3-none-any.whl", "has_sig": false, "md5_digest": "b2442c52304e779ddc0e45550e31c940", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 51722, "upload_time": "2020-01-25T13:13:58", "upload_time_iso_8601": "2020-01-25T13:13:58.515139Z", "url": "https://files.pythonhosted.org/packages/31/e4/f4f82539e2eef7359a5eb8412b0b8895ffac78f3017bf78f84f485332ae5/agnes-0.0.7-py3-none-any.whl", "yanked": false, "yanked_reason": null } ], "0.0.7.1": [ { "comment_text": "", "digests": { "md5": "8763bd89f73a5613f3858992c61dbbf6", "sha256": "a84e0135bf80f29daa40c87cad9f50bc992810371c27f65d92cd4e2d91022e58" }, "downloads": -1, "filename": "agnes-0.0.7.1-py3-none-any.whl", "has_sig": false, "md5_digest": "8763bd89f73a5613f3858992c61dbbf6", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 51588, "upload_time": "2020-02-09T13:06:28", "upload_time_iso_8601": "2020-02-09T13:06:28.450813Z", "url": "https://files.pythonhosted.org/packages/68/e5/a447bbf074b6c95c50fa21fee8594e10766c10be53258746ece9cfab7fde/agnes-0.0.7.1-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "2bb8293b6849cb5f270cfcd29bd30f15", "sha256": "d0d1b314a3b0ea7c7b12039f8890932c58afe5c6311cdb6740639a5d204fb982" }, "downloads": -1, "filename": "agnes-0.0.7.1.tar.gz", "has_sig": false, "md5_digest": "2bb8293b6849cb5f270cfcd29bd30f15", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 36743, "upload_time": "2020-02-09T13:06:29", "upload_time_iso_8601": "2020-02-09T13:06:29.658271Z", "url": "https://files.pythonhosted.org/packages/db/f8/54a03a10b016fdee624e63b18466fe114d0f3b90a316e777440929014bb6/agnes-0.0.7.1.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.7.2": [ { "comment_text": "", "digests": { "md5": "df67a35329c6becca9c22e935a258874", "sha256": "45542a759c887680865d58fe4fd5a97f7ea6b34e5e399e9eda9756871db7c1fd" }, "downloads": -1, "filename": "agnes-0.0.7.2-py3-none-any.whl", "has_sig": false, "md5_digest": "df67a35329c6becca9c22e935a258874", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 51641, "upload_time": "2020-02-09T13:57:39", "upload_time_iso_8601": "2020-02-09T13:57:39.637461Z", "url": "https://files.pythonhosted.org/packages/0b/91/777daf8e3afcf2a5b45fb9ac9229d762ffa8c8c477f6660d258c21dd8662/agnes-0.0.7.2-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "a343ef60adcdb032655c68450a3352ad", "sha256": "e468af3346dc314e57864a7a3d919e0c4439bf2a310af45a4e80ed0016558e59" }, "downloads": -1, "filename": "agnes-0.0.7.2.tar.gz", "has_sig": false, "md5_digest": "a343ef60adcdb032655c68450a3352ad", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 36837, "upload_time": "2020-02-09T13:57:40", "upload_time_iso_8601": "2020-02-09T13:57:40.859639Z", "url": "https://files.pythonhosted.org/packages/20/ff/c0b7a7193425d13fae27d6ceaa7badeef52cfeb41302584c460f72bf51cc/agnes-0.0.7.2.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.7.3": [ { "comment_text": "", "digests": { "md5": "7aa7f5e0a79f80ea6e6c8edae0de286b", "sha256": "123ad705c0d1f20dff4680f5ab9af49473b3a3054637f9ecd368cb512818d30e" }, "downloads": -1, "filename": "agnes-0.0.7.3-py3-none-any.whl", "has_sig": false, "md5_digest": "7aa7f5e0a79f80ea6e6c8edae0de286b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 53455, "upload_time": "2020-02-24T14:27:29", "upload_time_iso_8601": "2020-02-24T14:27:29.548325Z", "url": "https://files.pythonhosted.org/packages/03/78/a9f9de43431f3dd1575c313a9f2f26dfd54701b33c969951e616339a5292/agnes-0.0.7.3-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "297c7e2b6a836bdcb54bbd46eb582294", "sha256": "4cfc05ba8f7188312a674abc95157e535f07c5b08e73a5d0024d6ff27a84fd9f" }, "downloads": -1, "filename": "agnes-0.0.7.3.tar.gz", "has_sig": false, "md5_digest": "297c7e2b6a836bdcb54bbd46eb582294", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 39404, "upload_time": "2020-02-24T14:27:30", "upload_time_iso_8601": "2020-02-24T14:27:30.867479Z", "url": "https://files.pythonhosted.org/packages/a8/c1/97c7067950c36bd21bdc34f32ded1dfecc5d3a56497b405a7dd90288997c/agnes-0.0.7.3.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.7.4": [ { "comment_text": "", "digests": { "md5": "a98f2502dbd26ce4777a03902393a6f0", "sha256": "287e11ccce48f5921b33785981699beb30df3d0481eaa83af406a4e01ebc1126" }, "downloads": -1, "filename": "agnes-0.0.7.4-py3-none-any.whl", "has_sig": false, "md5_digest": "a98f2502dbd26ce4777a03902393a6f0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 55903, "upload_time": "2020-05-29T17:26:59", "upload_time_iso_8601": "2020-05-29T17:26:59.958520Z", "url": "https://files.pythonhosted.org/packages/ec/f0/be6e55f3dea2cb58feef5e224e0e28b61bf5ab1bac1d5b551e3469a0c5de/agnes-0.0.7.4-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "524ea49d27423d1aa08c637a1266e96c", "sha256": "bbbeabd6bef6a05b2d49833031938ad8274846d797b0b65ac2486272ea5d7fb2" }, "downloads": -1, "filename": "agnes-0.0.7.4.tar.gz", "has_sig": false, "md5_digest": "524ea49d27423d1aa08c637a1266e96c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 41704, "upload_time": "2020-05-29T17:27:00", "upload_time_iso_8601": "2020-05-29T17:27:00.996614Z", "url": "https://files.pythonhosted.org/packages/25/46/797a5733a81f243a2d89b1a9617117b84f0c2344d33a35647ebf17cdf6f5/agnes-0.0.7.4.tar.gz", "yanked": false, "yanked_reason": null } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "a98f2502dbd26ce4777a03902393a6f0", "sha256": "287e11ccce48f5921b33785981699beb30df3d0481eaa83af406a4e01ebc1126" }, "downloads": -1, "filename": "agnes-0.0.7.4-py3-none-any.whl", "has_sig": false, "md5_digest": "a98f2502dbd26ce4777a03902393a6f0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 55903, "upload_time": "2020-05-29T17:26:59", "upload_time_iso_8601": "2020-05-29T17:26:59.958520Z", "url": "https://files.pythonhosted.org/packages/ec/f0/be6e55f3dea2cb58feef5e224e0e28b61bf5ab1bac1d5b551e3469a0c5de/agnes-0.0.7.4-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "524ea49d27423d1aa08c637a1266e96c", "sha256": "bbbeabd6bef6a05b2d49833031938ad8274846d797b0b65ac2486272ea5d7fb2" }, "downloads": -1, "filename": "agnes-0.0.7.4.tar.gz", "has_sig": false, "md5_digest": "524ea49d27423d1aa08c637a1266e96c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 41704, "upload_time": "2020-05-29T17:27:00", "upload_time_iso_8601": "2020-05-29T17:27:00.996614Z", "url": "https://files.pythonhosted.org/packages/25/46/797a5733a81f243a2d89b1a9617117b84f0c2344d33a35647ebf17cdf6f5/agnes-0.0.7.4.tar.gz", "yanked": false, "yanked_reason": null } ], "vulnerabilities": [] }