{
    "info": {
        "author": "Luca Pasqualini",
        "author_email": "pasqualini@diism.unisi.it",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "License :: OSI Approved :: BSD License",
            "Programming Language :: Python :: 3.6",
            "Topic :: Scientific/Engineering :: Artificial Intelligence"
        ],
        "description": "USienaRL\n*********\n\nLuca Pasqualini - SAILab - University of Siena\n############################################################\n\nThis is a *python 3.6* and above library for Reinforcement Learning (RL) experiments.\n\nThe idea behind this library is to generate an intuitive yet versatile system to generate RL agents, experiments, models, etc.\nThe library is modular, and allow for easy experiment iterations and validation. It is currently used in my research and it\nwas built first for that purpose.\n\n**Note:** this is not meant to be an entry level framework. Users are expected to be at least somewhat experienced in both\npython and reinforcement learning. The framework is easy to extend but almost all the agent-related and environment-related\nwork should be done by yourself.\n\n**Features**\n\n*Included in package:*\n\n- Model, Agent, Environment, Experiment, Exploration Policy abstract classes with their interrelationships hidden inside the implementation\n- Customizable experiments in which, besides the default metrics, additional metrics can be used to validate or pass the experiment:\n    - Metrics of experiments are always related to rewards obtained (per-step or per-episode) and training episodes (usually the minimum the better)\n- Utility functions to run the same experiment in multiple equal iterations with automated folder setup and organization, registering the following metrics:\n    - Average total reward (reward in one episode) over training, validation and test\n    - Average scaled reward (reward per step) over training, validation and test\n    - Mean and standard deviation of average total and scaled reward over test in all experiment iterations (if more than one)\n    - Mean and standard deviation of maximum total and scaled reward over test in all experiment iterations (if more than one)\n    - Mean and standard deviation of minimum training episodes over test in all experiment iterations (if more than one)\n    - Experiment iteration achieving best results in each one of the metrics described above\n    - Easy to use .csv file with all the results for each experiment iteration\n    - Plots of total and scaled rewards over both all training and validation episodes, saved as .png files\n- Many state-of-the-art algorithms already implemented in pre-defined models, including:\n    - Tabular Temporal Difference Q-Learning, SARSA, Expected SARSA with Prioritized Experience Replay memory buffer\n    - Deep Temporal Difference Q-Learning (DQN), SARSA, Expected SARSA with Prioritized Experience Replay memory buffer\n    - Double Deep Temporal Difference Q-Learning (DDQN) with Prioritized Experience Replay memory buffer\n    - Dueling Temporal Difference Q-Learning (DDDQN) with Prioritized Experience Replay memory buffer\n    - Vanilla Policy Gradient (VPG) with General Advantage Estimate buffer using rewards-to-go\n    - Proximal Policy Optimization (PPO) with General Advantage Estimate buffer using rewards-to-go and early stopping\n- Many state-of-the-art exploration policies, including:\n    - Epsilon Greedy with tunable decay rate, start value and end value\n    - Boltzmann sampling with tunable temperature decay rate, start value and end value\n- Config class to define the hidden layers of all Tensorflow graphs (including the CNN)\n- Default Pass-Through interface class to allow communications between agents and environments\n- Additive action mask for all the algorithms supporting it (only discrete action sets):\n    - The mask supports two values: -infinity (mask) and 0.0 (pass-through)\n    - If not supplied, the mask is by default full pass-through\n\n*Not included in package:*\n\n- Extensive set of benchmarks for each algorithm in the OpenAI gym environment\n- Default agents, OpenAI gym environment and benchmark experiment classes to test the benchmarks by yourself\n\nFor additional example of usage of this framework, take a look a these github pages:\n\n- `TicTacToeRL <https://github.com/InsaneMonster/TicTacToeRL>`_\n- `UltimateTicTacToeRL <https://github.com/InsaneMonster/UltimateTicTacToeRL>`_\n\n**License**\n\n*BSD 3-Clause License*\n\nFor additional information check the provided license file.\n\n**How to install**\n\nIf you only need to use the framework, just download the pip package *usienarl* and import the package in your scripts.\n\nWhen installing, make sure to choose the version suiting your computing capabilities.\nIf you have CUDA installed, the gpu version is advised. Otherwise, just use the cpu version.\nTo choose a version, specify your extra require during install:\n\n- pip install usienarl[tensorflow-gpu] to install the tensorflow-gpu version\n- pip install usienarl[tensorflow] to install the tensorflow using cpu version\n\n**Note:** failure in specifying the extra require will cause tensorflow to not be installed, and as such the library won't\nbe usable at all. For instance, this is not allowed, *unless you already have tensorflow installed*:\n\n- pip install usienarl\n\nIf you want to improve/modify/extends the framework, or even just try my own benchmarks at home, download or clone\nthe `git repository <https://github.com/InsaneMonster/USienaRL>`_.\nYou are welcome to open issues or participate in the project. Note that the benchmarks are built to run using tensorflow-gpu.\n\n**Requirements**\n\nBesides Tensorflow, with this package also the following packages will be installed in your environment:\n\n- NumPy\n- SciPy\n- Pandas\n- Matplotlib\n\n**How to use**\n\nFor a simple use case, refer to benchmark provided in the `repository <https://github.com/InsaneMonster/USienaRL>`_. For advanced use, refer to the built-in documentation\nand to the provided source code in the `repository <https://github.com/InsaneMonster/USienaRL>`_.\n\n**Current issues**\n\nFrom the save-restore standpoint it could be useful to implement an easy way to pass a metagraph\nwithout the need to redefine the entire agent (maybe serializing the agent somehow?).\n\nAn experiment can right now work only in a specific environment. It could be interesting to test multiple environments\nboth from a curriculum learning perspective (it can still be done using multiple subsequent experiments, however) and from\na generalization perspective (train one one, validate on another, etc).\n\nA way to check if environments are compatible one another would be required too if what said above is implemented.\n\n**Changelog**\n\n- Added Proximal Policy Optimization algorithm to Policy Optimization models",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/InsaneMonster/USienaRL",
        "keywords": "Reinforcement Learning Reinforcement-Learning RL Machine Machine-Learning ML library framework SAILab USiena Siena",
        "license": "BSD 3-Clause",
        "maintainer": "",
        "maintainer_email": "",
        "name": "usienarl",
        "package_url": "https://pypi.org/project/usienarl/",
        "platform": "",
        "project_url": "https://pypi.org/project/usienarl/",
        "project_urls": {
            "Homepage": "https://github.com/InsaneMonster/USienaRL"
        },
        "release_url": "https://pypi.org/project/usienarl/0.5.0/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "University of Siena Reinforcement Learning library - SAILab",
        "version": "0.5.0"
    },
    "last_serial": 5970481,
    "releases": {
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4e488e3a985bb0d78509c49dbb4fd0aa",
                    "sha256": "8dd5d3822a52248960c2832eecab991042b1bf4091d462bf0e240cb61ce74244"
                },
                "downloads": -1,
                "filename": "usienarl-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "4e488e3a985bb0d78509c49dbb4fd0aa",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 44123,
                "upload_time": "2019-07-19T08:50:35",
                "url": "https://files.pythonhosted.org/packages/cf/7e/d93e9a52ae5dcd15fcff9d8019302e7940b8e23fcea74a53247984aa177c/usienarl-0.2.0.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "cfb33120daf49cca8134c35dd3c9cbf7",
                    "sha256": "658dc89cf99d9853052ce0b1c9760e55e8a51d591d85bebd09916498217162a5"
                },
                "downloads": -1,
                "filename": "usienarl-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "cfb33120daf49cca8134c35dd3c9cbf7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 44562,
                "upload_time": "2019-07-26T10:00:47",
                "url": "https://files.pythonhosted.org/packages/8b/69/fdfc53cf1f3cc7cabd30596442c4201c98e197e6a3f1da53e74ff6b2114c/usienarl-0.2.1.tar.gz"
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6716bb31d3265af0b52e621fb716acb3",
                    "sha256": "4528363dde47c952e191f0b7f1158843176b0cef3ec10e2bace7583f69eba068"
                },
                "downloads": -1,
                "filename": "usienarl-0.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "6716bb31d3265af0b52e621fb716acb3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 44498,
                "upload_time": "2019-07-27T09:18:54",
                "url": "https://files.pythonhosted.org/packages/c6/9d/6570a73727f6117330310a3d41759e970831dd7ec31d633febb081edbe69/usienarl-0.2.2.tar.gz"
            }
        ],
        "0.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "59c23fe48dc7100517fa96e75b6b53bc",
                    "sha256": "9b7247bd999df993edb69e4400e048fbb2a0c1f31ac61e5d2f032ab0165e208d"
                },
                "downloads": -1,
                "filename": "usienarl-0.2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "59c23fe48dc7100517fa96e75b6b53bc",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 44503,
                "upload_time": "2019-07-27T15:59:11",
                "url": "https://files.pythonhosted.org/packages/84/a6/a4d4b580b1fc7e7be18aea1786dbbc393882658dbb8d86245d7cd7289e7d/usienarl-0.2.3.tar.gz"
            }
        ],
        "0.2.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "8e5a697ce7d935d4d84564f2496a03fc",
                    "sha256": "2731c6e98bbfdad562ad4d0518dae3452099212f2552dad4182151f00259214c"
                },
                "downloads": -1,
                "filename": "usienarl-0.2.4.tar.gz",
                "has_sig": false,
                "md5_digest": "8e5a697ce7d935d4d84564f2496a03fc",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 44670,
                "upload_time": "2019-07-28T13:04:17",
                "url": "https://files.pythonhosted.org/packages/93/02/eac6dc2d4aa080bb8a9f16932db5c62b7030961860097793bc5ee9068281/usienarl-0.2.4.tar.gz"
            }
        ],
        "0.3.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ade795cb2ec0ce2fb5b9f92dd3471502",
                    "sha256": "200fd02a87e0c7109347f1cbb99502aa9aa7c41f477d488a18dbf29a48a55e2a"
                },
                "downloads": -1,
                "filename": "usienarl-0.3.4.tar.gz",
                "has_sig": false,
                "md5_digest": "ade795cb2ec0ce2fb5b9f92dd3471502",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 46157,
                "upload_time": "2019-08-24T14:44:10",
                "url": "https://files.pythonhosted.org/packages/e6/a9/60a317b0df9635a72fdf923518a7c41ad69797b7b360a4cc9eae5f63cc51/usienarl-0.3.4.tar.gz"
            }
        ],
        "0.3.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5ddec83e0e4b912fcd9b76c33e5c8809",
                    "sha256": "06f677c0e5d662ab71b1dfe43f77a12f3707573048f1314b7e839f362d5a00fb"
                },
                "downloads": -1,
                "filename": "usienarl-0.3.5.tar.gz",
                "has_sig": false,
                "md5_digest": "5ddec83e0e4b912fcd9b76c33e5c8809",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 45896,
                "upload_time": "2019-08-24T21:44:14",
                "url": "https://files.pythonhosted.org/packages/fe/bb/a0ec861e0ea34847a15a156a60d41f6c8aa30bf7cf323216f79803004a6a/usienarl-0.3.5.tar.gz"
            }
        ],
        "0.3.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4f555370028f6889267e932b890c7865",
                    "sha256": "566a7a20a01ef76ac31ad4bc57b313943af458f5f10ea21757173361624e4450"
                },
                "downloads": -1,
                "filename": "usienarl-0.3.6.tar.gz",
                "has_sig": false,
                "md5_digest": "4f555370028f6889267e932b890c7865",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 46101,
                "upload_time": "2019-08-25T08:36:28",
                "url": "https://files.pythonhosted.org/packages/79/37/e78194dd8a63b08b8eb49b8dde84f7eb1cd60ef5841133f459a010d6474b/usienarl-0.3.6.tar.gz"
            }
        ],
        "0.3.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b91cf20af0e3ca29211ac42e2dfe7e8d",
                    "sha256": "620528f14ccdb8cd8bbe9fe03d0e60f183e6dceeea86c5416fa1aef3ff19076e"
                },
                "downloads": -1,
                "filename": "usienarl-0.3.7.tar.gz",
                "has_sig": false,
                "md5_digest": "b91cf20af0e3ca29211ac42e2dfe7e8d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 46038,
                "upload_time": "2019-08-27T20:20:09",
                "url": "https://files.pythonhosted.org/packages/b7/00/d338a174bec7d8b73f8baa78edddf26cffd5efed012b5f432bce1449d89e/usienarl-0.3.7.tar.gz"
            }
        ],
        "0.4.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d1ef122123412b140d0da17a25a4585f",
                    "sha256": "aee0a9fe35831916b96195fd35bab74bb5809c182d800bf631b16f95a1ed880e"
                },
                "downloads": -1,
                "filename": "usienarl-0.4.0.tar.gz",
                "has_sig": false,
                "md5_digest": "d1ef122123412b140d0da17a25a4585f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 47413,
                "upload_time": "2019-09-05T09:00:39",
                "url": "https://files.pythonhosted.org/packages/77/0a/7d6533d8562e106550c6e6b7dba6d7670184b36beeec20dcbe380d4a2515/usienarl-0.4.0.tar.gz"
            }
        ],
        "0.4.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "34110d9c766bf559b3884e68e43a1053",
                    "sha256": "5eb3e86d20b9edc79c3060bd174f35a75d6e95654aa478b6cedb357d721aa9df"
                },
                "downloads": -1,
                "filename": "usienarl-0.4.1.tar.gz",
                "has_sig": false,
                "md5_digest": "34110d9c766bf559b3884e68e43a1053",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 47706,
                "upload_time": "2019-09-06T11:28:58",
                "url": "https://files.pythonhosted.org/packages/ad/a7/8faecbd2f6985ec315129b33d92d9c11c3734019433f77344f9b55c23ba5/usienarl-0.4.1.tar.gz"
            }
        ],
        "0.4.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e5ecaa99691fc3c3ba4d3e31f5fa0c39",
                    "sha256": "7e2ca8528b8df5d37e4014dcbad7bb1842c40c7f7db65522a4ab0b842af3a195"
                },
                "downloads": -1,
                "filename": "usienarl-0.4.2.tar.gz",
                "has_sig": false,
                "md5_digest": "e5ecaa99691fc3c3ba4d3e31f5fa0c39",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 48469,
                "upload_time": "2019-09-08T08:17:04",
                "url": "https://files.pythonhosted.org/packages/33/60/c446f7fd4e52cf278efc53d3e7781ae26c065dfb9f5840549e980f0f2fdb/usienarl-0.4.2.tar.gz"
            }
        ],
        "0.4.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "bb97ce73215610300b8832b4bdf0e8a6",
                    "sha256": "942e9117eb7c642aae37e48014fb66851436ea6a9b11e7713b0b93a331b04487"
                },
                "downloads": -1,
                "filename": "usienarl-0.4.3.tar.gz",
                "has_sig": false,
                "md5_digest": "bb97ce73215610300b8832b4bdf0e8a6",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 48103,
                "upload_time": "2019-09-08T08:27:54",
                "url": "https://files.pythonhosted.org/packages/63/6e/37db0e6790d518fb8b72659e7f882098b87cdad3c7e075ab15726e4d4060/usienarl-0.4.3.tar.gz"
            }
        ],
        "0.4.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "053cc66df6ff761f0609f3f8f7ebced1",
                    "sha256": "d72c6c13abe9973122e73a741448f0891b403fd14a7ba303ff053ba18d8a38f8"
                },
                "downloads": -1,
                "filename": "usienarl-0.4.4.tar.gz",
                "has_sig": false,
                "md5_digest": "053cc66df6ff761f0609f3f8f7ebced1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 48110,
                "upload_time": "2019-09-08T13:37:14",
                "url": "https://files.pythonhosted.org/packages/b5/4d/0f6f39fbc34560bc458cc0404b78b7cec67e3fc9e06e491d61a6d8007ba3/usienarl-0.4.4.tar.gz"
            }
        ],
        "0.4.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4a7bae1975b2460ad897519abf04e1c3",
                    "sha256": "7b8fc25dc632f9ba49e9f43b0c3c78126f7272468402d47b2c871976019460fb"
                },
                "downloads": -1,
                "filename": "usienarl-0.4.5.tar.gz",
                "has_sig": false,
                "md5_digest": "4a7bae1975b2460ad897519abf04e1c3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 42082,
                "upload_time": "2019-09-18T10:32:44",
                "url": "https://files.pythonhosted.org/packages/fd/05/c5b44514a60f8ac8314af5c54e33c4418c05502319cde1ee83c4746b93c5/usienarl-0.4.5.tar.gz"
            }
        ],
        "0.4.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a4151b92d9ba1557874fc0c98a39a718",
                    "sha256": "4ecbc1c881d78bd94c7f3be395bd04e4dfa6eb424c65591fbe03e4f3408440c2"
                },
                "downloads": -1,
                "filename": "usienarl-0.4.6.tar.gz",
                "has_sig": false,
                "md5_digest": "a4151b92d9ba1557874fc0c98a39a718",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 42904,
                "upload_time": "2019-09-22T09:43:50",
                "url": "https://files.pythonhosted.org/packages/22/d0/e1d150839e9c6fa94c880ac799d4b90878ed7fe50a2d5db9834c6006f1d9/usienarl-0.4.6.tar.gz"
            }
        ],
        "0.4.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fd9ef9aa792b3bb88169314294f51e14",
                    "sha256": "687508c452c880b495f483da45ce23447e3f1e7faafdbd8195e491a167aa8d1d"
                },
                "downloads": -1,
                "filename": "usienarl-0.4.7.tar.gz",
                "has_sig": false,
                "md5_digest": "fd9ef9aa792b3bb88169314294f51e14",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 42469,
                "upload_time": "2019-09-22T12:53:16",
                "url": "https://files.pythonhosted.org/packages/8c/f4/249ccb539d7f8efa27884f1c25464e41c7b36feb90c23b54364c70e5943e/usienarl-0.4.7.tar.gz"
            }
        ],
        "0.5.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "df0d50e0f81d2f4d07496d06d6b08d50",
                    "sha256": "81896e0c957c7f5a801a20497aba048abb7b752279a78f6b62a06923354f5c72"
                },
                "downloads": -1,
                "filename": "usienarl-0.5.0.tar.gz",
                "has_sig": false,
                "md5_digest": "df0d50e0f81d2f4d07496d06d6b08d50",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 43376,
                "upload_time": "2019-10-14T09:01:01",
                "url": "https://files.pythonhosted.org/packages/ac/0a/a0969258dd29eec47bd1425792b4768773a9481e6d09da2cb55ebe01aea4/usienarl-0.5.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "df0d50e0f81d2f4d07496d06d6b08d50",
                "sha256": "81896e0c957c7f5a801a20497aba048abb7b752279a78f6b62a06923354f5c72"
            },
            "downloads": -1,
            "filename": "usienarl-0.5.0.tar.gz",
            "has_sig": false,
            "md5_digest": "df0d50e0f81d2f4d07496d06d6b08d50",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 43376,
            "upload_time": "2019-10-14T09:01:01",
            "url": "https://files.pythonhosted.org/packages/ac/0a/a0969258dd29eec47bd1425792b4768773a9481e6d09da2cb55ebe01aea4/usienarl-0.5.0.tar.gz"
        }
    ]
}