{
    "info": {
        "author": "PyLoa Developers",
        "author_email": "pyloadevelopers@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Intended Audience :: Developers",
            "Intended Audience :: Education",
            "Intended Audience :: Science/Research",
            "License :: OSI Approved :: MIT License",
            "Operating System :: MacOS",
            "Operating System :: Microsoft :: Windows",
            "Operating System :: POSIX",
            "Operating System :: Unix",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7",
            "Topic :: Scientific/Engineering"
        ],
        "description": "## PyLoa - Learning on-line Algorithms with Python\n\n`pyloa` is a research repository for analyzing the performance of classic on-line algorithms vs. modern Machine\nLearning, specifically Reinforcement Learning, approaches. PyLoa ships with an implementation of two commonly known \non-line problems as `environments`:\n\n* `(k,n)-paging-problem` with a `cache_size k` and `n pages` for a sequence of page-requests\n* `(k,n)-coloring-problem` with `k colors` for a graph with `n vertices` \n\nPyLoa allows for `agents` to be \n\n* trained on such `enviroments` (problem definitions) that require on-line solutions, \n* evaluated against commonly used heuristics or any state-of-the-art algorithm,\n* exploited (extrapolation of a *potentially* worst case problem instances) to determine a solution's [competitve ratio](https://en.wikipedia.org/wiki/Competitive_analysis_(online_algorithm)).\n\n---\n\n#### Dependencies\n\n`pyloa` is developed for Python 3.5+ and has the following package dependencies:\n\n```python\nmatplotlib==3.0.3  \nscipy==1.2.1  \ntensorflow==1.13.1  \ntqdm==4.31.1  \nnumpy==1.16.2\n```\n\n---\n#### Installation\n\nWe recommend using `pyloa` within a [virtual environment](https://docs.python.org/3.5/library/venv.html):\n\n    mkdir myproject\n    cd myproject\n    python3 -m venv virtualenv/\n    source virtualenv/bin/activate\n\nUpdate [`pip`](https://pypi.org/project/pip/) and [`setuptools`](https://pypi.org/project/setuptools/) before continuing:\n\n    pip install --upgrade pip setuptools\n\nAfterwards you can install `pyloa` either from its latest [PyPI stable](https://google.com) release\n\n    pip install pyloa\n\n**or** from its latest [development release](https://github.com/pyloa/PyLoa/tree/master/pyloa) on GitHub\n\n    pip install git+https://github.com/pyloa/PyLoa.git\n\n---\n#### General Usage\n\n`pyloa` can be used in three different ways to analyze an on-line problem; each depicted via a so called runmode \n(`train`, `eval`, `gen`). Any runemode can be invoked via its positional argument and requires a python-configuration-file.\n\n    pyloa {train,gen,eval} --config path/to/hyperparams.py\n\nhyperparams depicts the setting of the experiment at hand; it must hold a dictionary named `params`, which moreover **must \ncontain** dictionaries for the keys `instance`, `environment` and `agent`. \n\n* `params[\"\u00ecnstance\"]`: Must define a configuration of a subclass implementation of [`pyloa.instance.InstanceGenerator`](https://github.com/pyloa/PyLoa/blob/master/pyloa/instance/instancegenerator.py), \nwhich generates problem instances for the domain. As an example, for the `(k,n)-paging-problem` a simple generator could \nrandomly generate a sequence of requests of length `sequence_size`, whereas each request is within [1, n].   \n* `params[\"agent\"]`: Must define a configuration of a subclass implementation of [`pyloa.agent.Agent`](https://github.com/pyloa/PyLoa/blob/master/pyloa/agent/agent.py), \nwhich observes a state `s` of its environment, acts with action `a` accordingly, receives reward `r` and observes \ntransitioned state `s'`. For toy problem instances a simple Q-learning table implementation would suffice. \n* `params[\"environment\"]`: Must define a configuration of a subclass implementation of [`pyloa.environment.Environment`](https://github.com/pyloa/PyLoa/blob/master/pyloa/environment/environment.py), \nwhich consumes a problem instance and let's the agent *play* until it terminates. An `environment` constitutes as a problem\ndefinition.\n\nA minimal example for learning the `(5,6)-paging-problem` with a [`QTableAgent`](https://github.com/pyloa/PyLoa/blob/master/pyloa/agent/qtable.py) on a \n[`PagingEnvironment`](https://github.com/pyloa/PyLoa/blob/master/pyloa/environment/environment.py#L105) can be invoked with\n\n    pyloa train --config hyperparams.py\n\nand the hyperparams.py as following:\n\n```python\nfrom pyloa.instance import RandomSequenceGenerator\nfrom pyloa.environment import DefaultPagingEnvironment\nfrom pyloa.agent import QTableAgent\n\n# vars\nsequence_size = 1000\nmax_page = 6\nmin_page = 1\nepisodes = 250\n\n# hyperparams\nparams = {\n    'checkpoint_step': episodes//10,\n    'instance': {\n        'type': RandomSequenceGenerator,\n        'sequence_size': sequence_size,\n        'sequence_number': episodes,\n        'min_page': min_page,\n        'max_page': max_page,\n    },\n    'environment': {\n        'type': DefaultPagingEnvironment,\n        'sequence_size': sequence_size,\n        'cache_size': 5,\n        'num_pages': max_page - min_page + 1,\n    },\n    'agent': {\n        'type': QTableAgent,\n        'discount_factor': 0.55,\n        'learning_rate': 0.001,\n        'epsilon': 0.0,\n        'epsilon_delta': 13 / (episodes * 10),\n        'epsilon_max': 0.99,\n        'save_file': \"/home/me/models/\",\n    },\n}\n```\n\nThis example is defined in [examples/0_train_qtable_paging/hyperparams.py](https://github.com/pyloa/PyLoa/tree/master/examples/0_train_qtable_paging) and can be run with\n\n    pyloa train --config examples/0_train_qtable_paging/hyperparams.py\n\nThe resulting run can be seen [here](http://google.com). In total there are five toy examples, which can be run on any system, \ndefined in the examples directory.\n\n#### Runmodes\n\nPyLoa has three different runmodes: `train` ,`eval` and `gen`. There are slight adaptions to be made for the configuration file \ndepending on the selected runmode; we encourage checking the examples for reference (on a site note: hyperparams are loaded and validated\nin [pyloa.utils.load](https://github.com/pyloa/PyLoa/blob/master/pyloa/utils/load.py#L13)). Semantically the three different runmodes stand for: \n\n* train: An `RLAgent` will be trained for `episode`-many instances, generated by an `InstanceGenerator`, on his `environment`. \nEvery `checkpoint_step`-many instances a checkpoint of `RLAgent` will be saved.  \n* eval: All trained `RLagents` nested within `root_dir` will be evaluated on `episode`-many instances, generated by an `InstanceGenerator`.\nAdditionally non-trainable agents may be defined and evaluated alongside.\n* gen: Currently **only applicable** for the `(k,n)-paging-problem`. A genetic algorithm empirically determines a `PagingAgent`'s \n(approximate) competitive ratio. \n\nEach runmode will create TFEvent-files for TensorBoard in its experiment's output directory. \n\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/pyloa/PyLoa",
        "keywords": "on-line algorithms,paging,vertex coloring,machine learning",
        "license": "MIT",
        "maintainer": "PyLoa Developers",
        "maintainer_email": "pyloadevelopers@gmail.com",
        "name": "pyloa",
        "package_url": "https://pypi.org/project/pyloa/",
        "platform": "any",
        "project_url": "https://pypi.org/project/pyloa/",
        "project_urls": {
            "Bug Tracker": "https://github.com/pyloa/PyLoa/issues",
            "Documentation": "https://github.com/pyloa/PyLoa/tree/master/pyloa",
            "Homepage": "https://github.com/pyloa/PyLoa",
            "Source Code": "https://github.com/pyloa/PyLoa/tree/master/pyloa"
        },
        "release_url": "https://pypi.org/project/pyloa/1.0.3/",
        "requires_dist": [
            "matplotlib (==3.0.3)",
            "scipy (==1.2.1)",
            "tensorflow (==1.13.1)",
            "tqdm (==4.31.1)",
            "numpy (==1.16.2)"
        ],
        "requires_python": ">=3.5",
        "summary": "PyLoa: Learning on-line Algorithms with Python",
        "version": "1.0.3"
    },
    "last_serial": 5527075,
    "releases": {
        "1.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "67ade7f331ce41edb934f161de0560b5",
                    "sha256": "502cd72fcd66b15e13317b471d99e20f4a371014d6db532247889243c5cce078"
                },
                "downloads": -1,
                "filename": "pyloa-1.0.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "67ade7f331ce41edb934f161de0560b5",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5",
                "size": 67402,
                "upload_time": "2019-07-13T14:50:33",
                "url": "https://files.pythonhosted.org/packages/22/f7/655600403e3ac453f8c08edf4b6e98a1b23cf53753dda88e103f9ef89252/pyloa-1.0.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "1c047fce6265286b24e31f628aa68bdb",
                    "sha256": "f088d040456d2ab7d8b39acc1e90e051c4a4a614bdfe7312dbc54aecffc28da9"
                },
                "downloads": -1,
                "filename": "pyloa-1.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "1c047fce6265286b24e31f628aa68bdb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5",
                "size": 46114,
                "upload_time": "2019-07-13T14:50:36",
                "url": "https://files.pythonhosted.org/packages/70/7a/daa5c9d7a774bde7375a34e68c5f178b0ea9fb43ee903e3490c651e11ead/pyloa-1.0.3.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "67ade7f331ce41edb934f161de0560b5",
                "sha256": "502cd72fcd66b15e13317b471d99e20f4a371014d6db532247889243c5cce078"
            },
            "downloads": -1,
            "filename": "pyloa-1.0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "67ade7f331ce41edb934f161de0560b5",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.5",
            "size": 67402,
            "upload_time": "2019-07-13T14:50:33",
            "url": "https://files.pythonhosted.org/packages/22/f7/655600403e3ac453f8c08edf4b6e98a1b23cf53753dda88e103f9ef89252/pyloa-1.0.3-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "1c047fce6265286b24e31f628aa68bdb",
                "sha256": "f088d040456d2ab7d8b39acc1e90e051c4a4a614bdfe7312dbc54aecffc28da9"
            },
            "downloads": -1,
            "filename": "pyloa-1.0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "1c047fce6265286b24e31f628aa68bdb",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.5",
            "size": 46114,
            "upload_time": "2019-07-13T14:50:36",
            "url": "https://files.pythonhosted.org/packages/70/7a/daa5c9d7a774bde7375a34e68c5f178b0ea9fb43ee903e3490c651e11ead/pyloa-1.0.3.tar.gz"
        }
    ]
}