{
    "info": {
        "author": "Adri\u00e1n Herrera",
        "author_email": "adr.her.arc.95@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Intended Audience :: Science/Research",
            "License :: OSI Approved :: MIT License",
            "Natural Language :: English",
            "Programming Language :: Python :: 3.6",
            "Topic :: Scientific/Engineering :: Artificial Intelligence"
        ],
        "description": "HDeepRM\n=======\n\nFramework for evaluating *workload management* policies based on\n*deep reinforcement learning* for *heterogeneous* clusters.\n\n.. include-overview-start\n\nOverview\n--------\n\n*HDeepRM* is a Python framework for evaluating workload management policies\nbased on deep reinforcement learning for heterogeneous clusters. It\nleverages the `Batsim ecosystem <https://gitlab.inria.fr/batsim>`_\nfor simulating a heterogeneous workload management context. This is composed\nof the *simulator*, `Batsim <https://gitlab.inria.fr/batsim/batsim>`_ and the\n*decision system*, `PyBatsim <https://gitlab.inria.fr/batsim/pybatsim>`_.\n\nHDeepRM provides a heterogeneity layer on top of PyBatsim, which adds support\nfor user-defined resource hierarchies. Memory capacity and bandwidth conflicts\nare added along with interdependence when consolidating or scattering jobs across\nthe data centre.\n\nIt offers a flexible API for developing deep reinforcement learning agents.\nThese may be trained by providing real workload traces in\n`SWF format <http://www.cs.huji.ac.il/labs/parallel/workload/swf.html>`_ along\nwith platforms defined in the format specified in `Platforms <TODO>`_. They can\nbe further evaluated and tested against classic policies.\n\nInstallation Prerequisites\n~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nHDeepRM is distributed as a Python package on\n`PyPi <https://pypi.org/project/hdeeprm/>`_.\nIn order to download and install it, the following software is needed:\n\n- Python3.6+, find your OS in this\n  `installation guide <https://realpython.com/installing-python/>`_.\n- Pip, the Python package manager. If not already available with the Python\n  installation, follow the\n  `official guide <https://pip.pypa.io/en/stable/installing/>`_.\n\nInstallation\n~~~~~~~~~~~~\n\nFor installing HDeepRM, just download the package from PyPi:\n\n.. code-block:: bash\n\n  pip install --upgrade --user hdeeprm\n\nIf ``pip`` is mapped to Python 2.x, try:\n\n.. code-block:: bash\n\n  pip3 install --upgrade --user hdeeprm\n\nWhen working with multiple Python versions, use:\n\n.. code-block:: bash\n\n  python3.6 -m pip install --upgrade --user hdeeprm\n\nThis should download the ``hdeeprm`` package with all its dependencies,\nwhich are:\n\n- ``defusedxml`` >= 0.5.0: secure XML generation and parsing.\n- ``gym`` >= 0.12.0: environment, actions and observations definitions.\n- ``lxml`` >= 4.3.2: generation of the XML tree. Backend for ``defusedxml``.\n- ``numpy`` >= 1.16.2: efficient data structure operations.\n- ``procset`` >= 1.0: closed-interval sets for resource selection.\n- ``pybatsim`` >= 3.1.0: decision system and main interface to interact\n  with Batsim.\n- ``torch`` >= 1.0.1.post2: deep learning library for agent definition.\n\nUsage Prerequisites\n~~~~~~~~~~~~~~~~~~~\n\nThe simulation side is done by Batsim, which is needed in order to run\nHDeepRM experiments. Follow the `official installation docs\n<https://batsim.readthedocs.io/en/latest/installation.html>`_ for instructions.\n\nLaunching experiments\n~~~~~~~~~~~~~~~~~~~~~\n\nIn order to experiment with HDeepRM, an integrated launcher is provided:\n\n.. code-block:: bash\n\n  hdeeprm-launch -a <agent.py> -cw <custom_workload.json> -im <saved_model.pt> -om <to_save_model.pt> <options.json>\n\nThe ``options.json`` specifies the experiment parameters. The JSON structure\nis as follows:\n\n.. code-block:: json\n\n  {\n    \"seed\": 0,\n    \"nb_resources\": 0,\n    \"nb_jobs\": 0,\n    \"workload_file_path\": \"\",\n    \"platform_file_path\": \"\",\n    \"pybatsim\": {\n      \"log_level\": \"\",\n      \"env\": {\n        \"objective\": \"\",\n        \"actions\": {\n          \"selection\": [\n            {\"\": []}\n          ],\n          \"void\": false\n        },\n        \"observation\": \"\",\n        \"queue_sensitivity\": 0.0,\n      },\n      \"agent\": {\n        \"type\": \"\",\n        \"run\": \"\",\n        \"hidden\": 0,\n        \"lr\": 0.0,\n        \"gamma\": 0.0\n      }\n    }\n  }\n\nGlobal options:\n\n* ``seed`` - The random seed for evaluation reproducibility.\n* ``nb_resources`` - Total number of cores in the simulated platform.\n* ``nb_jobs`` - Total number of jobs to generate in the workload.\n* ``workload_file_path`` - Location of the original SWF formatted workload.\n* ``platform_file_path`` - Location of the original\n  HDeepRM JSON formatted platform.\n\nPyBatsim options:\n\n* ``log_level`` - Logging level for showing insights from the simulation. See `Logging <https://docs.python.org/3.6/howto/logging.html>`_ for reference on possible values.\n\nPyBatsim - Environment options:\n\n* ``objective`` - Metric to be optimised by the agent. See `Objectives <TODO>`_ for an explanation and recognised values.\n* ``actions`` - Subset of actions for the simulation. If not specified, all 37 actions in HDeepRM are used.\n* ``observation`` - Type of observation to use, one of *normal*, *small* or *minimal*.\n* ``queue_sensitivity`` - Sensitivity of the observation to variations in job queue size. See `Hyperparameters - Queue Sensitivity <TODO>`_.\n\nPyBatsim - Common agent options:\n\n* ``type`` - Type of the scheduling agent, one of *CLASSIC* or *LEARNING*.\n\nPyBatsim - `Learning <https://hdeeprm.readthedocs.io/en/latest/source/packages/hdeeprm.agent.html#hdeeprm.agent.LearningAgent>`_ agent options:\n\n* ``run`` - Type of run for the learning agent, one of *train* or *test*.\n  When training, the agent's inner model is updated,\n  whereas testing is meant for evaluation purposes.\n* ``hidden`` - Number of units in each hidden layer from the agent's inner model. See `Hyperparameters - Hidden units <TODO>`_.\n* ``lr`` - Learning rate for updating the agent's inner model. See `Hyperparameters - Learning rate <TODO>`_.\n* ``gamma`` - Discount factor for rewards. See `Hyperparameters - Reward Discount Factor <TODO>`_.\n\nThis is an example of an ``options.json`` file\nfor a classic agent:\n\n.. code-block:: json\n\n  {\n    \"seed\": 2009,\n    \"nb_resources\": 175,\n    \"nb_jobs\": 1000,\n    \"workload_file_path\": \"/workspace/workloads/my_workload.swf\",\n    \"platform_file_path\": \"/workspace/platforms/my_platform.json\",\n    \"pybatsim\": {\n      \"log_level\": \"DEBUG\",\n      \"env\": {\n        \"objective\": \"avg_utilization\",\n        \"actions\": {\n          \"selection\": [\n            {\"shortest\": [\"high_mem_bw\"]}\n          ],\n          \"void\": false\n        },\n        \"observation\": \"normal\",\n        \"queue_sensitivity\": 0.05\n      },\n      \"agent\": {\n        \"type\": \"CLASSIC\"\n      }\n    }\n  }\n\n\nThis is another example of an ``options.json`` file,\nin this case for a learning agent:\n\n.. code-block:: json\n\n  {\n    \"seed\": 1995,\n    \"nb_resources\": 175,\n    \"nb_jobs\": 1000,\n    \"workload_file_path\": \"/workspace/workloads/my_workload.swf\",\n    \"platform_file_path\": \"/workspace/platforms/my_platform.json\",\n    \"pybatsim\": {\n      \"log_level\": \"WARNING\",\n      \"env\": {\n        \"objective\": \"makespan\",\n        \"actions\": {\n          \"selection\": [\n            {\"first\": [\"high_gflops\", \"high_mem_bw\"]},\n            {\"smallest\": [\"\"]}\n          ],\n          \"void\": false\n        },\n        \"queue_sensitivity\": 0.01\n      },\n      \"agent\": {\n        \"type\": \"LEARNING\",\n        \"run\": \"train\",\n        \"hidden\": 128,\n        \"lr\": 0.001,\n        \"gamma\": 0.99\n      }\n    }\n  }\n\nOptional command line arguments are available:\n\n- ``-a`` - The file containing your developed learning agent for evaluation.\n  See `agent examples <TODO>`_ for reference.\n\n- ``-cw`` - If you are thinking about proof-of-concept experiments, you\n  may need to define your own workload. Doing this in SWF is tedious, thus\n  this option allows for passing a custom workload defined in Batsim JSON format.\n\n- ``-im`` - PyTorch trained models are usually saved in ``.pt`` files. This\n  option allows for loading a previously trained model to bootstrap the agent.\n\n- ``-om`` - If you want to save the model after the simulation is finished, specify\n  the output file in this option. This is usually combined with *train* runs.\n\n.. include-overview-end\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/RaMdsC/hdeeprm",
        "keywords": "deep reinforcement learning workload management job scheduling resource simulator framework heterogeneous cluster batsim",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "hdeeprm",
        "package_url": "https://pypi.org/project/hdeeprm/",
        "platform": "",
        "project_url": "https://pypi.org/project/hdeeprm/",
        "project_urls": {
            "Homepage": "https://github.com/RaMdsC/hdeeprm"
        },
        "release_url": "https://pypi.org/project/hdeeprm/0.3.1b1/",
        "requires_dist": [
            "defusedxml (>=0.5.0)",
            "evalys (>=4.0.4)",
            "gym (>=0.12.0)",
            "lxml (>=4.3.2)",
            "matplotlib (>=3.0.3)",
            "numpy (>=1.16.2)",
            "procset (>=1.0)",
            "pybatsim (>=3.1.0)",
            "torch (>=1.0.1.post2)"
        ],
        "requires_python": "",
        "summary": "Evaluate Deep Reinforcement Learning policies for heterogeneous Workload Management",
        "version": "0.3.1b1"
    },
    "last_serial": 4960215,
    "releases": {
        "0.1.0b0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a223e8ceab140fbbccc4017c5e90af43",
                    "sha256": "3ea57317d968aa2ec1818a6d803506cd358ed2312890afedfdf89604d8456cdc"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.1.0b0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a223e8ceab140fbbccc4017c5e90af43",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 25005,
                "upload_time": "2019-02-25T22:47:59",
                "url": "https://files.pythonhosted.org/packages/48/74/e27d35db2c641d0912776f4540aeee8b7518fbad0a49a3a0d3a3629c994c/hdeeprm-0.1.0b0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "ae7d34f1a3d1afa5a78ea168ea7a89cf",
                    "sha256": "d44d89fc6f16c5db2a3cf0018a56ed80a04378679b7020f4f00da79cf533ba9f"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.1.0b0.tar.gz",
                "has_sig": false,
                "md5_digest": "ae7d34f1a3d1afa5a78ea168ea7a89cf",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20922,
                "upload_time": "2019-02-25T22:48:01",
                "url": "https://files.pythonhosted.org/packages/29/6c/2ff817299a6a193020531ca01ba0df09f7505025b32119c17378cf16b17f/hdeeprm-0.1.0b0.tar.gz"
            }
        ],
        "0.1.0b1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a8260ebd68e6d1773bb8133d35057e03",
                    "sha256": "cb16bf22eba7291103307f8e8426ef6a723dbc042bd30b6e32f4112aefa7b30b"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.1.0b1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a8260ebd68e6d1773bb8133d35057e03",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 35041,
                "upload_time": "2019-02-27T17:58:09",
                "url": "https://files.pythonhosted.org/packages/ab/d2/d994abc7a0916f604b47c57e62a1cadff29945dc5d5dbe8b11fd545cb60c/hdeeprm-0.1.0b1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7ae081482c03954ae8297d82bc6a4a3b",
                    "sha256": "07c926ad490b84226bbf321f49426e831ef282101e0b004a28d84f580b3f8dc4"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.1.0b1.tar.gz",
                "has_sig": false,
                "md5_digest": "7ae081482c03954ae8297d82bc6a4a3b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 31992,
                "upload_time": "2019-02-27T17:58:10",
                "url": "https://files.pythonhosted.org/packages/4b/79/892e31ecd7a2137d3f576bdc848465427dbcc384d514b2b9de5317f73003/hdeeprm-0.1.0b1.tar.gz"
            }
        ],
        "0.2.1b0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "13d8dc09b18091fe6fe0261788a893a7",
                    "sha256": "568559e5b4d765b6448876751bdfe45832395021a617f10746133e810412b378"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.2.1b0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "13d8dc09b18091fe6fe0261788a893a7",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 38406,
                "upload_time": "2019-03-13T15:43:28",
                "url": "https://files.pythonhosted.org/packages/c0/ae/1379b610f2f627f0b6749fa5c9affd139a03c21c9ca860358cff27ed47ef/hdeeprm-0.2.1b0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "a0eb650a6cb2bafd46c5b115ce03a77e",
                    "sha256": "b2855892eb15a522341a2faa2bed4149faa86cc64c73a6f97fc2811408f22fa1"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.2.1b0.tar.gz",
                "has_sig": false,
                "md5_digest": "a0eb650a6cb2bafd46c5b115ce03a77e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 35363,
                "upload_time": "2019-03-13T15:43:29",
                "url": "https://files.pythonhosted.org/packages/5f/2b/7f99e7b86ba8835219b3ffa9e65108df619b2c99f7f1b2c618c91c96743d/hdeeprm-0.2.1b0.tar.gz"
            }
        ],
        "0.3.0b0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b229280339c2a8898ff7e40d11a2e8c2",
                    "sha256": "88851d5febb6446255a61a65b6fa7fc65df8fedc52379c03a99d129bc0a40af4"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.3.0b0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b229280339c2a8898ff7e40d11a2e8c2",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 40869,
                "upload_time": "2019-03-17T15:29:34",
                "url": "https://files.pythonhosted.org/packages/f5/c8/856365fedbc655f073d294bce3dbd4a253b779976721e534f7f2b9f54593/hdeeprm-0.3.0b0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "a4285ba5b673a4b0036db9b87159c753",
                    "sha256": "99e71590b1363f62b65e49607f27ea5d72131feee19dfe2977d1b6e2349a3b8c"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.3.0b0.tar.gz",
                "has_sig": false,
                "md5_digest": "a4285ba5b673a4b0036db9b87159c753",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 37852,
                "upload_time": "2019-03-17T15:29:36",
                "url": "https://files.pythonhosted.org/packages/08/6a/0a865bea905bd818cd96038f91c1154e6b0dc453537c805762afc830c764/hdeeprm-0.3.0b0.tar.gz"
            }
        ],
        "0.3.1b0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e3d81b621649f101b6d26114b811fc59",
                    "sha256": "cbb1285875387bf19d2b595a39b6a4505925756df4f03a689524042935966d35"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.3.1b0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "e3d81b621649f101b6d26114b811fc59",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 41325,
                "upload_time": "2019-03-18T18:29:19",
                "url": "https://files.pythonhosted.org/packages/5c/0b/f37706327d5aae327d374dfceb0805b018406330ebe4c6355acf65271f1d/hdeeprm-0.3.1b0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "af3e5fa68aabbe34ae2225b6acb2b3c7",
                    "sha256": "e40161b16502bda95e99fba071c8107861669b5fd5bbdfc1351c9695db0aa4a8"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.3.1b0.tar.gz",
                "has_sig": false,
                "md5_digest": "af3e5fa68aabbe34ae2225b6acb2b3c7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 38290,
                "upload_time": "2019-03-18T18:29:20",
                "url": "https://files.pythonhosted.org/packages/34/bb/addaee8345a253fd468c6e09f6fd7e9144e5bb7bf19100b14229f179e109/hdeeprm-0.3.1b0.tar.gz"
            }
        ],
        "0.3.1b1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0f4ce976311988b37fb7628f9baa5782",
                    "sha256": "eda0a33b2838723882b60790bec1a12535583d8cbed64da2a80c883120d1d3d1"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.3.1b1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "0f4ce976311988b37fb7628f9baa5782",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 41367,
                "upload_time": "2019-03-19T18:26:02",
                "url": "https://files.pythonhosted.org/packages/f5/eb/47e2f2dc951c9e0a3325dfc9ecae91644875d06456e1895560722f19ca0b/hdeeprm-0.3.1b1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "00bbae10dfc13ef4e9b678de50e90c7c",
                    "sha256": "b6c0a186571169de9ce15ec8284a3aa78b1cf7f0531439335f1db56238fa053f"
                },
                "downloads": -1,
                "filename": "hdeeprm-0.3.1b1.tar.gz",
                "has_sig": false,
                "md5_digest": "00bbae10dfc13ef4e9b678de50e90c7c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 38400,
                "upload_time": "2019-03-19T18:26:05",
                "url": "https://files.pythonhosted.org/packages/0e/55/d1d9b42bbf3688f548041448a386df7a8bc4ba36e081aef5a3b77b5c1a7f/hdeeprm-0.3.1b1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "0f4ce976311988b37fb7628f9baa5782",
                "sha256": "eda0a33b2838723882b60790bec1a12535583d8cbed64da2a80c883120d1d3d1"
            },
            "downloads": -1,
            "filename": "hdeeprm-0.3.1b1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "0f4ce976311988b37fb7628f9baa5782",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 41367,
            "upload_time": "2019-03-19T18:26:02",
            "url": "https://files.pythonhosted.org/packages/f5/eb/47e2f2dc951c9e0a3325dfc9ecae91644875d06456e1895560722f19ca0b/hdeeprm-0.3.1b1-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "00bbae10dfc13ef4e9b678de50e90c7c",
                "sha256": "b6c0a186571169de9ce15ec8284a3aa78b1cf7f0531439335f1db56238fa053f"
            },
            "downloads": -1,
            "filename": "hdeeprm-0.3.1b1.tar.gz",
            "has_sig": false,
            "md5_digest": "00bbae10dfc13ef4e9b678de50e90c7c",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 38400,
            "upload_time": "2019-03-19T18:26:05",
            "url": "https://files.pythonhosted.org/packages/0e/55/d1d9b42bbf3688f548041448a386df7a8bc4ba36e081aef5a3b77b5c1a7f/hdeeprm-0.3.1b1.tar.gz"
        }
    ]
}