{
"info": {
"author": "Thiago P. Bueno",
"author_email": "thiago.pbueno@gmail.com",
"bugtrack_url": null,
"classifiers": [
"Development Status :: 3 - Alpha",
"Environment :: Console",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
"Natural Language :: English",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3",
"Topic :: Scientific/Engineering :: Artificial Intelligence"
],
"description": "# tf-mdp [![Build Status][travis.svg]][travis] [![Documentation Status][readthedocs-badge]][readthedocs] [![License][license.svg]][license]\n\nProbabilistic planning in continuous state-action MDPs using TensorFlow.\n\n**tf-mdp** is an implementation based on the paper:\n\n> Thiago P. Bueno; Leliane N. de Barros; Denis D. Mau\u00e1; Scott Sanner
\n> **Deep Reactive Policies for Planning in Stochastic Nonlinear Domains**
\n> In AAAI, 2019.\n\n# Quickstart\n\n**tf-mdp** is a Python3.5+ package available in PyPI.\n\n```text\n$ pip3 install tf-mdp\n```\n\nPlease make sure you have a running TensorFlow version on your system before pip-installing this package.\n\n# Features\n\n**tf-mdp** solves discrete-time continuous state-action MDPs.\n\nThe domains/instances are specified using the [RDDL][rddl] language.\n\nIt is built on the following packages available on the Python3 RDDL toolkit:\n\n- [pyrddl][pyrddl]: RDDL lexer/parser.\n- [rddlgym][rddlgym]: A toolkit for working with RDDL domains.\n- [rddl2tf][rddl2tf]: RDDL2TensorFlow compiler.\n- [tf-rddlsim][tf-rddlsim]: A RDDL simulator running in TensorFlow.\n\nPlease refer to each project documentation for further details.\n\n\n# Usage\n\n```text\n$ tfmdp --help\n\nusage: tfmdp [-h] [-l LAYERS [LAYERS ...]]\n [-a {none,sigmoid,tanh,relu,relu6,crelu,elu,selu,softplus,softsign}]\n [-iln] [-b BATCH_SIZE] [-hr HORIZON] [-e EPOCHS]\n [-lr LEARNING_RATE]\n [-opt {Adadelta,Adagrad,Adam,GradientDescent,ProximalGradientDescent,ProximalAdagrad,RMSProp}]\n [-lfn {linear,mse}] [-ld LOGDIR] [-v]\n rddl\n\nProbabilistic planning in continuous state-action MDPs using TensorFlow.\n\npositional arguments:\n rddl RDDL file or rddlgym domain id\n\noptional arguments:\n -h, --help show this help message and exit\n -l LAYERS [LAYERS ...], --layers LAYERS [LAYERS ...]\n number of units in each hidden layer in policy network\n -a {none,sigmoid,tanh,relu,relu6,crelu,elu,selu,softplus,softsign}, --activation {none,sigmoid,tanh,relu,relu6,crelu,elu,selu,softplus,softsign}\n activation function for hidden layers in policy\n network\n -iln, --input-layer-norm\n input layer normalization flag\n -b BATCH_SIZE, --batch-size BATCH_SIZE\n number of trajectories in a batch (default=256)\n -hr HORIZON, --horizon HORIZON\n number of timesteps (default=40)\n -e EPOCHS, --epochs EPOCHS\n number of timesteps (default=200)\n -lr LEARNING_RATE, --learning-rate LEARNING_RATE\n optimizer learning rate (default=0.001)\n -opt {Adadelta,Adagrad,Adam,GradientDescent,ProximalGradientDescent,ProximalAdagrad,RMSProp}, --optimizer {Adadelta,Adagrad,Adam,GradientDescent,ProximalGradientDescent,ProximalAdagrad,RMSProp}\n loss optimizer (default=RMSProp)\n -lfn {linear,mse}, --loss-fn {linear,mse}\n loss function (default=linear)\n -ld LOGDIR, --logdir LOGDIR\n log directory for data summaries (default=/tmp/tfmdp)\n -v, --verbose verbosity mode\n```\n\n# Examples\n\n```text\n$ tfmdp Reservoir-20 -l 2048 -iln -a elu -b 256 -hr 40 -e 200 -lr 0.001 -lfn mse -v\n\nRunning tf-mdp v0.5.2 ...\n\n>> RDDL: Reservoir-20\n>> logdir: /tmp/tfmdp\n\n>> Policy Net:\nlayers = [2048]\nactivation = elu\ninput layer norm = True\n\n>> Hyperparameters:\nepochs = 200\nlearning rate = 0.001\nbatch size = 256\nhorizon = 40\n\n>> Optimization:\noptimizer = RMSProp\nloss function = mse\n\n>> Loading model ...\nDone in 0.059091 sec.\n\n>> Optimizing...\n2019-04-15 16:17:16.383099: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2\nEpoch 199: loss = 1036054272.0000000\nDone in 184.721894 sec.\n\n>> Performance:\ntotal reward = -3637.6018, reward per timestep = -90.9400\n```\n\n```text\n$ tfmdp HVAC-3 -l 256 128 64 32 -iln -a elu -b 256 -hr 40 -e 200 -lr 0.0001 -lfn mse -v\n\nRunning tf-mdp v0.5.2 ...\n\n>> RDDL: HVAC-3\n>> logdir: /tmp/tfmdp\n\n>> Policy Net:\nlayers = [256,128,64,32]\nactivation = elu\ninput layer norm = True\n\n>> Hyperparameters:\nepochs = 200\nlearning rate = 0.0001\nbatch size = 256\nhorizon = 40\n\n>> Optimization:\noptimizer = RMSProp\nloss function = mse\n\n>> Loading model ...\nDone in 0.042337 sec.\n\n>> Optimizing...\n2019-04-15 16:20:25.744165: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2\nEpoch 199: loss = 131730186240.0000000\nDone in 60.739938 sec.\n\n>> Performance:\ntotal reward = -305691.7500, reward per timestep = -7642.2937\n```\n\n```text\n$ tfmdp Navigation-v2 -l 256 128 64 32 -a elu -b 128 -hr 20 -e 200 -lr 0.001 -lfn mse -v\n\nRunning tf-mdp v0.5.2 ...\n\n>> RDDL: Navigation-v2\n>> logdir: /tmp/tfmdp\n\n>> Policy Net:\nlayers = [256,128,64,32]\nactivation = elu\ninput layer norm = False\n\n>> Hyperparameters:\nepochs = 200\nlearning rate = 0.001\nbatch size = 128\nhorizon = 20\n\n>> Optimization:\noptimizer = RMSProp\nloss function = mse\n\n>> Loading model ...\nDone in 0.038808 sec.\n\n>> Optimizing...\n2019-04-15 16:21:30.444619: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2\nEpoch 199: loss = 6183.8642586\nDone in 19.277676 sec.\n\n>> Performance:\ntotal reward = -78.4958, reward per timestep = -3.9248\n```\n\n# Documentation\n\nPlease refer to [https://tf-mdp.readthedocs.io/][readthedocs] for the code documentation.\n\n\n# Support\n\nIf you are having issues with tf-mdp, please let me know at: [thiago.pbueno@gmail.com](mailto://thiago.pbueno@gmail.com).\n\n# License\n\nCopyright (c) 2018-2019 Thiago Pereira Bueno All Rights Reserved.\n\ntf-mdp is free software: you can redistribute it and/or modify it\nunder the terms of the GNU Lesser General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or (at\nyour option) any later version.\n\ntf-mdp is distributed in the hope that it will be useful, but\nWITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser\nGeneral Public License for more details.\n\nYou should have received a copy of the GNU Lesser General Public License\nalong with tf-mdp. If not, see http://www.gnu.org/licenses/.\n\n[license.svg]: https://img.shields.io/aur/license/yaourt.svg\n[license]: https://github.com/thiagopbueno/tf-mdp/blob/master/LICENSE\n[pyrddl]: https://github.com/thiagopbueno/pyrddl\n[rddl2tf]: https://github.com/thiagopbueno/rddl2tf\n[rddl]: http://users.cecs.anu.edu.au/~ssanner/IPPC_2011/RDDL.pdf\n[rddlgym]: https://github.com/thiagopbueno/rddlgym\n[readthedocs-badge]: https://readthedocs.org/projects/tf-mdp/badge/?version=latest\n[readthedocs]: https://tf-mdp.readthedocs.io/en/latest\n[tf-rddlsim]: https://github.com/thiagopbueno/tf-rddlsim\n[travis.svg]: https://travis-ci.org/thiagopbueno/tf-mdp.svg?branch=master\n[travis]: https://travis-ci.org/thiagopbueno/tf-mdp",
"description_content_type": "",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/thiagopbueno/tf-mdp",
"keywords": "planning,tensorflow,rddl,mdp",
"license": "GNU General Public License v3.0",
"maintainer": "",
"maintainer_email": "",
"name": "tf-mdp",
"package_url": "https://pypi.org/project/tf-mdp/",
"platform": "",
"project_url": "https://pypi.org/project/tf-mdp/",
"project_urls": {
"Homepage": "https://github.com/thiagopbueno/tf-mdp"
},
"release_url": "https://pypi.org/project/tf-mdp/0.5.3/",
"requires_dist": null,
"requires_python": "",
"summary": "Probabilistic planning in continuous state-action MDPs using TensorFlow.",
"version": "0.5.3"
},
"last_serial": 5152746,
"releases": {
"0.1.0": [
{
"comment_text": "",
"digests": {
"md5": "362a68aa74f7ec77d56ba0e8c099a4c6",
"sha256": "b4c4053b7a9efb702e627e22692cf9ad2b9cd4352ea9a2d2e323b26a1ac9971d"
},
"downloads": -1,
"filename": "tf-mdp-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "362a68aa74f7ec77d56ba0e8c099a4c6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 1544,
"upload_time": "2018-08-20T13:48:10",
"url": "https://files.pythonhosted.org/packages/b9/f1/612135802cfc260d39225d08e43e679859be0e2a030cdfa36d3b9e878fe8/tf-mdp-0.1.0.tar.gz"
}
],
"0.1.1": [
{
"comment_text": "",
"digests": {
"md5": "84e7ce293619afd05a9aa9ea446a610b",
"sha256": "da4d254466d602bbf4d107ec4a66f382333ca74602a03c1c380a3b44ad89babd"
},
"downloads": -1,
"filename": "tf-mdp-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "84e7ce293619afd05a9aa9ea446a610b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5701,
"upload_time": "2018-08-20T20:58:22",
"url": "https://files.pythonhosted.org/packages/48/43/d6fd8fc07153f03ea04bc8ab85518c9ae6b9e1f43bed6f7ad40a03bc21ec/tf-mdp-0.1.1.tar.gz"
}
],
"0.2.0": [
{
"comment_text": "",
"digests": {
"md5": "2618b517ecfe61f6aa4ff70775e08a1d",
"sha256": "ae77c9aaa4a2962a77d0513b4d2913b1930fbd2f99ff301465c7e900ec6f55c2"
},
"downloads": -1,
"filename": "tf-mdp-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "2618b517ecfe61f6aa4ff70775e08a1d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 10234,
"upload_time": "2018-08-30T22:15:43",
"url": "https://files.pythonhosted.org/packages/ff/71/6fff0b03033659c3fa36537d7056676e55a7c567a4d1cb826cd27bf28752/tf-mdp-0.2.0.tar.gz"
}
],
"0.3.0": [
{
"comment_text": "",
"digests": {
"md5": "8ccf73beed5ea91aae6881a305be523d",
"sha256": "be6afa09dbd1abcb6d6dae395bd4180b0974a2e58390df0b64e1f16cb6bd92aa"
},
"downloads": -1,
"filename": "tf-mdp-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "8ccf73beed5ea91aae6881a305be523d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 10183,
"upload_time": "2018-08-31T14:54:17",
"url": "https://files.pythonhosted.org/packages/56/a7/2cfe766838bc87196e65dc2f2c0e7b41b528502bec1e4dd8d86fb3b69618/tf-mdp-0.3.0.tar.gz"
}
],
"0.4.1": [
{
"comment_text": "",
"digests": {
"md5": "9438f403b6b6ed9d1035271c3ea7b884",
"sha256": "8be919008561788bb950b5921b58e9928a2be982547efc2c079cc6fe25b8fea1"
},
"downloads": -1,
"filename": "tf-mdp-0.4.1.tar.gz",
"has_sig": false,
"md5_digest": "9438f403b6b6ed9d1035271c3ea7b884",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 6251,
"upload_time": "2018-10-21T15:09:18",
"url": "https://files.pythonhosted.org/packages/ba/35/c5cfdfa86cd90aa3b2b054f46fd3c1d1b801a15a7118d9f481466dd5464d/tf-mdp-0.4.1.tar.gz"
}
],
"0.4.3": [
{
"comment_text": "",
"digests": {
"md5": "f436dd89b91f25b79106b000f77b48ac",
"sha256": "dd1c05df9363c53cb8e2410724088f8c7ccd424867f12310ceee8b6a68130a95"
},
"downloads": -1,
"filename": "tf-mdp-0.4.3.tar.gz",
"has_sig": false,
"md5_digest": "f436dd89b91f25b79106b000f77b48ac",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 10422,
"upload_time": "2018-11-05T16:35:15",
"url": "https://files.pythonhosted.org/packages/39/c5/435084f38ecbcc80d77a593312dd8d3d8071cf9ce5af88f05e1a2aedb141/tf-mdp-0.4.3.tar.gz"
}
],
"0.4.4": [
{
"comment_text": "",
"digests": {
"md5": "51a3403946b8bd480f3ce4557e8dc1c1",
"sha256": "073ec2c48dfac4eaabc860de4db40d50af5ff46e5e74f25dcec85d5c359d24c6"
},
"downloads": -1,
"filename": "tf-mdp-0.4.4.tar.gz",
"has_sig": false,
"md5_digest": "51a3403946b8bd480f3ce4557e8dc1c1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 10511,
"upload_time": "2018-11-15T22:16:10",
"url": "https://files.pythonhosted.org/packages/7e/34/1720fcf39a29bf053cd1844da4aa59d441e67253befe9a2260b55ea2fe4f/tf-mdp-0.4.4.tar.gz"
}
],
"0.5.0": [
{
"comment_text": "",
"digests": {
"md5": "f58e8a4eb8c009cbbbc3023ceb74e3fe",
"sha256": "529f16861f4ef1c3d963db8c6984484ce524c71b14f80bcc8ff66920bc70bfa1"
},
"downloads": -1,
"filename": "tf-mdp-0.5.0.tar.gz",
"has_sig": false,
"md5_digest": "f58e8a4eb8c009cbbbc3023ceb74e3fe",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 15911,
"upload_time": "2019-04-02T21:39:21",
"url": "https://files.pythonhosted.org/packages/f9/b5/8d0002dead6630d87618a3c6a72308713a7c617d6680bfaa9cde25d91044/tf-mdp-0.5.0.tar.gz"
}
],
"0.5.2": [
{
"comment_text": "",
"digests": {
"md5": "23d96f7fe9c32bf78fef5b0101641a8e",
"sha256": "b281184160d74ea5cf07ac8608fd57ccfc762ba3cca40492eeffea692b4e9a7c"
},
"downloads": -1,
"filename": "tf-mdp-0.5.2.tar.gz",
"has_sig": false,
"md5_digest": "23d96f7fe9c32bf78fef5b0101641a8e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 19435,
"upload_time": "2019-04-15T20:31:35",
"url": "https://files.pythonhosted.org/packages/be/d5/138720e50885f3353f3ae91135077f4b625a0893a5fb403e752efbc0a9e0/tf-mdp-0.5.2.tar.gz"
}
],
"0.5.3": [
{
"comment_text": "",
"digests": {
"md5": "bd682f05b171bcfcdff6ff318e341ebc",
"sha256": "42029f1e6c0a77b8af9c2293bd58b9c677192ad14ddacd285b62cf642759f9da"
},
"downloads": -1,
"filename": "tf-mdp-0.5.3.tar.gz",
"has_sig": false,
"md5_digest": "bd682f05b171bcfcdff6ff318e341ebc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 21151,
"upload_time": "2019-04-17T01:28:10",
"url": "https://files.pythonhosted.org/packages/6b/90/f47af467058dfe835f8f2fc0277e041d535d44fb4bf29575439edca19844/tf-mdp-0.5.3.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "bd682f05b171bcfcdff6ff318e341ebc",
"sha256": "42029f1e6c0a77b8af9c2293bd58b9c677192ad14ddacd285b62cf642759f9da"
},
"downloads": -1,
"filename": "tf-mdp-0.5.3.tar.gz",
"has_sig": false,
"md5_digest": "bd682f05b171bcfcdff6ff318e341ebc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 21151,
"upload_time": "2019-04-17T01:28:10",
"url": "https://files.pythonhosted.org/packages/6b/90/f47af467058dfe835f8f2fc0277e041d535d44fb4bf29575439edca19844/tf-mdp-0.5.3.tar.gz"
}
]
}