{ "info": { "author": "Ethan Brooks", "author_email": "ethanbrooks@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3", "Topic :: Scientific/Engineering :: Artificial Intelligence" ], "description": "Tensorflow Run Manager\n======================\n\nMachine learning engineers often run multiple versions of an algorithm concurrently. However, this can make keeping track of and reproducing runs difficult. This simple utility solves this problem by maintaining a database in human-readable YAML formal that tracks\n\n* A unique name assigned to each run.\n* A description of each run.\n* The exact command used for the run.\n* The date and time of the run.\n* The most recent commit before the run.\n\nInstallation\n------------\nThe only external prerequisites of this tool are ``tmux`` and ``git``. After that, ``pip install run-manager``.\n\nImportant paths and files\n-------------------------\nWhen you run ``runs new``, the utility automatically creates the following directory structure:\n\n.. code-block:: console\n\n /\n \n checkpoints/\n tensorboard//\n\nRuns Database\n~~~~~~~~~~~~~\nYAML file that stores historical information about Tensorflow runs.\n\nRun Name\n~~~~~~~~\nThis is a unique value that you assign to each run. The ``runs`` section explains how the program deals with collisions.\n\n``checkpoints`` directory\n~~~~~~~~~~~~~~~~~~~~~~~~~\nDirectory where model checkpoints are saved. Used in ``tf.train.Saver().save(sess, /.ckpt)``.\n\n``tensorboard`` directory\n~~~~~~~~~~~~~~~~~~~~~~~~~\nDirectory where events are saved. Used in ``tf.summary.FileWriter(//)``.\n\nConfiguration\n-------------\nRuns can be extensively configured using command-line arguments, but the following values can also be configured in a ``.runsrc`` file:\n\n=================== =============== ======================================================================================================================================================\nname default description\n=================== =============== ======================================================================================================================================================\n``runs-dir`` ``.runs/`` The name to use for your Runs Directory.\n``db-filename`` ``.runs.yml`` The name that you choose to save your runs database with.\n``tb-dir-flag`` ``--tb-dir`` The flag that gets passed to your program that specifies ``//``. If ``None``, no flag will be passed to your program.\n``save-path-flag`` ``--save-path`` The flag that gets passed to your program that specifies ``/``. If ``None``, no flag will be passed to your program.\n``column-width`` ``30`` The default column width for the ``runs table`` command.\n``virtualenv-path`` ``None`` The path to your virtual environment directory, if you're using one. Used in the following command: ``Source /bin/activate``.\n``extra-flags`` ``[]`` Flag, value pairs for extra, custom flags. The strings ```` and ```` will get replaced with the appropriate value.\n=================== =============== ======================================================================================================================================================\n\nThe program expects to find the ``.runsrc`` in a parent of the current working directory. Unless specified otherwise, the ``.runs/`` directory will be adjacent to the ``.runsrc`` in the file structure.\n\nHere is an example ``.runsrc`` file:\n\n.. code-block:: yaml\n\n runs-dir: .lstm-runs/\n db-filename: lstm-runs.yml\n tb-dir-flag: None\n save-path-flag: -s\n column-width: 10\n virtualenv-path: /home/ethan/virtualenvs/baselines/\n extra-flags:\n - [--goal-log-dir, /goal-logs/.log]\n\nAssumptions\n-----------\nThis program tries to assume as little about your program as possible, while providing useful functionality. These assumptions are as follows:\n\n* You call the ``runs`` command from a directory whose parent contains the runs directory.\n* Your program lives in a Git repository.\n* The Git working tree is not dirty (if it is, the program will throw an informative error).\n* Your program accepts a ``--tb-dir`` flag, which your program uses in ``tf.train.Saver().save(sess, )``, and a ``--save-path`` flag, which your program uses in ``tf.train.Saver().restore(sess, )``. If your flags are different and you don't feel like changing them, you can specify the new flag names using command-line arguments (``--tb-dir-flag`` and ``--save-path-flag``) or in your ``.runsrc`` (see the `Configuration`_ section for more info). If you don't want to pass either flag to your program, set ``--tb-dir-flag`` or ``--save-path-flag`` (or the associated values in your ``.runsrc``) to `None`.\n\n\nSubcommands\n-----------\nFor detailed descriptions of each subcommand and its arguments, run\n\n.. code-block:: console\n\n runs -h\n\n``new``\n~~~~~~~\nStart a new run and build the file structure (see `Important paths and files`_).\n\nIt will add an entry to the database keyed by name, with the following values:\n\n* command\n* commit\n* datetime\n* description\n* host\n\nFinally, it will execute the command in ``tmux``.\n\n.. code-block:: console\n\n runs new 'run-name' 'python main.py' --description='Description of program'\n\n*Note:* the ``--tb-dir`` and ``--save-path`` flags will be automatically\nappended to this command argument, so do not include them in the ````\nargument.\n\n``delete``\n~~~~~~~~~~\nDelete all runs matching pattern. This command also deletes associated tensorboard and checkpoint files.\n\n.. code-block:: console\n\n \u276f runs delete \"continuous.*\"\n Delete the following runs?\n continuous0\n continuous1\n continuous21509805012\n continuous2\n continuous11509804959\n continuous3\n continuous31509805040\n\n``list``\n~~~~~~~~\nList all runs matching pattern.\n\n.. code-block:: console\n\n \u276f runs list --pattern=\"continuous.*\"\n continuous21509805012\n continuous0\n continuous11509804959\n continuous31509805040\n continuous1\n continuous2\n continuous3\n\n``table``\n~~~~~~~~~\nDisplay entries in run-database in table form.\n\n.. code-block:: console\n\n \u276f runs table\n name command commit datetime description host\n ----------------------------- --------------------------------- --------------------------------- -------------------------- --------------------------------- ------\n continuous2 CUDA_VISIBLE_DEVICES=1 python ... 90c0ad704e54d5152d897a4e978cc7... 2017-11-03T13:46:48.633364 Run multiple runs to test stoc... rldl3\n continuous3 CUDA_VISIBLE_DEVICES=1 python ... 90c0ad704e54d5152d897a4e978cc7... 2017-11-03T13:47:09.951233 Run multiple runs to test stoc... _\n continuous1 CUDA_VISIBLE_DEVICES=1 python ... 90c0ad704e54d5152d897a4e978cc7... 2017-11-03T13:42:39.879031 Run multiple runs to test stoc... _\n house-cnn-no-current-pos python train.py --timesteps-pe... 9fb9b5a 2017-10-28T18:07:44.246089 This is the refactored CNN on ... _\n room-with-original-cnn python run_custom.py --timeste... 8a5e1c2 2017-10-28T17:09:49.971061 Test original cnn on room.mjcf _\n continuous11509804959 CUDA_VISIBLE_DEVICES=1 python ... 90c0ad704e54d5152d897a4e978cc7... 2017-11-04T10:15:59.373633 Run multiple runs to test stoc... _\n continuous31509805040 CUDA_VISIBLE_DEVICES=1 python ... 90c0ad704e54d5152d897a4e978cc7... 2017-11-04T10:17:20.286275 Run multiple runs to test stoc... rldl4\n room-cnn-no-current-pos python train.py --timesteps-pe... 2873fbf 2017-10-28T18:08:10.615461 This is the refactored CNN on ... rldl4\n continuous21509805012 CUDA_VISIBLE_DEVICES=1 python ... 90c0ad704e54d5152d897a4e978cc7... 2017-11-04T10:16:52.129656 Run multiple runs to test stoc... _\n\n\nTo filter by regex, use ``--pattern`` flag.\n\n``lookup``\n~~~~~~~~~~\nLookup specific value associated with database entry.\n\n.. code-block:: console\n\n \u276f runs lookup continuous0 commit\n da6030dd973c810c330d9635eb8d9c2105bdfe2f\n\n``reproduce``\n~~~~~~~~~~~~~\nPrint out commands for reproducing run.\n\n.. code-block:: console\n\n \u276f runs reproduce continuous0 \n To reproduce:\n git checkout da6030dd973c810c330d9635eb8d9c2105bdfe2f\n runs new continuous0 'python run_custom.py --timesteps-per-batch=2048 --continuous-actions --neg-reward --use-cnn' --description='None'\n\n\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/lobachevzky/tf-run-manager", "keywords": "tensorflow utilities development", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "tf-run-manager", "package_url": "https://pypi.org/project/tf-run-manager/", "platform": "", "project_url": "https://pypi.org/project/tf-run-manager/", "project_urls": { "Homepage": "https://github.com/lobachevzky/tf-run-manager" }, "release_url": "https://pypi.org/project/tf-run-manager/2.1.6/", "requires_dist": [ "paramiko (==2.3.1)", "termcolor (==1.1.0)", "PyYAML (==3.12)", "tabulate (==0.8.1)" ], "requires_python": "", "summary": "A utility for tracking and reproducing Tensorflow runs.", "version": "2.1.6" }, "last_serial": 3524920, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "b4c7878ac1254a25da1cb661322b7c4c", "sha256": "ad6965a375ded7246cd7811a71e2d94c22c35a27a0964c4fbdcc44c6a1ec6580" }, "downloads": -1, "filename": "tf_run_manager-1.0.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "b4c7878ac1254a25da1cb661322b7c4c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 6306, "upload_time": "2017-11-07T20:58:40", "url": "https://files.pythonhosted.org/packages/ae/b6/9015eee4cd72e3a0700bbf4d8c38167e412c338f431bd0fc68428f9a3f2f/tf_run_manager-1.0.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "72834a561d95406115f7cd80f8721269", "sha256": "3f80046ce4f01177b7ad08b59ec1a37b15aa11651233cf52fcd27ed504de8971" }, "downloads": -1, "filename": "tf-run-manager-1.0.0.tar.gz", "has_sig": false, "md5_digest": "72834a561d95406115f7cd80f8721269", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3668, "upload_time": "2017-11-07T20:58:41", "url": "https://files.pythonhosted.org/packages/cb/ac/243223455ad5bcdecf691e98b99abb35ba63f6fb78976bbf761dac624ec9/tf-run-manager-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "da1e4c8e33cf7fc07234ec2fc17ea03b", "sha256": "1925ae190ef28ec44026f5e53b4e73a068e651226fbd67f81826c6298bd3c4aa" }, "downloads": -1, "filename": "tf_run_manager-1.0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "da1e4c8e33cf7fc07234ec2fc17ea03b", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 6249, "upload_time": "2017-11-08T15:59:41", "url": "https://files.pythonhosted.org/packages/4b/30/292c85b1249943318b328253ec0eb7a3349309be150c20925fbfa6ece674/tf_run_manager-1.0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "99ccb559d0f113afe94a6a8f440b953b", "sha256": "f3624ec1f81b8de2ec720740a7fb7fd5c6a5f9ae5624ed04ee9599543e350dcd" }, "downloads": -1, "filename": "tf-run-manager-1.0.1.tar.gz", "has_sig": false, "md5_digest": "99ccb559d0f113afe94a6a8f440b953b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3653, "upload_time": "2017-11-08T15:59:44", "url": "https://files.pythonhosted.org/packages/ef/b4/b09ad3deaa5c18672ed10b87b783e3ca5c3aed2d47096c172d179d20a100/tf-run-manager-1.0.1.tar.gz" } ], "1.0.2": [ { "comment_text": "", "digests": { "md5": "a679b7452ce4d9d92b03940e0891a5fb", "sha256": "7c3e6f1be58f2db978be2646b1d4fae0ac247a1cdc39117be0cd835873b1fe95" }, "downloads": -1, "filename": "tf_run_manager-1.0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a679b7452ce4d9d92b03940e0891a5fb", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7819, "upload_time": "2017-11-08T18:42:32", "url": "https://files.pythonhosted.org/packages/73/3b/d39ce06f76ad5e1dfcc741d78c23fef8d2c43b0ec4d34e957785c04503f4/tf_run_manager-1.0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "41396db041455a62408ae60eab7569f4", "sha256": "3a295c97a2676b8fe90a6defdbca06f465e59c36718f647449778191fb4a93c9" }, "downloads": -1, "filename": "tf-run-manager-1.0.2.tar.gz", "has_sig": false, "md5_digest": "41396db041455a62408ae60eab7569f4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4562, "upload_time": "2017-11-08T18:42:33", "url": "https://files.pythonhosted.org/packages/e0/c1/da0374bd05a462bc42e058bd70c218731624470d31be6d132fb2414ec40a/tf-run-manager-1.0.2.tar.gz" } ], "1.0.3": [ { "comment_text": "", "digests": { "md5": "2cb9ad796a5fd16f6c436e251b4e1bc1", "sha256": "6fecc8345df84396642ebff93ee56177ad0e549624a9d683b21a132cc7101e24" }, "downloads": -1, "filename": "tf_run_manager-1.0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "2cb9ad796a5fd16f6c436e251b4e1bc1", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8159, "upload_time": "2017-11-08T22:20:21", "url": "https://files.pythonhosted.org/packages/19/b3/c4891d67ac77f2939d0ac3d666860e3b66f9981cf97002d08a75991d1233/tf_run_manager-1.0.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "6861796b4401044300948e7ebd395f40", "sha256": "c3a4a8fe4c8a2d7e50d5338ba05863cf671215e3b451d88deca52ef6b27a42db" }, "downloads": -1, "filename": "tf-run-manager-1.0.3.tar.gz", "has_sig": false, "md5_digest": "6861796b4401044300948e7ebd395f40", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4827, "upload_time": "2017-11-08T22:20:22", "url": "https://files.pythonhosted.org/packages/74/14/2559227e03161a7b91cdca9c38c438416b17c7ba71a303b3e65e992f7013/tf-run-manager-1.0.3.tar.gz" } ], "1.0.4": [ { "comment_text": "", "digests": { "md5": "4201319e013315f45b2afbf6694be408", "sha256": "1de6e289b0a270382388d3fef001284c7ad1d0cf60533c526484bbb5bbae5f80" }, "downloads": -1, "filename": "tf_run_manager-1.0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "4201319e013315f45b2afbf6694be408", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8322, "upload_time": "2017-11-08T22:55:59", "url": "https://files.pythonhosted.org/packages/92/6f/339d4a574fe25d9a01bb33eb1b4910ac90c1953bea09a5689b89d4c55836/tf_run_manager-1.0.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2ccce61d3de753d2106637115e7d1eea", "sha256": "40099a5e4fa7c142622843b6bd10f94996d7ac8603024c607b9abb6d6cfccdd6" }, "downloads": -1, "filename": "tf-run-manager-1.0.4.tar.gz", "has_sig": false, "md5_digest": "2ccce61d3de753d2106637115e7d1eea", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4922, "upload_time": "2017-11-08T22:56:00", "url": "https://files.pythonhosted.org/packages/de/91/86056513b0363bc9760903a3bd7ca4889529aa12f04692828f5ec6361cae/tf-run-manager-1.0.4.tar.gz" } ], "1.0.5": [ { "comment_text": "", "digests": { "md5": "7d98d78e24b568841240167aada95bfa", "sha256": "4e48563367ef81e53f267dd309544501c7ca8c060c56079ba46dd423a9e4e25e" }, "downloads": -1, "filename": "tf_run_manager-1.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "7d98d78e24b568841240167aada95bfa", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 13169, "upload_time": "2017-12-01T23:04:08", "url": "https://files.pythonhosted.org/packages/18/8b/e127c3b6a7600b9ddf4397499643fdcfcf7c35f351011a1b839087808a73/tf_run_manager-1.0.5-py2.py3-none-any.whl" } ], "1.0.6": [ { "comment_text": "", "digests": { "md5": "fd039f1634e8dfff7ffc672998fcd110", "sha256": "2c15157cdef249d12c938aeb2f49a6043dd8aca6951f745d5691e463ac76c613" }, "downloads": -1, "filename": "tf_run_manager-1.0.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "fd039f1634e8dfff7ffc672998fcd110", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 13156, "upload_time": "2017-12-01T23:11:16", "url": "https://files.pythonhosted.org/packages/b7/68/915c90410a1710d730c30a1e5cb2d9612d9b91b0a753bf4aea5aebc9203d/tf_run_manager-1.0.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c86e3512bec497050a2501dbd1108360", "sha256": "d58c41936a8c1796e4986432709c132fb6afbc77bdbb2220b2f699fad8e409aa" }, "downloads": -1, "filename": "tf-run-manager-1.0.6.tar.gz", "has_sig": false, "md5_digest": "c86e3512bec497050a2501dbd1108360", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9475, "upload_time": "2017-12-01T23:11:17", "url": "https://files.pythonhosted.org/packages/3b/66/18807b043dbb90b55f78cf1f55b8cf8e8cb0880b4866ce61575737f2b4e1/tf-run-manager-1.0.6.tar.gz" } ], "1.1.6": [ { "comment_text": "", "digests": { "md5": "a1bab6c3ec6a4b45772c3ba57d5df2d6", "sha256": "5c481ec6ce1ebd48655e3c515c7f6c3a432cbdf522c2650e1fb562f090c1cf95" }, "downloads": -1, "filename": "tf_run_manager-1.1.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a1bab6c3ec6a4b45772c3ba57d5df2d6", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 13608, "upload_time": "2018-01-22T21:20:49", "url": "https://files.pythonhosted.org/packages/cf/10/ff69265cf6c8f601ef84f865059c1558d520487cff1eb201541b5665f2a3/tf_run_manager-1.1.6-py2.py3-none-any.whl" } ], "2.1.6": [ { "comment_text": "", "digests": { "md5": "c959c4738bb9d8e895e8a7f5effbb8b5", "sha256": "8b786a64baab5c3ec87789c5fc93f4f9b1f25d9c1f4b0571511744922bccb965" }, "downloads": -1, "filename": "tf_run_manager-2.1.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "c959c4738bb9d8e895e8a7f5effbb8b5", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 21019, "upload_time": "2018-01-26T18:31:36", "url": "https://files.pythonhosted.org/packages/b2/b8/c45aa3ada9e88130526066671fc11a8524c23f3d8a6b8ee0b9cbc23ec219/tf_run_manager-2.1.6-py2.py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "c959c4738bb9d8e895e8a7f5effbb8b5", "sha256": "8b786a64baab5c3ec87789c5fc93f4f9b1f25d9c1f4b0571511744922bccb965" }, "downloads": -1, "filename": "tf_run_manager-2.1.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "c959c4738bb9d8e895e8a7f5effbb8b5", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 21019, "upload_time": "2018-01-26T18:31:36", "url": "https://files.pythonhosted.org/packages/b2/b8/c45aa3ada9e88130526066671fc11a8524c23f3d8a6b8ee0b9cbc23ec219/tf_run_manager-2.1.6-py2.py3-none-any.whl" } ] }