{ "info": { "author": "Mo Zhou", "author_email": "cdluminate@gmail.com", "bugtrack_url": null, "classifiers": [], "description": "TQ -- Simple Command Line Job Manager\n===\n\n[![Latest Version](https://pypip.in/version/tq1/badge.svg)](https://pypi.python.org/pypi/tq1/)\n\n![tqls1](tqls1.png)\n\n#### Description in English:\n\nTQ (Task Queue) is a simple Command Line Job Manager. In principle TQ is\na very flexible and smart atd(8), which could arrange a series of jobs in\nan efficient way.\n\n(1) By default TQ will run the jobs one by one in the FIFO order.\n\n(2) A job with high priority will be processed earlier.\n\n(3) Given the estimated occupancy coefficient, jobs can be executed in\n parallel as long as possible.\n\nThe management of job queue is based on SQLite3 database, in which\ninformation about every job, including the start and end time, is stored.\n\n#### \u4e2d\u6587\u63cf\u8ff0(Description in Chinese):\n\n\u5728\u4e00\u4f4d\u5927\u4f6c\u7684\u6002\u607f\u4e0b\uff0c\u6211\u628a\u81ea\u5df1\u70bc\u4e39\u7528\u7684\u961f\u5217\u7ba1\u7406\u5668tq\u4f20\u5230\u4e86pypi\u4e0a\u3002\ntq\u672c\u8d28\u4e0a\u5c31\u662f\u4e00\u4e2a\u5f02\u5e38\u7075\u6d3b\u7684atd\uff0c\u53ef\u4ee5\u7528\u6765\u5b89\u6392\u6267\u884c\u4e00\u7cfb\u5217\u8017\u65f6\u7684\u547d\u4ee4\u3002\n\u9ed8\u8ba4\u60c5\u51b5\u4e0btq\u4f1a\u4e32\u884c\uff08FIFO\uff09\u6267\u884c\u7ed9\u5b9a\u7684\u4efb\u52a1\u961f\u5217\u3002\u5982\u679c\u6307\u5b9a\u4e86\u4f18\u5148\u7ea7\n\u6216\u8005\u9884\u8ba1\u8d44\u6e90\u5360\u7528\u6bd4\uff0ctq\u8fd8\u80fd\u6839\u636e\u60c5\u51b5\u4f18\u5148\u6267\u884c\u9ad8\u4f18\u5148\u4efb\u52a1\uff0c\u6216\u8005\u5e76\u884c\n\u6267\u884c\u82e5\u5e72\u975e\u72ec\u5360\u7684\u4efb\u52a1\u3002\u961f\u5217\u7684\u7ba1\u7406\u4f9d\u8d56\u4e8eSQLite3\u6570\u636e\u5e93\uff0c\u6bcf\u4e00\u4e2a\u4efb\u52a1\n\u7684\u5404\u7c7b\u4fe1\u606f\uff0c\u5305\u62ec\u8d77\u6b62\u65f6\u95f4\uff0c\u90fd\u4f1a\u88ab\u8bb0\u5f55\u5728\u5176\u4e2d\u3002\n\n## Features\n\n1. TQ keeps your command in a queue and execute them one by one.\n\n2. Priority attribute is supported. The scheduler will execute high priority command lines first.\n\n3. Resource occupancy coefficient is supported. The scheduler will execute command lines in parallel as long as possible.\n\n4. The program output will be redirected to `tq.out` and `tq.err` under the directory where the task was added.\n\n5. The queue is saved in SQLite3. Won't lose any data even if a powerloss had happend.\n\n6. Dependency free and light weight. This tool only depends on python3 itself. (with SQLite3 support)\n\n## Install\n\nThis tools is available on Pypi. Just issue the following command:\n```\npip3 install tq1\n```\nNote that `python3 >= 3.6` is required due to new language features used\nin the code. There is no plan to support lower version of python.\n\n## Real-Life Example\n\nTQ can be used to deal with some commands in an async manner. e.g.\n```\n$ tq -- git push # Doesn't block. Have it done in async.\n$ vim mycode.py\n```\n\nTQ can be used to manage a series of computation experiments, such as\ndeep learning experiments, e.g.\n```\n$ tq r5 -- caffe train -solver net1forfun.prototxt\n$ tq r5 -- caffe train -solver net2forfun.prototxt\n$ tq 1 5 -- python3 train.py --lr 1e-2\n$ tq 1 5 -- python3 train.py --lr 1e-3\n$ tq 1 5 -- python3 train.py --lr 1e-4\n$ tq 1 5 -- python3 train.py --lr 1e-5\n$ tq p10 -- python3 important_train.py\n```\nOne can just put many computation tasks in the queue, and TQ will smartly\nschedule these experiments according to the given priority and resource\noccupancy parameters.\n\n## Usage\n\n```\nUsage: tq ACTION [COMMAND_ARGS]\n tq [P R] -- TASK\n\nAvailable Actions:\n start start TQ's daemon\n stop stop TQ's daemon\n log dump log to screen\n ls fancy print of task queue\n db print database content to screen\n rm remove task with specified id, see ID with tq ls\n clean remove finished tasks from queue\n purge remove log file and sqlite3 db file\n\nApending Task:\n -- TASK append TASK to the queue\n p

-- TASK append TASK with priority P to the queue\n r -- TASK append TASK with resource occupancy R to the queue\n P R -- TASK append TASK with priority P and estimated occupancy R\n int P default 0 range [INT_MIN, INT_MAX], large=important\n int R detault 10 range [1, 10], large=consuming\n```\n\n## Usage Examples\n\n```\n0. Daemon: to start or stop the daemon\n tq start\n tq stop\n1. Serial: the two given tasks should be executed one by one\n tq -- sleep 100\n tq -- sleep 100\n2. Parallel: each task occupies 40% of resource.\n In this example two tasks will be active at the same time.\n tq r4 -- sleep 100\n tq r4 -- sleep 100\n tq r4 -- sleep 100\n3. Priority: break the FIFO order of tasks. 1 > default Priority.\n tq p1 -- sleep 100\n4. Special Case: run the given task right away ignoring Pri and Rsc\n tq 1 0 -- sleep 100\n```\n\n## See Also\n\natd(8), HTCondor, Torque, Slurm, PBS.\n\nhttps://wiki.debian.org/HighPerformanceComputing\n\n## License\n\nMIT License.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/CDLuminate/tq", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "tq1", "package_url": "https://pypi.org/project/tq1/", "platform": "", "project_url": "https://pypi.org/project/tq1/", "project_urls": { "Homepage": "https://github.com/CDLuminate/tq" }, "release_url": "https://pypi.org/project/tq1/0.4.6/", "requires_dist": null, "requires_python": "", "summary": "Simple Command Line Job Manager", "version": "0.4.6" }, "last_serial": 5285017, "releases": { "0.2.1": [ { "comment_text": "", "digests": { "md5": "3bda6a9141ab4426fb918adb2a9ad5eb", "sha256": "00817071baad30af7ca458d05e665370da7b23248468e07c831c4d035827fa13" }, "downloads": -1, "filename": "tq1-0.2.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "3bda6a9141ab4426fb918adb2a9ad5eb", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8403, "upload_time": "2018-04-12T16:52:50", "url": "https://files.pythonhosted.org/packages/11/f5/69615067658fa6d8dc9fffe075e16c7cea10c2696481ae1f797699f052b5/tq1-0.2.1-py2.py3-none-any.whl" } ], "0.3": [ { "comment_text": "", "digests": { "md5": "5236eb49ae4cef2333554039c4520c97", "sha256": "25e4b5f93d547cd18d76332b0cbf04d5e63c7e096188d796ca8197239e101697" }, "downloads": -1, "filename": "tq1-0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "5236eb49ae4cef2333554039c4520c97", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8728, "upload_time": "2018-04-14T13:32:16", "url": "https://files.pythonhosted.org/packages/cd/26/c56b44a339ebb7c8e06944a6aa9d88c9f3e4e3570107a2f51bd4ca8fd194/tq1-0.3-py2.py3-none-any.whl" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "41d1625bf44ab3b6060305dfcf1a715f", "sha256": "f23027bdf0a80dd6f6ce91a68629dbbcaa43de9dcbdd35f5c231788de669ac7b" }, "downloads": -1, "filename": "tq1-0.3.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "41d1625bf44ab3b6060305dfcf1a715f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10829, "upload_time": "2018-04-15T16:11:11", "url": "https://files.pythonhosted.org/packages/c6/ea/e1a9ea7a3bae02acc1c429f75fa598b6fc1c5c7355311255509b08eae944/tq1-0.3.1-py2.py3-none-any.whl" } ], "0.3.2": [ { "comment_text": "", "digests": { "md5": "6858fc778855b51a81eb13391265aed8", "sha256": "3cda3bfebbd40fe11938e7b17914f11505b560aac5e184cdb34661e3f2cd8633" }, "downloads": -1, "filename": "tq1-0.3.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "6858fc778855b51a81eb13391265aed8", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10990, "upload_time": "2018-04-16T02:13:00", "url": "https://files.pythonhosted.org/packages/eb/50/78e78239750bd3dd394a0a2dada7d007f660b72c22e8506927267d8a2b1a/tq1-0.3.2-py2.py3-none-any.whl" } ], "0.3.3": [ { "comment_text": "", "digests": { "md5": "e327c0b84429a05041878480736cf419", "sha256": "31e36b3a16fc2c1a2136b47e2941efc41deec4daa68df5a6ad5c8621bd189d9d" }, "downloads": -1, "filename": "tq1-0.3.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e327c0b84429a05041878480736cf419", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 11358, "upload_time": "2018-04-17T03:28:03", "url": "https://files.pythonhosted.org/packages/a7/eb/d5cbdaf8e8b19f1a71089dd5be0bd3983edb9de65e61852ac3ff0a197498/tq1-0.3.3-py2.py3-none-any.whl" } ], "0.4": [ { "comment_text": "", "digests": { "md5": "3336df45a413b40b8772d2cc26fa9a1c", "sha256": "e02061b088272dff9dbd1ee836558cd73fae35e95315cd6b0f1c2985b0885ff1" }, "downloads": -1, "filename": "tq1-0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "3336df45a413b40b8772d2cc26fa9a1c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10150, "upload_time": "2018-04-23T06:43:13", "url": "https://files.pythonhosted.org/packages/1c/7c/0db5dbb7864b0e37a1dde0f085c296501dc43b18a8d4cb294e0043e83e60/tq1-0.4-py2.py3-none-any.whl" } ], "0.4.1": [ { "comment_text": "", "digests": { "md5": "fd88d84ee16c30e8f0b98eb86147fec8", "sha256": "e7d888f5b6a1e3715a1fd318660599f01c4bbf4fd900dd113ba059912b0674d2" }, "downloads": -1, "filename": "tq1-0.4.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "fd88d84ee16c30e8f0b98eb86147fec8", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10402, "upload_time": "2018-04-24T09:35:57", "url": "https://files.pythonhosted.org/packages/9c/80/79bad7c45cda89551acd2d515a4ae2613b6530edcede8f1bac8fef2f32f1/tq1-0.4.1-py2.py3-none-any.whl" } ], "0.4.2": [ { "comment_text": "", "digests": { "md5": "efb6ac2004183c50d28b2efcb4f30915", "sha256": "d2f89fcfc0a83c5e1157dd8ff72e3095d9014a0f4137de7202ec621e6fd2568f" }, "downloads": -1, "filename": "tq1-0.4.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "efb6ac2004183c50d28b2efcb4f30915", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 11091, "upload_time": "2018-04-25T08:26:08", "url": "https://files.pythonhosted.org/packages/96/c2/985fb5bb3021e28451ba133d85785b37ba81373be2f4382be852ff8ec69f/tq1-0.4.2-py2.py3-none-any.whl" } ], "0.4.3": [ { "comment_text": "", "digests": { "md5": "8bc9f09cf58de575839897912d0ea6d0", "sha256": "fca476076da665a2ac0bafa6539c3e8ebfe8556bb53c1728dc2da0e60ed5b8df" }, "downloads": -1, "filename": "tq1-0.4.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "8bc9f09cf58de575839897912d0ea6d0", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 11148, "upload_time": "2018-04-26T17:17:34", "url": "https://files.pythonhosted.org/packages/f7/ab/6ee492c9c9ddcfcdeb44d1ffac3eabfa152b8ebfec3d36ea15a9eebd8082/tq1-0.4.3-py2.py3-none-any.whl" } ], "0.4.4": [ { "comment_text": "", "digests": { "md5": "ff2b438e7abcf1e1dd13c8606dbbe532", "sha256": "e356df1e174ab9c8969ebab697388088e1868c73727b98f3ae3449ecc5108ea2" }, "downloads": -1, "filename": "tq1-0.4.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "ff2b438e7abcf1e1dd13c8606dbbe532", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 11235, "upload_time": "2018-04-27T12:10:50", "url": "https://files.pythonhosted.org/packages/0f/56/508702c7c42c9585a43662cb161eeac40b3ab3936ce1fc573e54a69e141a/tq1-0.4.4-py2.py3-none-any.whl" } ], "0.4.5": [ { "comment_text": "", "digests": { "md5": "eff276faeead6b47688eec7a116d564e", "sha256": "41d6706c48af5d612cdf55e6966e1189920b4061e3769b6925253df27a13079f" }, "downloads": -1, "filename": "tq1-0.4.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "eff276faeead6b47688eec7a116d564e", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 11303, "upload_time": "2018-06-04T04:30:55", "url": "https://files.pythonhosted.org/packages/a9/e0/77ab2bf430e00b22e0d4d6b572cb98ebc3b69401259a2e64f3647a9d1249/tq1-0.4.5-py2.py3-none-any.whl" } ], "0.4.6": [ { "comment_text": "", "digests": { "md5": "2ec9bcf275f92aa4ca53f3eb0688b9c7", "sha256": "f731612721f1db73b7ff28f1ded5c8cfa8ed255950c4a5344989eb569282d20b" }, "downloads": -1, "filename": "tq1-0.4.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "2ec9bcf275f92aa4ca53f3eb0688b9c7", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 12204, "upload_time": "2019-05-18T06:28:16", "url": "https://files.pythonhosted.org/packages/d6/85/79bffff8f5751eb3d96ae7ad0a6a6e448051794a3e5bda82492ede8b4429/tq1-0.4.6-py2.py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "2ec9bcf275f92aa4ca53f3eb0688b9c7", "sha256": "f731612721f1db73b7ff28f1ded5c8cfa8ed255950c4a5344989eb569282d20b" }, "downloads": -1, "filename": "tq1-0.4.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "2ec9bcf275f92aa4ca53f3eb0688b9c7", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 12204, "upload_time": "2019-05-18T06:28:16", "url": "https://files.pythonhosted.org/packages/d6/85/79bffff8f5751eb3d96ae7ad0a6a6e448051794a3e5bda82492ede8b4429/tq1-0.4.6-py2.py3-none-any.whl" } ] }