{ "info": { "author": "jizong", "author_email": "jizong.peng.1@etsmtl.net", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.7" ], "description": "# `GPUQueue` A very simple GPU tool - To run multiple jobs with assigned (limited) GPU resources\n\nIt provides very simple and basic function of dynamically utilize given GPUs with a large job array. It can be used to automatically identify the GPU that has been released by a newly-ended program.\n\n### Examples\n\n---\n`python` interface\n``` python\nfrom gpu_queue import JobSubmitter\n\njob_array = [\n \"python -c 'import os, time;print(\\\"GPU num utilized\\\",os.environ[\\\"CUDA_VISIBLE_DEVICES\\\"]);time.sleep(3)'\",\n \"python -c 'import os, time;print(\\\"GPU num utilized\\\",os.environ[\\\"CUDA_VISIBLE_DEVICES\\\"]);time.sleep(2)'\",\n \"python -c 'import os, time;print(\\\"GPU num utilized\\\",os.environ[\\\"CUDA_VISIBLE_DEVICES\\\"]);time.sleep(0.5)'\",\n \"python -c 'import os, time;print(\\\"GPU num utilized\\\",os.environ[\\\"CUDA_VISIBLE_DEVICES\\\"]);time.sleep(0.5)'\",\n \"python -c 'import os, time;print(\\\"GPU num utilized\\\",os.environ[\\\"CUDA_VISIBLE_DEVICES\\\"]);time.sleep(3)'\",\n \"python -c 'import os, time;print(\\\"GPU num utilized\\\",os.environ[\\\"CUDA_VISIBLE_DEVICES\\\"]);time.sleep(1)'\",\n]\n\nJ = JobSubmitter(job_array, [0, 1, 2])\nJ.submit_jobs()\n```\nOutput:\n```\n6 jobs has been saved\nGPU num utilized 0\nGPU num utilized 2\nGPU num utilized 1\nGPU num utilized 2\nGPU num utilized 2\nGPU num utilized 1\nall jobs has been run\nsucessful jobs: 6\n\nfailed jobs: 0\n```\n`gpuqueue` can be directly used in the bash \n`Bash` interface\n```bash\n#!/usr/bin/env bash\n\n# example of typical machine learning hyper-parameter tuning \n# mean teacher for semi supervised learning\n\nsave_dir=cifar10/labeled_sample_4000/augment_img\nEMA_decay=0.999\n\ndeclare -a StringArray=(\n\"python classify_main.py Trainer.name=MeanTeacherTrainer Config=config/cifar_mt_config.yaml Trainer.save_dir=${save_dir}/meanteacherbaseline RegScheduler.max_value=0 Trainer.EMA_decay=${EMA_decay} \"\n\"python classify_main.py Trainer.name=MeanTeacherTrainer Config=config/cifar_mt_config.yaml Trainer.save_dir=${save_dir}/meanteacher_0.1 RegScheduler.max_value=0.1 Trainer.EMA_decay=${EMA_decay} \"\n\"python classify_main.py Trainer.name=MeanTeacherTrainer Config=config/cifar_mt_config.yaml Trainer.save_dir=${save_dir}/meanteacher_1 RegScheduler.max_value=1 Trainer.EMA_decay=${EMA_decay} \"\n\"python classify_main.py Trainer.name=MeanTeacherTrainer Config=config/cifar_mt_config.yaml Trainer.save_dir=${save_dir}/meanteacher_10 RegScheduler.max_value=10 Trainer.EMA_decay=${EMA_decay} \"\n\"python classify_main.py Trainer.name=MeanTeacherTrainer Config=config/cifar_mt_config.yaml Trainer.save_dir=${save_dir}/meanteacher_20 RegScheduler.max_value=20 Trainer.EMA_decay=${EMA_decay} \"\n\"python classify_main.py Trainer.name=MeanTeacherTrainer Config=config/cifar_mt_config.yaml Trainer.save_dir=${save_dir}/meanteacher_50 RegScheduler.max_value=50 Trainer.EMA_decay=${EMA_decay} \"\n)\n# just using 0 and 1 gpus for those jobs\ngpuqueue \"${StringArray[@]}\" --available_gpus 0 1\n```\n\n\n---\n### install \n```bash\npip install gpuqueue\n```\n\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/jizongFox/GPUQueues", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "GPUQueue", "package_url": "https://pypi.org/project/GPUQueue/", "platform": "", "project_url": "https://pypi.org/project/GPUQueue/", "project_urls": { "Homepage": "https://github.com/jizongFox/GPUQueues" }, "release_url": "https://pypi.org/project/GPUQueue/0.0.3/", "requires_dist": null, "requires_python": "", "summary": "", "version": "0.0.3", "yanked": false, "yanked_reason": null }, "last_serial": 6056090, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "933801b773c12f1191ed1d067247a455", "sha256": "969e778c7e6502b9700d0a07ffe48ca4b5514262eecabe06fb03e42126e1cd45" }, "downloads": -1, "filename": "GPUQueue-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "933801b773c12f1191ed1d067247a455", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 3644, "upload_time": "2019-10-29T03:53:15", "upload_time_iso_8601": "2019-10-29T03:53:15.434428Z", "url": "https://files.pythonhosted.org/packages/d6/a7/665965ee812b47ba593837026fcc4bcb0116252dab9013f748f363147ad0/GPUQueue-0.0.1-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "6c24387acaaa60f8abea53d913e2eb62", "sha256": "cf86d01ada01eff43ada0248794c6dc9a0f037e5a5e848964f0741c5592a1625" }, "downloads": -1, "filename": "GPUQueue-0.0.1.tar.gz", "has_sig": false, "md5_digest": "6c24387acaaa60f8abea53d913e2eb62", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3117, "upload_time": "2019-10-29T03:53:17", "upload_time_iso_8601": "2019-10-29T03:53:17.558384Z", "url": "https://files.pythonhosted.org/packages/d1/92/a9aa690b28c3c9a4bb5d8567c22c8abc1254a0a4a5944329eb901cd3d919/GPUQueue-0.0.1.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "20bd477b07d7deb2a9f88a58e52d4ef1", "sha256": "98236ed17bbdcec5c74c0699d1dc7583d75481a61424ac2368f8a85a99c9fe42" }, "downloads": -1, "filename": "GPUQueue-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "20bd477b07d7deb2a9f88a58e52d4ef1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 4003, "upload_time": "2019-10-29T04:09:31", "upload_time_iso_8601": "2019-10-29T04:09:31.327248Z", "url": "https://files.pythonhosted.org/packages/e7/f4/c4fcbe0e8b008aab1d0652e17a6660a746b594aef3935544d86c2403b97c/GPUQueue-0.0.2-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "185d8a6c40c46273566334f37ba38d58", "sha256": "b2811f948be5cec502c749f592002dea04cda7ef49e5c0ec4ef09f25b07d6bb4" }, "downloads": -1, "filename": "GPUQueue-0.0.2.tar.gz", "has_sig": false, "md5_digest": "185d8a6c40c46273566334f37ba38d58", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3533, "upload_time": "2019-10-29T04:09:32", "upload_time_iso_8601": "2019-10-29T04:09:32.483783Z", "url": "https://files.pythonhosted.org/packages/f0/eb/04854f79c39aed707c72c8b5b6f598f846d6a70fde1f2c71d8b222fc6789/GPUQueue-0.0.2.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "d6489b4f0308f8ed5eebfd41a9ab5d94", "sha256": "2c4f63b5e2cfe08e3f7e8caceacbeedc039caf610915ae1b076c83b59d863453" }, "downloads": -1, "filename": "GPUQueue-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "d6489b4f0308f8ed5eebfd41a9ab5d94", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 4771, "upload_time": "2019-10-31T04:32:47", "upload_time_iso_8601": "2019-10-31T04:32:47.202829Z", "url": "https://files.pythonhosted.org/packages/20/fc/c0176ae76b31efea144c0d8fa1027751dc071bba85f59d2a4e80b18716e8/GPUQueue-0.0.3-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "2c9d4d37f0d8221dde85d52cceeeac42", "sha256": "6b8c7fd192d922984893ceaac5e152f8408f0015f6eea07b9188d4efe5b405b0" }, "downloads": -1, "filename": "GPUQueue-0.0.3.tar.gz", "has_sig": false, "md5_digest": "2c9d4d37f0d8221dde85d52cceeeac42", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4025, "upload_time": "2019-10-31T04:32:48", "upload_time_iso_8601": "2019-10-31T04:32:48.786783Z", "url": "https://files.pythonhosted.org/packages/24/6e/bebbb9b4d850a26829266f5976c725bf4b4b515ebcf633dfb51bae03109a/GPUQueue-0.0.3.tar.gz", "yanked": false, "yanked_reason": null } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "d6489b4f0308f8ed5eebfd41a9ab5d94", "sha256": "2c4f63b5e2cfe08e3f7e8caceacbeedc039caf610915ae1b076c83b59d863453" }, "downloads": -1, "filename": "GPUQueue-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "d6489b4f0308f8ed5eebfd41a9ab5d94", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 4771, "upload_time": "2019-10-31T04:32:47", "upload_time_iso_8601": "2019-10-31T04:32:47.202829Z", "url": "https://files.pythonhosted.org/packages/20/fc/c0176ae76b31efea144c0d8fa1027751dc071bba85f59d2a4e80b18716e8/GPUQueue-0.0.3-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "2c9d4d37f0d8221dde85d52cceeeac42", "sha256": "6b8c7fd192d922984893ceaac5e152f8408f0015f6eea07b9188d4efe5b405b0" }, "downloads": -1, "filename": "GPUQueue-0.0.3.tar.gz", "has_sig": false, "md5_digest": "2c9d4d37f0d8221dde85d52cceeeac42", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4025, "upload_time": "2019-10-31T04:32:48", "upload_time_iso_8601": "2019-10-31T04:32:48.786783Z", "url": "https://files.pythonhosted.org/packages/24/6e/bebbb9b4d850a26829266f5976c725bf4b4b515ebcf633dfb51bae03109a/GPUQueue-0.0.3.tar.gz", "yanked": false, "yanked_reason": null } ], "vulnerabilities": [] }