{ "info": { "author": "Wes Winham", "author_email": "winhamwr@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: OS Independent", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.3", "Topic :: Software Development" ], "description": "# distributed-nose Distribute your tests across multiple nodes with no hassle\n\n## TLDR\n\n### Before\n\nMachine 1:\n\n\t$ nosetests long_test_suite\n\t...\n\tRan 1312 tests in 401.109s\n\nMachine 2:\n\n\t$ echo \"I'm bored :(\"\n\tI'm bored :(\n\t$ uptime | awk -F'load average:' '{ print \"Load: \" $2 }'\n\tLoad: 0.00, 0.00, 0.00\n\nDeveloper:\n\n\t$ google-chrome http://news.ycombinator.com\n\n### After\n\nMachine 1:\n\n\t$ export NOSE_NODES=2;\n\t$ export NOSE_NODE_NUMBER=1;\n\t$ nosetests long_test_suite\n\t...\n\tRan 660 tests in 220.502s\n\nMachine 2:\n\n\t$ echo \"I feel loved\"\n\tI feel loved\n\t$ export NOSE_NODES=2;\n\t$ export NOSE_NODE_NUMBER=2;\n\t$ nosetests long_test_suite\n\t...\n\tRan 652 tests in 214.007s\n\n## The Saga of Your Test Suite\n\n### Oppression of Untested Code\n\nThe story of Your Test Suite has its heroes and its villains,\nits triumphs and its missteps.\nLike many other stories,\nit begins with a solitary hero.\nGraduating from university,\nshe joins on with a merry band of software engineering adventurers.\nLed by management and supported by a wide cast of characters,\nthey built things that the common folk loved and needed.\n\nSlowly slowly, things begin to change.\nPoorly-structured and strongly-coupled code sneaks in to the code base,\nmaking changes increasingly difficult\nand allowing devious bugs to emerge.\nParty members begin to regularly break disparate parts of the project,\nthe realm being too large to survey quickly for one person.\nAreas of code become off-limits,\nas dragons and unknown things claim all who attempt entry.\n\nThrough all this, our hero watches with a heavy heart.\nWhat is to be done about this untested code-base?\nSurely she can't change everything by herself?\nShe is just one person!\n\nFinally, the daily struggles wear her down.\n\n\"Enough is enough!\"\n\nShe resolves, pontificating on how to best strike\nagainst the oppression of untested code.\n\n\"I'll write tests, and commit them to source control!\"\n\nShe says,\nunderstanding that working code is the best possible argument.\n\n### A Nose-fueled Peace\n\nMethodically wielding and teaching [Nose](https://github.com/nose-devs/nose),\nshe recruits from within to form a merry band of like-minded developers.\nThey are the fellowship of TDD\nand they are determined to succeed.\nTests are committed in flurries of high-value code!\n\nTales of the fellowship's feats stir excitement in other developers.\nAnother hero decides that if running some tests is good,\nthen running all the tests all the time is better!\nBrandishing his hat of System Administration,\nhe utilizes extra server resources and\na Continuous Integration process is born.\n\nOur heroes grow over-confident in their success, however.\nMaking little attempt to justify the practice,\nword of time spent writing tests gets to management.\nManagerial Oversight moves to squash our fledgling fellowship.\n\n### Managerial Push-back\n\n\"If you have time to write tests,\nI must have not given you enough to do!\"\n\nThey snarl,\nthreatening with the most dangerous of weapons:\nThe sling of passive-aggressive disagreement.\n\nOnce again, our heroes summon their courage,\nand through great effort,\nboldly vanquish Managerial Opposition with the sword of high quality.\nA new era of peace and productivity begins\nand The Project flourishes.\n\nBut that peace was not to last.\nThe specter of Slow Test Runs looms over our heroes.\nSoon, test failures go unnoticed.\nDevelopers aren't waiting for tests to finish!\nThe Test Suite falls in to decay.\n\n### Multiprocess Arrives\n\nDevelopers grumble and management freely lobs I-Told-You-Sos.\nThe darkness of the age before tests threatens to return.\n\n\"I'll speed up our tests!\"\n\nA voice cries out.\nIt's our hero again,\nvowing to strike at the root cause of decay.\n\nPouring over ancient texts,\nshe quickly discovers\n[Multiprocess](http://nose.readthedocs.org/en/latest/plugins/multiprocess.html).\n\n\"This will release us from our single-core shackles.\"\n\nShe proclaims, confident.\nHours later,\nher 8-core machine races through tests with alacrity.\nSoon, multiprocess is in use on the CI server,\nand test speed races forward.\n\nAll is right in test land!\nHer fellow developers throw a break-room party\nin her honor.\nPython Meetup talks flow like water\nand TDD is once again well-liked by all.\n\nLike the previous peace,\nthis too is temporary,\nas adding more cores is costly.\nFast forward several score tests,\nand the once-speedy test suite\nregularly brings the beefy, multi-core CI server to its knees.\n\n### Distributed Despair\n\nRemembering the ease of implementing Multiprocess,\nour hero knows the drill.\nShe'll simply distribute the tests across multiple servers!\nThe same ancient tomes are confidently consulted,\nas the solution seemingly obvious.\n\nBut wait!\n\nWhere are the simple guides?\n\nOur hero isn't a wizard.\nShe can't decipher the ancient scrolls of distributed systems.\nShe doesn't have time for formal wizard training,\nand the number of moving parts involved is boggling.\n\nDespair sets in as the once-speedy test sweet becomes slower and slower.\nThe outlook for our adventurers is bleak,\nas an evaluation of speeding up the tests themselves reveals immense challenge.\nSome even suggest abandoning the test suite entirely\nwith a new focus on speed!\n\nFactions form and the party threatens to splinter entirely!\n\n### Distributed-Nose: The Scalable Solution\n\n\"Wait a minute.\"\n\nA clear voice breaks through.\nIt's our hero, once again.\n\n\"Why do our test runners need to communicate to coordinate?\nOur memcached servers need the same level of coordination\nand achieve it with ease!\"\n\nTickled by the flickering of an idea,\nshe sets to work.\nShe is steadfast, pouring over tomes on\n[Consistent Hashing](http://en.wikipedia.org/wiki/Consistent_hashing).\nOnce the solution is clear in her mind,\nshe happens upon a fellow adventurer with sound advice.\n\n\"Have you tried distributed-nose?\"\n\n\"Unfortunately.\nI'm terribly allergic to pollen\nand I get that every spring.\"\n\nAfter the initial confusion is resolved,\nit becomes clear to our hero\nthat other adventurers have already crossed this path.\nHooray!\n\nShe quickly skims a strained narrative introduction,\nfinding it, frankly, quite derivative.\nSkipping to the operationalizable section of the tome,\nshe finds fast success.\nThis is the tool for them.\n\nWith minimal effort,\nThe Test Suite is distributed across 4 machines.\nTest time is slashed and\nThe Test Suite is saved.\nPondering the future, our hero realizes\nshe will never again face uncertainty about scalability.\nShe can always just add more machines!\n\nTest Suite scalability ensured,\nour heroes go on to face many other adventures and trials.\nBut never again would something come so close to erasing their very core\nas the oppression of untested code.\n\n## Why distributed-nose?\n\nScale your tests horizontally across unlimited machines with two test flags.\n\n## Installation\n\n1. Get the project source and install it\n\n $ pip install distributed-nose\n\n## Usage\n\nTo run half your tests on one machine and the other half on the other:\n\nMachine 1:\n\n\t$ nosetests --nodes 2 --node-number 1 long_test_suite\n\nMachine 2:\n\n\t$ nosetests --nodes 2 --node-number 2 long_test_suite\n\nAlternatively, you can use the environment variables:\n* `NOSE_NODES`\n* `NOSE_NODE_NUMBER`\n\n### Temporarily disabling test distribution\n\nIn the case that you're using environment variables\nto control test distribution,\nyou sometimes still might want to run a one-off test.\nInstead of fiddling with environment variables,\nyou can just use the `--distributed-disabled` flag.\n\n\t$ export NOSE_NODES=2;\n\t$ export NOSE_NODE_NUMBER=1;\n\t$ nosetests --distributed-disabled long_test_suite\n\n## Distribution algorithm\n\nTo determine which node runs which test,\ndistributed-nose relies on the [hash_ring](https://github.com/Doist/hash_ring)\nlibrary's consistent hashing implementation.\n\nBy default, tests are individually hashed to the ring.\nThis results in the most even distribution and the best speed if all tests have the same runtime.\nHowever, it duplicates class setup/teardown work.\nIf that's expensive, you may want to use `--hash-by-class` or set `NOSE_HASH_BY_CLASS`;\nthis will hash tests in the same class to the same node.\n\n## Running the test suite\n\nThe test suite requires nose, and can be run via `setup.py`:\n\n\t# python setup.py nosetests\n\n## Is it Awesome?\n\nYes. Increasingly so.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://policystat.github.com/distributed-nose", "keywords": "", "license": "BSD", "maintainer": "", "maintainer_email": "", "name": "distributed-nose", "package_url": "https://pypi.org/project/distributed-nose/", "platform": "any", "project_url": "https://pypi.org/project/distributed-nose/", "project_urls": { "Homepage": "http://policystat.github.com/distributed-nose" }, "release_url": "https://pypi.org/project/distributed-nose/1.2.0/", "requires_dist": [ "nose", "hashring" ], "requires_python": "", "summary": "Distribute your nose tests across multiple machines, hassle-free", "version": "1.2.0" }, "last_serial": 5209476, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "f3fd36aa9984150d6e7a410611385f2e", "sha256": "ca53fe7fca12c38954dc80d8c221d1e7779fd358d591d4ba1389864aa08e0b22" }, "downloads": -1, "filename": "distributed-nose-0.1.0.tar.gz", "has_sig": false, "md5_digest": "f3fd36aa9984150d6e7a410611385f2e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8384, "upload_time": "2013-02-05T23:48:29", "url": "https://files.pythonhosted.org/packages/1e/19/e80d1b11c2ce1e78c595de87e15ba4c4078c70dfcd3f77fd5169e7e05b13/distributed-nose-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "ebf427911d997e924a8e3f9f5f9bf9f0", "sha256": "af4a215d1b844220467acddbd4de5946fc9b73b67e2ef2c806859689ab057ba3" }, "downloads": -1, "filename": "distributed-nose-0.1.1.tar.gz", "has_sig": false, "md5_digest": "ebf427911d997e924a8e3f9f5f9bf9f0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8431, "upload_time": "2013-02-08T00:15:15", "url": "https://files.pythonhosted.org/packages/b9/ab/67b5ef77875fa7ea2d3238ffac7939318169b60b7ecbc8db212ebe81cf31/distributed-nose-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "46442f7934b46de2fb6dfa7574f162f0", "sha256": "0b49983c55d0db7eb5d30af79f8222d8903988cebe93d45f28824aebe3d0c943" }, "downloads": -1, "filename": "distributed-nose-0.1.2.tar.gz", "has_sig": false, "md5_digest": "46442f7934b46de2fb6dfa7574f162f0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8635, "upload_time": "2013-03-01T01:59:42", "url": "https://files.pythonhosted.org/packages/32/d6/015ea3d5600ef760997af86dda192247a6f3b0fad1690ee7e349aade1ffd/distributed-nose-0.1.2.tar.gz" } ], "1.0.0": [ { "comment_text": "", "digests": { "md5": "9aa90b38f14f6db693c33282647fc03d", "sha256": "de16c256f22b54772120d0bb97cd8ea571ab6c4aa512b31cc479689fcff12bfd" }, "downloads": -1, "filename": "distributed-nose-1.0.0.tar.gz", "has_sig": false, "md5_digest": "9aa90b38f14f6db693c33282647fc03d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8997, "upload_time": "2014-12-01T21:59:00", "url": "https://files.pythonhosted.org/packages/14/a6/ad3c249ee21769aaba19d7b2e4e30c4ec2fe073cbfd7a860d1306090c28b/distributed-nose-1.0.0.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "9aca0e874154a13573536d1253770821", "sha256": "00f3fd78fbcc9e21f5e64e19744339de04fa455e819279fbcb75b0b8513338be" }, "downloads": -1, "filename": "distributed_nose-1.1.0-py2-none-any.whl", "has_sig": false, "md5_digest": "9aca0e874154a13573536d1253770821", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 8406, "upload_time": "2019-01-16T21:20:03", "url": "https://files.pythonhosted.org/packages/49/f5/21ab628ca85baf3d64c9dca1be59900f7577f70925595bd38e051085f068/distributed_nose-1.1.0-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "180c4002f372a621ffb1137837f912fb", "sha256": "8035e8994ed26bce74332592d360c359df0e16fef618ecbfcd8ef87dfd8532c2" }, "downloads": -1, "filename": "distributed-nose-1.1.0.tar.gz", "has_sig": false, "md5_digest": "180c4002f372a621ffb1137837f912fb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9050, "upload_time": "2019-01-16T21:20:05", "url": "https://files.pythonhosted.org/packages/c6/ed/e7c78dc85806708a1819a94aa3654556b604f5d9909d36477eedceb14746/distributed-nose-1.1.0.tar.gz" } ], "1.2.0": [ { "comment_text": "", "digests": { "md5": "3054de7e407900e6dd24eca705b71bf7", "sha256": "37cbf9b969d3a8e9bef03658f83fc79b748481939311abe8d333b2b5737b4f50" }, "downloads": -1, "filename": "distributed_nose-1.2.0-py2-none-any.whl", "has_sig": false, "md5_digest": "3054de7e407900e6dd24eca705b71bf7", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 8409, "upload_time": "2019-04-30T18:08:46", "url": "https://files.pythonhosted.org/packages/43/46/f91e9f84c0d0013491dfecfa6dd114b751439dbe5c21f5ab3124b75220cc/distributed_nose-1.2.0-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "14b072b583b728ca3bf13d160a96bcc3", "sha256": "9e9f201fa9c5ea860ff8adcfbf8f1b3f63fec4597c2c8d8455b79925850a2f3b" }, "downloads": -1, "filename": "distributed-nose-1.2.0.tar.gz", "has_sig": false, "md5_digest": "14b072b583b728ca3bf13d160a96bcc3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9056, "upload_time": "2019-04-30T18:08:51", "url": "https://files.pythonhosted.org/packages/86/84/abfe42ee3001509e230fb4e5e6faf42b7a937e96e32ce76a221d4235bd2b/distributed-nose-1.2.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "3054de7e407900e6dd24eca705b71bf7", "sha256": "37cbf9b969d3a8e9bef03658f83fc79b748481939311abe8d333b2b5737b4f50" }, "downloads": -1, "filename": "distributed_nose-1.2.0-py2-none-any.whl", "has_sig": false, "md5_digest": "3054de7e407900e6dd24eca705b71bf7", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 8409, "upload_time": "2019-04-30T18:08:46", "url": "https://files.pythonhosted.org/packages/43/46/f91e9f84c0d0013491dfecfa6dd114b751439dbe5c21f5ab3124b75220cc/distributed_nose-1.2.0-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "14b072b583b728ca3bf13d160a96bcc3", "sha256": "9e9f201fa9c5ea860ff8adcfbf8f1b3f63fec4597c2c8d8455b79925850a2f3b" }, "downloads": -1, "filename": "distributed-nose-1.2.0.tar.gz", "has_sig": false, "md5_digest": "14b072b583b728ca3bf13d160a96bcc3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9056, "upload_time": "2019-04-30T18:08:51", "url": "https://files.pythonhosted.org/packages/86/84/abfe42ee3001509e230fb4e5e6faf42b7a937e96e32ce76a221d4235bd2b/distributed-nose-1.2.0.tar.gz" } ] }