{
    "info": {
        "author": "Ryan Fitzpatrick",
        "author_email": "rmfitzpatrick@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Framework :: Pytest",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: Implementation :: CPython",
            "Topic :: Software Development :: Testing"
        ],
        "description": "# pytest-mp: Multiprocess and Segregate Tests\n\n[![Build Status](https://travis-ci.org/ansible/pytest-mp.svg?branch=master)](https://travis-ci.org/ansible/pytest-mp)\n\n**pytest-mp** is a minimalist approach to distributing and segregating pytest tests across processes using python's [multiprocessing](https://docs.python.org/2/library/multiprocessing.html) library and is heavily inspired by [pytest-concurrent](https://github.com/reverbc/pytest-concurrent) and [pytest-xdist](https://github.com/pytest-dev/pytest-xdist).  As a very early beta, it doesn't pledge or intend to support the majority of platforms or use cases.  Design is based on supporting slow, io-bound testing with often tedious system under test configuration that can benefit from running several tests at one time.\n\n### Installation and Basic Usage\n```bash\ngit clone https://github.com/ansible/pytest-mp\npip install pytest-mp/\ncd my_test_dir\n# Most basic invocation that will spin up to as many test runner processes as multiprocessing.cpu_count() indicates.\npytest --mp\n# Create up to 4 concurrent child processes.\npytest --mp --np 4\npytest --multiprocessing --num-processes 4  # Same as above but with more informative option names.\n```\n\n\n### Test Running and Segregation Strategies\npytest-mp provides four test segregation strategies that come in handy for common test and fixture patterns.  Each strategy has its own performance (dis)advantages and caveats in terms of fixture scoping and invocations.\n\n```python\nimport pytest\n\n@pytest.mark.mp_group('SomeGroupName', 'free')  # free, serial, isolated_free, or isolated_serial\nclass TestSomething(object):\n\n    def test_one(self, fixture_one):\n        assert True\n\n    def test_two(self, fixture_two):\n        assert True\n\n\n@pytest.mark.mp_group('SomeAdditionalGroupName')  # uses free strategy by default\ndef test_three():\n    assert True\n\n\n@pytest.mark.mp_group(group='SomeOtherGroupName', strategy='isolated_serial')\ndef test_four(fixture_three):\n    assert True\n\n\n@pytest.mark.mp_group('SomeOtherGroupName')  # still uses previously-defined strategy isolated_serial\ndef test_five(fixture_three):\n    assert True\n```\n\n1. The **`free`** strategy distributes each test in a group to a fresh pytest session in a child process that invokes all sourced fixtures (regardless of scope), runs that single test, and calls all registered finalizers before joining.  This is the default test strategy for grouped and ungrouped tests.\n1. The **`serial`** strategy distributes each group of tests to a fresh pytest session in a child process that invokes all sourced fixtures (regardless of scope) and runs each test serially in the same process before tearing down.  This group is best suited for tests that require shared, highly-scoped fixtures that won't affect the state of the system under test for other tests.\n1. The **`isolated_free`** strategy is the same as `free`, but all tests in this group will be run separately in time from any other test group.  Best suited for tests with noisy or destructive fixtures that would affect the requirements of other tests, but that don't require a shared process.\n1. The **`isolated_serial`** strategy is the same as `serial`, but all tests in this group will be run separate in time from any other test group, essentially like a regular pytest invocation.  Best suited for tests with shared, noisy, or destructive fixtures.  Absolute pytest execution will be limited to a single process while these tests are running.\n\nFor example, of the tests defined above, `TestSomething.test_one`, `TestSomething.test_two`, and `test_three` could potentially be run at the same time among 3 processes, but `test_four` and `test_five` are guaranteed to run in the same process and with no other tests running in the background.\n\n### Synchronization\nGiven that tests generally run in child processes that emulate a fresh pytest session and that by nature pytest fixtures of class or greater scope are designed to be shared and invoked once by the test runner, some synchronization between test processes is needed to provide idempotency.  pytest-mp provides two session-scoped synchronization fixtures: `mp_message_board` and `mp_lock`, a `multiprocesssing.Manager.dict()` and `multiprocessing.Manager.Lock()` instance, respectively.\n\n```python\nimport pytest\n\n@pytest.fixture(scope='session')\ndef must_be_idempotent_without_teardown(mp_lock, mp_message_board, some_resource):\n    with mp_lock:\n        if mp_message_board.get('some_flag_your_fixture_creates'):\n            return\n        mp_message_board['some_flag_your_fixture_creates'] = True\n        some_resource.destructive_call()\n    return\n\n@pytest.fixture(scope='session')\ndef must_be_idempotent_with_teardown(mp_lock, mp_message_board, some_resource):\n    with mp_lock:\n        if mp_message_board.get('must_be_idempotent_setup'):\n            yielded = mp_message_board['must_be_idempotent_val']\n        else:\n            mp_message_board['must_be_idempotent_setup'] = True\n            yielded = some_resource.destructive_call()\n            mp_message_board['must_be_idempotent_val'] = yielded  # pickle!!!\n    yield yielded\n    with mp_lock:\n        if mp_message_board.get('must_be_idempotent_teardown'):\n            return\n        mp_message_board['must_be_idempotent_teardown'] = True\n        # Notice there is no synchronization w/ this teardown and active fixture consumers!\n        some_resource.cleanup()\n```\n\nA helper fixture `mp_trail()` that internally uses `mp_lock` and `mp_message_board` is provided to assist in the assurance that a single setup and teardown invocation of shared fixtures and test logic occurs with multiple test runners.  A __trail__ is any named, shared path with single 'start' and 'finish' events made available as context manager values.\n\n```python\n@pytest.fixture(scope='session')\ndef must_be_idempotent_with_teardown(mp_trail, some_resource):\n    with mp_trail('TrailName', 'start') as start:  #  mp_trail('TrailName') defaults to 'start'\n        if start:  # First invocation of `must_be_idempotent_with_teardown` detected.\n            yielded = some_resource.destructive_call()\n        else:  # setup has already occured by another test runner\n            yielded = some_resource.get_current_state()\n    yield yielded\n    with mp_trail('TrailName', 'finish') as finish:\n        if finish:  # Last finalizer/teardown detected.\n            # There are no other tests using this fixture at the moment\n            some_resource.cleanup()\n```\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/ansible/pytest-mp",
        "keywords": "",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "pytest-mp",
        "package_url": "https://pypi.org/project/pytest-mp/",
        "platform": "",
        "project_url": "https://pypi.org/project/pytest-mp/",
        "project_urls": {
            "Homepage": "https://github.com/ansible/pytest-mp"
        },
        "release_url": "https://pypi.org/project/pytest-mp/0.0.4/",
        "requires_dist": [
            "pytest",
            "psutil"
        ],
        "requires_python": "",
        "summary": "A test batcher for multiprocessed Pytest runs",
        "version": "0.0.4"
    },
    "last_serial": 3892383,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "25248d511338a17ca6d6b70eaaa8bc3c",
                    "sha256": "16ae063e772dfe7f1b776e21dfa1d58de266c33def196540c25d86f78fcfcdb4"
                },
                "downloads": -1,
                "filename": "pytest-mp-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "25248d511338a17ca6d6b70eaaa8bc3c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 13047,
                "upload_time": "2018-01-19T00:58:02",
                "url": "https://files.pythonhosted.org/packages/80/46/2f5ac0f11795b5ba2ef2549feb2f94945ac3f05d1146d9e7c1e444ea3077/pytest-mp-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a0656edc9416043570fe55ad56ade2be",
                    "sha256": "9fdf4a4471780dd54b797c4b90bac910dea663344da8baea51ab342aff922745"
                },
                "downloads": -1,
                "filename": "pytest-mp-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "a0656edc9416043570fe55ad56ade2be",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12769,
                "upload_time": "2018-03-14T20:48:19",
                "url": "https://files.pythonhosted.org/packages/1e/44/06593b43036a11f322f901608cd79941f498cde6296bb351d8bcba0da546/pytest-mp-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7ac9c505146078226a8d1f457f6bdcd5",
                    "sha256": "caf7d18c70e819f5d1346e79b8974c10f91f3a680d28f8bb7b491fb64a8e7faf"
                },
                "downloads": -1,
                "filename": "pytest-mp-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "7ac9c505146078226a8d1f457f6bdcd5",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12827,
                "upload_time": "2018-03-19T18:32:57",
                "url": "https://files.pythonhosted.org/packages/9e/64/8af70cd3ccea5545487fab757d7b872182d59df6c3de21fadf17f1d37c63/pytest-mp-0.0.3.tar.gz"
            }
        ],
        "0.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ed5e1010305a3d30a8a26a6e5ceb10b7",
                    "sha256": "02e4118c4d9bf0d8c43c9115289aa95ea4e1f7a74285d1a4b05f93e610fa3c73"
                },
                "downloads": -1,
                "filename": "pytest_mp-0.0.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "ed5e1010305a3d30a8a26a6e5ceb10b7",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 10416,
                "upload_time": "2018-05-23T18:50:08",
                "url": "https://files.pythonhosted.org/packages/51/d4/bdee66f94a3816415d7842e435ca3fcff813be18eb904e110e1c147fd4b6/pytest_mp-0.0.4-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "21254640c871f67d940110956699c0c3",
                    "sha256": "4b30a9cc514c9ec74d1b0b61ef5cc6d5d01477667d566e366e5204bc6d18bd3b"
                },
                "downloads": -1,
                "filename": "pytest-mp-0.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "21254640c871f67d940110956699c0c3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10475,
                "upload_time": "2018-05-23T18:50:09",
                "url": "https://files.pythonhosted.org/packages/61/ba/d570e2c2ac6156e4fa004438a85463dafabdd04f2f3c00b2efc2d8d84116/pytest-mp-0.0.4.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "ed5e1010305a3d30a8a26a6e5ceb10b7",
                "sha256": "02e4118c4d9bf0d8c43c9115289aa95ea4e1f7a74285d1a4b05f93e610fa3c73"
            },
            "downloads": -1,
            "filename": "pytest_mp-0.0.4-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "ed5e1010305a3d30a8a26a6e5ceb10b7",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 10416,
            "upload_time": "2018-05-23T18:50:08",
            "url": "https://files.pythonhosted.org/packages/51/d4/bdee66f94a3816415d7842e435ca3fcff813be18eb904e110e1c147fd4b6/pytest_mp-0.0.4-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "21254640c871f67d940110956699c0c3",
                "sha256": "4b30a9cc514c9ec74d1b0b61ef5cc6d5d01477667d566e366e5204bc6d18bd3b"
            },
            "downloads": -1,
            "filename": "pytest-mp-0.0.4.tar.gz",
            "has_sig": false,
            "md5_digest": "21254640c871f67d940110956699c0c3",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 10475,
            "upload_time": "2018-05-23T18:50:09",
            "url": "https://files.pythonhosted.org/packages/61/ba/d570e2c2ac6156e4fa004438a85463dafabdd04f2f3c00b2efc2d8d84116/pytest-mp-0.0.4.tar.gz"
        }
    ]
}