{
    "info": {
        "author": "Dan G",
        "author_email": "daniel.garcia@d2garcia.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Environment :: Console",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7",
            "Topic :: Software Development :: Libraries :: Python Modules"
        ],
        "description": "==========================================================\n``pymulproc`` a tiny multiprocessing communication library\n==========================================================\n\n.. image:: https://travis-ci.com/d2gex/pymulproc.svg?branch=master\n    :target: https://travis-ci.com/d2gex/pymulproc\n\n.. image:: https://img.shields.io/badge/pypi_package-0.1.2-brightgreen.svg\n    :target: #\n\n.. image:: https://img.shields.io/badge/coverage-98%25-brightgreen.svg\n    :target: #\n\n**pymulproc** is a tiny library to handle the communication between multiple processes without external\ndependencies other than Python's standard library. It is based purely in the multiprocessing library and provides a\ncommon interface for both PIPE and QUEUE communication.\n\npymulproc Protocol\n===================\n**pymulproc** uses a simple python list as the basis of the communication with the following fields:\n\n1. ``request``: a **required** string indicating the other peer what the operation is about.\n2. ``sender pid``: a **required** integer indicating the PID of the processing sending the datagram.\n3. ``recipient pid``: an **optional** integer indicating the PID of the process which this message is targeted to.\n4. ``data``: an **optional** python data structure containing the information intended for the other peer.\n\nIn turns the requests sent in the data structure could be anything however **pymulproc** uses the following standards:\n\n6. ``REQ_DO``: requests that indicates the other peer to do a task.\n7. ``REQ_FINISHED``: requests that indicates the other peer that the task has been done.\n8. ``REQ_DIE``: requests that indicates the other peer to stop and die as soon as possible. Practically a poison pill.\n\nExample of valid message structures are shown below. **The message must always have a length of 4**:\n\n.. code-block:: python\n\n    ['DO', 1234, 12345, {'value': 20}]  # request, sender PID, recipient PID, data\n    ['DO', 1234, 12345, None]  # request, sender PID, recipient PID\n    ['DIE', 1234, None, None]  # request, sender PID\n\npymulproc API\n===================\n**pymulproc** offers a common API interface for the conversation exchange between a parent process and its children\nfor both PIPE and QUEUE communication as follows:\n\n9.  ``send``: sends a message down the PIPE for 1:1 conversation or put a message in a JoinedQueue for 1:M, M:1 and M:M\n    conversations.\n10. ``receive``: check if there is a message in the PIPE or at the front of the QUEUE for the process making such\n    an enquiry. if the message is not intended for the process making the enquiry False is returned. In case the\n    communication is done via QUEUEs, the method puts back the message in the queue so that the targeted process can\n    later on fetch it.\n\nQueue's communication add two more specific method: ``queue_empty`` to check if the queue is empty of tasks and\n``queue_join`` to wait there until all the queue is empty.\n\nAmong the optional parameters of the  ``send`` method signature shown below, is worthwhile highlighting ``sender_pid``.\nIf provided, such integer is added to the message as sender PID. Otherwise the process will add its own\nPID.\n\n``send`` will try to place the message in the queue, for queue communication only, a few times before raising an\nexception. The amount of tries can be configured when instantiating the connection handler - see ``QueueCommunicationApi``\nclass' constructor for further details.\n\n.. code-block:: python\n\n    @abc.abstractmethod\n        def send(self, request, sender_pid=None, recipient_pid=None, data=None):\n\n``receive`` is a High Order function and may take a ``func`` keyword argument associated to a function that applies\nan operation to the message at the front of the queue. If the result is True, then the message is for the enquiring\nprocess. Otherwise it is 'reinserted' at the back of the queue for other processes to check on it.\n\nIf not parameters are passed, it is understood that the message at front of the queue is always for enquiring process.\n\nAdditionally if ``block=True`` is passed to ``receive``, the process enquiring the queue will block while the queue\nremains empty. It will then \"*wake up*\" and check the queue again when another process sends information in via\n``send``.\n\nAn example where the criteria to check if the message is for the enquiring process always fails, is shown below:\n\n.. code-block:: python\n\n    child.receive(lambda x: False)\n\n\nInstalling pymulproc\n====================\n\n**pymulproc** is available on PyPI_, so you can install it with ``pip``::\n\n    $ pip install pymulproc\n\n\npymulproc PIPE communication example\n======================================\nBelow a simple example of PIPE communication betwen a parent and a single child process is shown:\n\n.. code-block:: python\n\n    import multiprocessing\n    from pymulproc import factory, mpq_protocol\n\n    def test_simple_pipe_communication():\n\n        pipe_factory = factory.PipeCommunication()\n        parent = pipe_factory.parent()\n        child = pipe_factory.child()\n\n        def call_child(_child):\n            _child.send(mpq_protocol.REQ_TEST_CHILD)\n\n        child_process = multiprocessing.Process(name='child_process_',\n                                                target=call_child,\n                                                args=(child, ))\n        child_process.start()\n        child_process.join()\n        message = parent.receive()\n        request_offset = mpq_protocol.S_PID_OFFSET - 1\n        assert message[request_offset] == mpq_protocol.REQ_TEST_CHILD\n\npymulproc simple 1:N QUEUE communication example\n=================================================\nThe example below shows how child processes send some data back to the parent. Notice how the parent passes no ``func``\nparameter to ``receive`` as all messages placed in the queue by the child processes are intended for the parent itself:\n\n.. code-block:: python\n\n    import multiprocessing\n    from pymulproc import factory, mpq_protocol\n\n\n    class ChildProcess:\n        def __init__(self, identifier, parent_pid, conn):\n            self.id = identifier\n            self.conn = conn\n            self.parent_pid = parent_pid\n            self.pid = multiprocessing.current_process().pid\n\n        def is_message_for_me(self, message):\n            '''The message is for me if either the recipient_pid coincides with my pid or is None - None indicates\n            that the message is for everyone\n            '''\n            return message[mpq_protocol.S_PID_OFFSET + 1] == self.pid or not message[mpq_protocol.S_PID_OFFSET + 1]\n\n        def run(self, **kwargs):\n            '''Sends the data passed as keyword parameter to the parent process:\n            '''\n\n            data = kwargs.get('data', None)\n            self.conn.send(mpq_protocol.REQ_FINISHED, data=data)\n\n\n    def call_child(identifier, parent_pid, q_factory, data):\n        child = ChildProcess(identifier, parent_pid, q_factory.child())\n        child.run(data=data)\n\n\n    def test_children_to_parent_communication():\n        '''Simple test where all child processes send a message to the parent process\n\n        All children are initiated with a value that is sent to the parent for it to process it.\n        '''\n\n        queue_factory = factory.QueueCommunication()\n        parent = queue_factory.parent()\n        parent_pid = multiprocessing.current_process().pid\n\n        # Prepare list of processes to start and pass the value = 3 to each child process\n        child_processes = []\n        val = 3\n        for offset in range(5):\n            child_process = multiprocessing.Process(name=f'child_process_{offset}',\n                                                    target=call_child,\n                                                    args=(offset + 1, parent_pid, queue_factory, val))\n            child_processes.append(child_process)\n\n        # Start processes\n        for child in child_processes:\n            child.start()\n\n        # Wait for the processes to finish\n        for child in child_processes:\n            child.join()\n\n        # Receive the data from all children\n        counter = 0\n        data_offset = mpq_protocol.S_PID_OFFSET + 2\n        while not parent.queue_empty():\n            message = parent.receive()\n            counter += message[data_offset]\n\n        # Ensure the queue is empty - no loose strings\n        parent.queue_join()\n\n        # Ensure we got the right data from children\n        assert counter == val * len(child_processes)\n\n\nMore examples\n=============\n\nFor a more complex example look at the test test_parent_full_duplex_communication_with_children_stress_test_ where\na full duplex communication between the parent and child processes occurs. Also a poison pill is sent to all children\nwhen they are no longer needed.\n\n.. _test_parent_full_duplex_communication_with_children_stress_test: https://github.com/d2gex/pymulproc/blob/master/tests/test_queue_communication.py\n.. _PyPI: http://pypi.python.org/pypi/bleach\n\n\n",
        "description_content_type": "text/x-rst",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/d2gex/pymulproc",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "pymulproc",
        "package_url": "https://pypi.org/project/pymulproc/",
        "platform": "",
        "project_url": "https://pypi.org/project/pymulproc/",
        "project_urls": {
            "Homepage": "https://github.com/d2gex/pymulproc"
        },
        "release_url": "https://pypi.org/project/pymulproc/0.1.2/",
        "requires_dist": null,
        "requires_python": ">=3.6",
        "summary": "A tiny python library to handle multiprocessing communication",
        "version": "0.1.2"
    },
    "last_serial": 5678275,
    "releases": {
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b8cd6824c40252e645f17a736bb2b73b",
                    "sha256": "c3f28669fcf4e2ae27033a8609416011d48d666259ec0a673dcb066a65dd8f95"
                },
                "downloads": -1,
                "filename": "pymulproc-0.1.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b8cd6824c40252e645f17a736bb2b73b",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6",
                "size": 9362,
                "upload_time": "2019-04-16T12:43:20",
                "url": "https://files.pythonhosted.org/packages/0a/22/8db9c28488f4fd95122d7be271e54010a8db5177106503e5a2fabb91a41a/pymulproc-0.1.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "159a9df38d1e9e4dcaf124b07a5939ce",
                    "sha256": "f524ea435b16f5c6f6fd35af32782df8b223aa89d1ae5ada6527041559d50c97"
                },
                "downloads": -1,
                "filename": "pymulproc-0.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "159a9df38d1e9e4dcaf124b07a5939ce",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6",
                "size": 17058,
                "upload_time": "2019-04-16T12:43:22",
                "url": "https://files.pythonhosted.org/packages/44/ea/8c60c65b38610ed3e0828246be1e701225b38915d175aabaaa2a78d461d5/pymulproc-0.1.1.tar.gz"
            }
        ],
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "150f29af9f76ebcd6b5c9d08e0a0a1ec",
                    "sha256": "2b208960f71c5ecdfdce60cddb7a421a5c00f1ab24753c828285de355bb1ca20"
                },
                "downloads": -1,
                "filename": "pymulproc-0.1.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "150f29af9f76ebcd6b5c9d08e0a0a1ec",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6",
                "size": 9457,
                "upload_time": "2019-08-14T17:08:30",
                "url": "https://files.pythonhosted.org/packages/60/96/b3e40d9a076a1225da01c62d74f45bcc17592968711318131996d13f57cb/pymulproc-0.1.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "cc4f254ccffc964f0c1b278a2fd27d93",
                    "sha256": "9d2b7e87f9c29eeab7c5ec592c15ee852be9ce8c575bc66053dbdc2cb0ef9e43"
                },
                "downloads": -1,
                "filename": "pymulproc-0.1.2.tar.gz",
                "has_sig": false,
                "md5_digest": "cc4f254ccffc964f0c1b278a2fd27d93",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6",
                "size": 18843,
                "upload_time": "2019-08-14T17:08:31",
                "url": "https://files.pythonhosted.org/packages/04/af/d0bf79ae104491d196a2caabf43eea2885f74ab1944e9192f04b962bbbf9/pymulproc-0.1.2.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "150f29af9f76ebcd6b5c9d08e0a0a1ec",
                "sha256": "2b208960f71c5ecdfdce60cddb7a421a5c00f1ab24753c828285de355bb1ca20"
            },
            "downloads": -1,
            "filename": "pymulproc-0.1.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "150f29af9f76ebcd6b5c9d08e0a0a1ec",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.6",
            "size": 9457,
            "upload_time": "2019-08-14T17:08:30",
            "url": "https://files.pythonhosted.org/packages/60/96/b3e40d9a076a1225da01c62d74f45bcc17592968711318131996d13f57cb/pymulproc-0.1.2-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "cc4f254ccffc964f0c1b278a2fd27d93",
                "sha256": "9d2b7e87f9c29eeab7c5ec592c15ee852be9ce8c575bc66053dbdc2cb0ef9e43"
            },
            "downloads": -1,
            "filename": "pymulproc-0.1.2.tar.gz",
            "has_sig": false,
            "md5_digest": "cc4f254ccffc964f0c1b278a2fd27d93",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.6",
            "size": 18843,
            "upload_time": "2019-08-14T17:08:31",
            "url": "https://files.pythonhosted.org/packages/04/af/d0bf79ae104491d196a2caabf43eea2885f74ab1944e9192f04b962bbbf9/pymulproc-0.1.2.tar.gz"
        }
    ]
}