{
    "info": {
        "author": "herrlich10",
        "author_email": "herrlich10@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Intended Audience :: Developers",
            "Intended Audience :: Science/Research",
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 3",
            "Topic :: Software Development :: Libraries",
            "Topic :: Utilities"
        ],
        "description": ".. image:: https://img.shields.io/pypi/v/paraproc.svg\n   :target: https://pypi.python.org/pypi/paraproc\n\n.. image:: https://img.shields.io/badge/license-MIT-green.svg\n   :target: https://github.com/herrlich10/paraproc/blob/master/LICENSE.txt\n\nOverview\n========\nParaproc is a simple library that helps you easily parallelize your computation\n(over independent chunks of data) across multiple processes in Python, especially \nwhen you want to mix callings to external command line programs and hand brew \nPython functions together in your data processing pipeline.\n\nUnder the hood, it combines subprocess and multiprocessing, and uses a process pool\nto schedule the jobs. It also provides a numpy.ndarray interface to access \nshared-memory across multiple processes.\n\nParaproc supports both Python 2 and 3, with numpy as the only external dependency. \nIt is contained in only one Python file, so it can be easily copied into your project. \n(The copyright and license notice must be retained.)\n\nCode snippets that demonstrate the basic usage of the library can be found later\nin this documentation, and in the demo_*.py files.\n\nBugs can be reported to https://github.com/herrlich10/paraproc. \nThe code can also be found there.\n\nQuick starts\n============\nExecute commands in parallel\n----------------------------\nYou can run both Python codes and command line programs in parallel:\n\n.. code:: python\n\n    import os\n    import paraproc\n    def my_job():\n        print(os.getpid())\n\n    pc = paraproc.PooledCaller()\n    for k in range(5):\n        pc.check_call(my_job) \n    for k in range(5):\n        pc.check_call('echo $$', shell=True) # For linux/mac\n    pc.wait()\n\nThe ``pc.check_call()`` method will return immediatedly. The actual execution of \nthe queued commands are delayed until you call ``pc.wait()``.\n\nUse shared-memory\n-----------------\nYou can load large data in shared-memory, and read or write them \nas a normal numpy array from multiple processes:\n\n.. code:: python\n\n    import numpy as np\n    import paraproc\n    def slow_operation(k, x):\n        x.acquire()\n        x[:100000,:] += 1 # Write access\n        res = np.mean(x) # Read access\n        x.release()\n        print('#{0}: mean = {1}'.format(k, res))\n\n    a = paraproc.SharedMemoryArray.from_array(np.random.rand(1000000,500)) # About 4 GB\n    pc = paraproc.PooledCaller()\n    for k in range(pc.pool_size):\n        pc.check_call(slow_operation, k, a)\n    pc.wait()\n\nThe data in ``a`` is shared in memory across all children processes and \nnever copied even with write accesses.\n\n",
        "description_content_type": "text/x-rst",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/herrlich10/paraproc",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "paraproc",
        "package_url": "https://pypi.org/project/paraproc/",
        "platform": "",
        "project_url": "https://pypi.org/project/paraproc/",
        "project_urls": {
            "Homepage": "https://github.com/herrlich10/paraproc"
        },
        "release_url": "https://pypi.org/project/paraproc/0.1.3/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Easy parallel processing in Python",
        "version": "0.1.3"
    },
    "last_serial": 3938122,
    "releases": {
        "0.1.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "dc66035a482d86aacf8fd9b666b3d1f5",
                    "sha256": "0537b804047a72eb6ac30f254a67bd78151d6c93b8428bc427aca9cb3833f49c"
                },
                "downloads": -1,
                "filename": "paraproc-0.1.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "dc66035a482d86aacf8fd9b666b3d1f5",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 6422,
                "upload_time": "2018-06-07T02:50:51",
                "url": "https://files.pythonhosted.org/packages/5e/31/c6b9e7f406ed04457f6591e0f5ace88de2d16c52b536633a465fbac13582/paraproc-0.1.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "33b961d2be1bf1314d9f54543d0028a0",
                    "sha256": "0687cc1cf9a14dc58229c51941b51c0bb494193b493b451e59dcd5e6e33d8650"
                },
                "downloads": -1,
                "filename": "paraproc-0.1.3.tar.gz",
                "has_sig": false,
                "md5_digest": "33b961d2be1bf1314d9f54543d0028a0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6237,
                "upload_time": "2018-06-07T02:50:53",
                "url": "https://files.pythonhosted.org/packages/a4/95/d95b1cf999c7493fc241df974864ef5c11f3f67de4f5e20274d4a2d19b86/paraproc-0.1.3.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "dc66035a482d86aacf8fd9b666b3d1f5",
                "sha256": "0537b804047a72eb6ac30f254a67bd78151d6c93b8428bc427aca9cb3833f49c"
            },
            "downloads": -1,
            "filename": "paraproc-0.1.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "dc66035a482d86aacf8fd9b666b3d1f5",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 6422,
            "upload_time": "2018-06-07T02:50:51",
            "url": "https://files.pythonhosted.org/packages/5e/31/c6b9e7f406ed04457f6591e0f5ace88de2d16c52b536633a465fbac13582/paraproc-0.1.3-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "33b961d2be1bf1314d9f54543d0028a0",
                "sha256": "0687cc1cf9a14dc58229c51941b51c0bb494193b493b451e59dcd5e6e33d8650"
            },
            "downloads": -1,
            "filename": "paraproc-0.1.3.tar.gz",
            "has_sig": false,
            "md5_digest": "33b961d2be1bf1314d9f54543d0028a0",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 6237,
            "upload_time": "2018-06-07T02:50:53",
            "url": "https://files.pythonhosted.org/packages/a4/95/d95b1cf999c7493fc241df974864ef5c11f3f67de4f5e20274d4a2d19b86/paraproc-0.1.3.tar.gz"
        }
    ]
}