{ "info": { "author": "Matthew J. Aburn", "author_email": "mattja6@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 2 - Pre-Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License (GPL)", "Operating System :: OS Independent", "Programming Language :: Python", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: System :: Distributed Computing" ], "description": "distob\n======\n| Distributed computing made easier, using remote objects\n| N.B. this is a development pre-release: still a lot left to be done\n\nOverview\n--------\nDistob will take your existing python objects, or a sequence of objects,\nand scatter them onto many IPython parallel engines, which may be\nrunning on a single computer or on a cluster.\n\nIn place of the original objects, proxy objects are kept on the client\ncomputer that provide the same interface as the original objects. You\ncan continue to use these as if the objects were still local. All\nmethods are passed through to the remote objects, where computation is done.\n\nIn particular, sending numpy arrays to the cluster is supported. \n\nA numpy array can also be scattered across the cluster, along a particular axis. Operations on the array can then be automatically done in parallel (either using ufuncs, or by using ``vectorize()`` below)\n\nNote: numpy with ``__numpy_ufunc__`` feature enabled (not yet released) is required to support distributed array arithmetic and distributed ufuncs. You can get numpy with this experimental feature enabled here: https://github.com/mattja/numpy/archive/master.zip\n\nDistob is an object layer built on top of ``ipyparallel``, so it will\nmake use of your default IPython parallel profile. This allows different\ncluster architectures, local CPUs, SSH nodes, PBS, Amazon EC2, etc.\n\nfunctions\n---------\n| ``scatter(obj)`` Distribute any object (or list of objects) to remote iPython engines, return a proxy.\n| ``gather(obj)`` Fetch back a distributed object (or list), making it local again.\n|\n| ``vectorize(f)`` Turn an ordinary function (that takes a single object or array) into one that acts in parallel on a scattered list or array. ``apply(f, obj)`` is the same as ``vectorize(f)(obj)``\n\ndistributed numpy arrays\n~~~~~~~~~~~~~~~~~~~~~~~~\n| ``scatter(a, axis=2)`` Distribute a single numpy array along axis 2, returning a DistArray.\n| \n| Arithmetic operations can freely mix ordinary arrays with the new array types.\n| Normal numpy ufuncs can also be used on the distributed arrays.\n| Arithmetic and ufunc computations will automatically be routed to an engine, or executed in parallel on several engines, depending on where the data is. (requires numpy with the ``__numpy_ufunc__`` feature enabled)\n\n| ``concatenate``, ``vstack``, ``hstack``, ``dstack``, ``expand_dims``, ``transpose``, ``rollaxis``, ``split``, ``vsplit``, ``hsplit``, ``dsplit``, ``broadcast_arrays``:\n| These work like the numpy functions of the same name. But these can be used with a mix of ordinary ndarrays, RemoteArrays and DistArrays, performing array structural changes while keeping the actual data distributed across multiple engines.\n| For example, stacking several RemoteArrays gives a DistArray, without needing to move data.\n\n| The distributed arrays so far support basic indexing, slices and advanced integer indexing.\n\nclasses\n-------\n| ``RemoteArray`` proxy object representing a remote numpy ndarray\n| ``DistArray`` a single ndarray distributed across multiple engines\n| \n| ``Remote`` base class, used when auto-creating ``Remote*`` proxy classes\n| ``@proxy_methods(base)`` class decorator for auto-creating ``Remote*`` proxy classes\n| ``ObjectHub`` dict interface giving refs to all distributed objects cluster-wide\n| ``ObjectEngine`` dict holding the distributed objects of a single IPython engine\n| ``Ref`` reference to a (possibly remote) object\n\nattributes\n----------\n``engine``: the ``ObjectEngine`` instance on each host (``ObjectHub`` on\nthe client)\n\nTODO\n----\n- Allow assignment to slices of remote arrays.\n\n- Properly implement caching of remote method results.\n\n- Auto-creation of proxy classes at runtime (depends uqfoundation/dill#58)\n\n- For ufunc execution, still need to implement ``reduce``, ``accumulate``, ``reduceat``, ``outer``, ``at`` methods.\n\n- Make proxy classes more robust, adapting ``wrapt`` (pypi.python.org/pypi/wrapt)\n\nThanks\n------\nIncorporates ``pylru.py`` by Jay Hutchinson,\nhttp://github.com/jlhutch/pylru\n\n``ipyparallel`` interactive parallel computing:\nhttps://ipyparallel.readthedocs.org/\n\n``dill`` by Mike McKerns for object serialization, see:\nhttp://trac.mystic.cacr.caltech.edu/project/pathos", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/mattja/distob/", "keywords": "", "license": "GPLv3+", "maintainer": "", "maintainer_email": "", "name": "distob", "package_url": "https://pypi.org/project/distob/", "platform": "any", "project_url": "https://pypi.org/project/distob/", "project_urls": { "Homepage": "http://github.com/mattja/distob/" }, "release_url": "https://pypi.org/project/distob/0.3.3/", "requires_dist": null, "requires_python": "", "summary": "Distributed computing made easier, using remote objects", "version": "0.3.3" }, "last_serial": 5180243, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "eb91d9cb795e16edf0fc9dd96981e26c", "sha256": "1e55ad08e61631750cfe983fa8bdff9765d5d9b978d3e3e909e1e468559e94b3" }, "downloads": -1, "filename": "distob-0.1.0.tar.gz", "has_sig": false, "md5_digest": "eb91d9cb795e16edf0fc9dd96981e26c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34903, "upload_time": "2014-08-07T06:21:59", "url": "https://files.pythonhosted.org/packages/c9/5c/327633a7f54d81d32027e9cb6489739e54f531dd7152940b03d0f9f7f5ed/distob-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "337461381d8383b8778ed9cf541e0abf", "sha256": "adbb51cbc2af1a9342e8b75d96be0a60691910bb0d2466333d78594379bfe919" }, "downloads": -1, "filename": "distob-0.1.1.tar.gz", "has_sig": false, "md5_digest": "337461381d8383b8778ed9cf541e0abf", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34856, "upload_time": "2014-08-07T14:29:30", "url": "https://files.pythonhosted.org/packages/58/80/8c8ab387d89a7474ec03734a7c53ab1a3f2945858438e24c186ab8870704/distob-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "8490e826f8713cac33f33ad2b937084b", "sha256": "af7a007313c44f21ef10a029872bb261c5b9dd4bd9ce5577ce6d394f8d1d873d" }, "downloads": -1, "filename": "distob-0.1.2.tar.gz", "has_sig": false, "md5_digest": "8490e826f8713cac33f33ad2b937084b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34997, "upload_time": "2014-08-12T06:36:43", "url": "https://files.pythonhosted.org/packages/37/7f/a44d1855a755987eab0aa0b3114d7ebedf3b4cab71c8108ac4de2a1c65b1/distob-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "613e46a58bf073cf081ad27b29517352", "sha256": "31670eab84f05ce27adf6138e0f05fd1e8f566ce7cf15b96c0d07ce78b214b26" }, "downloads": -1, "filename": "distob-0.1.3.tar.gz", "has_sig": false, "md5_digest": "613e46a58bf073cf081ad27b29517352", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42085, "upload_time": "2014-08-25T23:45:22", "url": "https://files.pythonhosted.org/packages/d8/2a/a393c085131c76d8dc1c3128b9c88ce4756a40391108ec5989b1f489f0b0/distob-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "572406756b530f69fef5d3f40ba6cf10", "sha256": "3a50a2abc47122e2af4a4f7254f3f2fd26d2ee7c722d433d7e4e18ca4f90a38a" }, "downloads": -1, "filename": "distob-0.1.4.tar.gz", "has_sig": false, "md5_digest": "572406756b530f69fef5d3f40ba6cf10", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 45185, "upload_time": "2014-09-11T23:54:57", "url": "https://files.pythonhosted.org/packages/0b/a9/74b010f89821fbf2067979cf9cd10534b60369034028e149248705030af2/distob-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "23762480e7ee6528cda0ea60f6e60a9f", "sha256": "cda86fe2b0ac09f45ff968fec01d5d6f17e5ee14f9c792e8a60195d3fb484bca" }, "downloads": -1, "filename": "distob-0.1.5.tar.gz", "has_sig": false, "md5_digest": "23762480e7ee6528cda0ea60f6e60a9f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 45529, "upload_time": "2014-09-19T01:55:52", "url": "https://files.pythonhosted.org/packages/1f/e2/095ed27a4dbacbf95aef2e193172aee83fb3030f1657489bf31d6b27d287/distob-0.1.5.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "0033a6561a5d30d4971182f10e5cf131", "sha256": "9a26ceb58ac66235636927e6d57b86cd765f35b3a8caaf9eb5b6188944d39593" }, "downloads": -1, "filename": "distob-0.2.0.tar.gz", "has_sig": false, "md5_digest": "0033a6561a5d30d4971182f10e5cf131", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 50586, "upload_time": "2014-10-01T03:08:15", "url": "https://files.pythonhosted.org/packages/a6/0b/327907d59d371a6df7def79ca8333b66cfecb1ed84b859107a5c69ffc8c9/distob-0.2.0.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "bd747a2c2fb21e1c4b0882b185f9f001", "sha256": "824598018490731b5d55aa1df0480edd440a4b67bec009954e3bcf69335e7319" }, "downloads": -1, "filename": "distob-0.3.0.tar.gz", "has_sig": false, "md5_digest": "bd747a2c2fb21e1c4b0882b185f9f001", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 54877, "upload_time": "2016-01-15T07:25:46", "url": "https://files.pythonhosted.org/packages/b8/fb/9b6ecf8b17467aaca2ca9a72b06820d2a0a58c3dec44b0a81615f0cb48b7/distob-0.3.0.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "3fb72147ab50d7587f06f9d9a76b7e01", "sha256": "81402234fceb79555b5f9a58f0df9ec373cc38d90b7264bfb4eabe2c8dc4187f" }, "downloads": -1, "filename": "distob-0.3.1.zip", "has_sig": false, "md5_digest": "3fb72147ab50d7587f06f9d9a76b7e01", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 63591, "upload_time": "2016-06-23T14:27:18", "url": "https://files.pythonhosted.org/packages/6d/a3/81749036f3c3ba5fc8846438537465fc8c4722679e447d677ff867154074/distob-0.3.1.zip" } ], "0.3.2": [ { "comment_text": "", "digests": { "md5": "3fd991925bea23aece49471fc651e0f0", "sha256": "5cab4a55ae0e9521c3ef0a737d2c2f169b7ea133ae94c1fda8b62210fc2369b3" }, "downloads": -1, "filename": "distob-0.3.2.tar.gz", "has_sig": false, "md5_digest": "3fd991925bea23aece49471fc651e0f0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 55196, "upload_time": "2017-01-09T13:17:17", "url": "https://files.pythonhosted.org/packages/3a/a8/d96f34522166ecd76f181ad428f047889cbb78f0bfc0842cbf2287b663e5/distob-0.3.2.tar.gz" } ], "0.3.3": [ { "comment_text": "", "digests": { "md5": "ee6c4dd05738fa08a430e64ccbf86b1a", "sha256": "93ddc0a8d0f1d1a41d66153b518dffad685a91b9bec9e99f40a3aca0f8065bee" }, "downloads": -1, "filename": "distob-0.3.3.tar.gz", "has_sig": false, "md5_digest": "ee6c4dd05738fa08a430e64ccbf86b1a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 55524, "upload_time": "2019-04-24T04:25:16", "url": "https://files.pythonhosted.org/packages/bb/48/9a0ba5ec94063ba5eea13f9f0f5f9c63f9979e93046930e83544a6c2dbe5/distob-0.3.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ee6c4dd05738fa08a430e64ccbf86b1a", "sha256": "93ddc0a8d0f1d1a41d66153b518dffad685a91b9bec9e99f40a3aca0f8065bee" }, "downloads": -1, "filename": "distob-0.3.3.tar.gz", "has_sig": false, "md5_digest": "ee6c4dd05738fa08a430e64ccbf86b1a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 55524, "upload_time": "2019-04-24T04:25:16", "url": "https://files.pythonhosted.org/packages/bb/48/9a0ba5ec94063ba5eea13f9f0f5f9c63f9979e93046930e83544a6c2dbe5/distob-0.3.3.tar.gz" } ] }