{ "info": { "author": "James Socol", "author_email": "me@jamessocol.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 2 - Pre-Alpha", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python", "Topic :: Utilities" ], "description": "========\npluribus\n========\n\nHaving just finished reading the original Google `MapReduce paper`_, I\nobviously felt the need to try to implement such a system in Python.\n\nMy goals are to implement enough of the functionality described in the\npaper to be usable, though I strongly warn against ever using this code\nfor anything real.\n\nSince one of the goals (see Goals, below) is simplicity from an end-user\nstandpoint, I am following some of Kenneth Reitz's advice_ and starting\nwith a readme and documentation.\n\n\nExamples\n========\n\nThe canonical word-count example::\n\n # myjob.py\n from pluribus import job\n\n\n @job.map_\n def emit_words(key, value):\n # key: document name\n # value: document contents\n for word in value.split():\n yield word, 1\n\n\n @job.reduce_\n def sum_occurences(key, values):\n # key: a word\n # values: a list of counts\n return sum(values)\n\n\nAssuming you're running everything on one host, you can ignore the\nnetwork connection information.\n\nStart a pluribus **master**::\n\n $ pluribus master\n\nStart a pluribus **worker** (or several hundred)::\n\n $ pluribus worker\n\nOn the master or on another machine that can talk to the master::\n\n $ pluribus job myjob\n # ... wait\n \n\n\nGoals\n=====\n\nExplicit goals are:\n\n* Simple to use, both as an administrator and end-user.\n* Well-documented.\n* Robust to worker failure.\n* Fast-enough.\n* Use only the Python (2.7+) standard library (at least to run).\n\nExplicit non-goals are:\n\n* Be a filesystem.\n* Robust to master failure.\n\n\n.. _MapReduce paper: http://research.google.com/archive/mapreduce.html\n.. _advice: http://docs.writethedocs.org/en/2013/conference/talks.html#kenneth-reitz", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/jsocol/pluribus", "keywords": null, "license": "Apache v2.0", "maintainer": null, "maintainer_email": null, "name": "pluribus", "package_url": "https://pypi.org/project/pluribus/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/pluribus/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/jsocol/pluribus" }, "release_url": "https://pypi.org/project/pluribus/0.0.1/", "requires_dist": null, "requires_python": null, "summary": "A pure-python highly-distributed MapReduce cluster.", "version": "0.0.1" }, "last_serial": 675854, "releases": { "0.0.1": [] }, "urls": [] }