{ "info": { "author": "Greg Leighton", "author_email": "grleighton@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Environment :: Console", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7" ], "description": "# RapMedusa\n\nRapMedusa is a Python module implementing MapReduce querying over a \nRedis key-value store. It aims to provide functionality that is similar \n(in some respects) to CouchDB's views feature and MongoDB's MapReduce database \ncommand.\n\n## Dependencies\n\nRapMedusa depends on Andy McCurdy's `redis-py` module, which can be \nobtained from [https://github.com/andymccurdy/redis-py][1]. Of course, \nyou'll also need a running [Redis][2] instance to connect to. In both cases,\nany version >= 2.0 should be compatible with RapMedusa.\n\n## Installation\n\n $ sudo pip install rapmedusa\n\nor\n\n $ sudo easy_install rapmedusa\n\nor from source:\n\n $ sudo python setup.py install\n\n## Overview\n\nFirst, import the required modules:\n\n :::python\n >>> import redis\n >>> from rapmedusa import emit, map_reduce\n\nNext, connect to a running Redis instance in the usual way:\n\n :::python\n >>> redis = redis.StrictRedis(host='localhost', port=6379, db=0)\n\nFinally, implementations of the map and reduce functions must be provided, and passed into a call to the map_reduce() function, along with the active connection to the Redis server:\n\n :::python\n >>> def myMap(key, val):\n ...\n emit(newKey, newVal)\n\n \n >>> def myReduce(key, values):\n ...\n return newVal\n\n >>> result = map_reduce(redis, myMap, myReduce)\n\nThis returns a Python dictionary object, containing the result of running the MapReduce job. Each key within the dictionary corresponds to a key passed into the reduce function, and contains the value computed by the reduce function for that key.\n\n\n## Details\n\nNow it's time to take a deeper look into how RapMedusa carries out a MapReduce job. There are basically 6 steps:\n\n\t1) Read the input data set from a specified Redis hash.\n\t2) Pass each key/value pair from the input data set to the registered map function.\n\t3) Organize key/value pairs emitted by the map function into a set of Redis lists, one list per distinct emitted key.\n\t4) Each of these lists is passed to the registered reduce function, along with the corresponding key.\n\t5) The result of each call to reduce is stored in the Redis hash reserved for the job output, under the key used in the reduce call.\n\t6) A Python dictionary representing the contents of the job output hash is returned.\n\t\nA natural question at this point is how are the input and output hash keys specified? These (and other temporary Redis keys used in the \nabove steps) can optionally be specified within the call to map_reduce(). Here's a list of the additional, optional parameters that can be \nspecified in the call:\n\n\t* inKey -- specifies the key under which the input data set is stored (defaults to 'rapmedusa:inputs')\n\t* outKey -- specifies the key under which the job output is stored (defaults to 'rapmedusa:outputs')\n\t* sortKey -- specifies the key prefix under which the output of the map function (step 3 above) is stored (defaults to 'rapmedusa:sortedVals')\n\t* sortedKeySet -- specifies the key under which the set formed from the list keys of step 3 is stored (defaults to 'rapmedusa:sortedKeySet')\n\t* cleanUp -- a boolean value indicating whether the temporary keys (sortKey, sortedKeySet) should be deleted from the Redis store upon the completion of the MapReduce job (defaults to True) \n\t\nYou'll rarely need to override the default values for sortedKey and sortedKeySet, as a naming conflict is highly unlikely. You may, however, wish to specify custom values for inKey and outKey that are easier to remember. \n\n\n## Examples\n\n### Example 1: Counting Ages\n\nThis example demonstrates a MapReduce job in which the input keys are mapped to person records, and the map function generates keys \nbased on one of the record entries, `age`.\n\n :::python\n\t>>> import redis\n\t>>> from rapmedusa import *\n\t\n\t>>> conn = redis.StrictRedis(host='localhost', port=6379, db=0)\n\t>>> conn.hset('myInput', 1, \"{'name': 'Chad', 'age': 43}\")\n\t>>> conn.hset('myInput', 2, \"{'name': 'Ron', 'age': 21}\")\n\t>>> conn.hset('myInput', 3, \"{'name': 'George', 'age': 54}\")\n\t>>> conn.hset('myInput', 4, \"{'name': 'Alice', 'age': 54}\")\n\n\t>>> def myMap(key, value):\n\t\t\tobj = eval(value)\n\t\t\temit(str(obj['age']), '1')\n\n\t\n\t>>> def myReduce(key, vals):\n\t\t\ttotal = 0\n\t\t\tfor v in vals:\n\t\t\t\ttotal += int(v)\n\t\t\treturn total\n\n\t>>> result = map_reduce(conn, myMap, myReduce, inKey='myInput')\n\t>>> print result\n\t{'54': '2', '21': '1', '43': '1'}\n\n\n\n\nAuthor\n------\n\nRapMedusa is developed and maintained by Greg Leighton (grleighton@gmail.com).\nThe most up-to-date version can be downloaded at [https://github.com/gleighto/rapmedusa][3].\n\n[1]: https://github.com/andymccurdy/redis-py\n[2]: http://redis.io\n[3]: https://github.com/gleighto/rapmedusa", "description_content_type": null, "docs_url": null, "download_url": "http://cloud.github.com/downloads/gleighto/rapmedusa/rapmedusa-1.0.tar.gz", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/gleighto/rapmedusa", "keywords": "Redis,key-value store,MapReduce", "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "rapmedusa", "package_url": "https://pypi.org/project/rapmedusa/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/rapmedusa/", "project_urls": { "Download": "http://cloud.github.com/downloads/gleighto/rapmedusa/rapmedusa-1.0.tar.gz", "Homepage": "http://github.com/gleighto/rapmedusa" }, "release_url": "https://pypi.org/project/rapmedusa/1.0/", "requires_dist": null, "requires_python": null, "summary": "Python implementation of MapReduce querying for Redis", "version": "1.0" }, "last_serial": 803295, "releases": { "1.0": [] }, "urls": [] }