{ "info": { "author": "Jo\u00e3o Jer\u00f3nimo", "author_email": "jj@crowdprocess.com", "bugtrack_url": null, "classifiers": [], "description": "CrowdProcess API Client for Python\n==================================\n\nThis is a client for `CrowdProcess `__'s\n`REST API `__.\n\nIt works in python 2.7 and 3.4+.\n\nInstalling\n----------\n\n::\n\n pip install crowdprocess\n\nor\n\n::\n\n easy_install crowdprocess\n\nUsage example\n-------------\n\n.. code:: python\n\n >>> from crowdprocess import CrowdProcess\n >>> crp = CrowdProcess('username', 'password')\n\n >>> x2 = crp.job('function Run (d) { return d*2; }')\n >>> results = x2(range(5)).results\n >>> list(results)\n [0, 2, 4, 6, 8, 10] # comes in a random order\n\nMore detailed use\n-----------------\n\nImporting and instanciating\n~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n.. code:: python\n\n >>> from crowdprocess import CrowdProcess\n >>> crp = CrowdProcess('username@email.com', 'password')\n\nTo get those credentials you must\n`register `__ with CrowdProcess.\n\nYou can also instanciate it with a token instead of a username and\npassword:\n\n.. code:: python\n\n >>> crp = CrowdProcess(token='3c46d593-5435-47c5-92aa-1613ade978c2')\n\nJobs\n~~~~\n\nCreating a job\n^^^^^^^^^^^^^^\n\nWith the ``CrowdProcess`` class instanciated above,\n\n.. code:: python\n\n >>> program='function Run (d) { return d }'\n >>> job = crp.job(program)\n >>> job.id\n '3c46d593-5435-47c5-92aa-1613ade978c2'\n\nInvoking ``crp.job`` with the ``program`` parameter automatically\ncreates a job in CrowdProcess and returns an instanciated ``Job``.\n\nAfter you get a ``job.id``, you can use it to get a ``Job`` again,\nwithout creating it:\n\n.. code:: python\n\n >>> job = crp.job(id='3c46d593-5435-47c5-92aa-1613ade978c2')\n\nListing jobs\n^^^^^^^^^^^^\n\n.. code:: python\n\n >>> crp.list_jobs()\n [{u'status': u'active', u'failed': 0, u'bid': 1, u'created': u'2014-05-14T10:07:52.747503Z', u'modified': u'2014-05-14T10:07:53.716147Z', u'browserHours': 137, u'finished': 1000, u'lastResult': u'2014-05-14T10:07:59.06Z', u'total': 1000, u'id': u'3c46d593-5435-47c5-92aa-1613ade978c2'}]\n\nPrettier:\n\n.. code:: python\n\n >>> jobs = crp.list_jobs()\n >>> print(json.dumps(jobs, sort_keys=True, indent=2))\n [\n {\n \"bid\": 1, \n \"browserHours\": 137, \n \"created\": \"2014-05-14T10:07:52.747503Z\", \n \"failed\": 0, \n \"finished\": 1000, \n \"id\": \"3c46d593-5435-47c5-92aa-1613ade978c2\", \n \"lastResult\": \"2014-05-14T10:07:59.06Z\", \n \"modified\": \"2014-05-14T10:07:53.716147Z\", \n \"status\": \"active\", \n \"total\": 1000\n }\n ]\n\nDeleting a job\n^^^^^^^^^^^^^^\n\n.. code:: python\n\n >>> job = crp.job(id='3c46d593-5435-47c5-92aa-1613ade978c2')\n >>> job.delete()\n\nDeleting all jobs\n^^^^^^^^^^^^^^^^^\n\n.. code:: python\n\n >>> crp.delete_jobs()\n\nTasks and Results\n-----------------\n\nAfter creating a job, you're all set to send it tasks and get back\nresults.\n\n``tasks`` can be any iterable object, ``results`` will be a generator:\n\n.. code:: python\n\n >>> job = crp.job('function Run (d) { return Math.pow(d, 2); }')\n >>> tasks = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n >>> results = job(tasks).results\n >>> list(results)\n [49, 64, 16, 25, 9, 36, 4, 81, 0, 1]\n\nwhich would be the same as,\n\n.. code:: python\n\n >>> job = crp.job('function Run (d) { return Math.pow(d, 2); }')\n >>> list(job(range(10)).results)\n [49, 64, 16, 25, 9, 36, 4, 81, 0, 1]\n\nwhich would also be the same as,\n\n.. code:: python\n\n >>> job = crp.job('function Run (d) { return Math.pow(d, 2); }')\n >>> def tasks():\n ... for i in range(10):\n ... yield i\n ... \n >>> list(job(tasks).results)\n [25, 64, 49, 16, 36, 9, 0, 81, 1, 4]\n\nNotice that the results never come in order.\n\nPro tip: you can use the results of one job as tasks of another job\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n.. code:: python\n\n >>> multiply = crp.job('function Run (d) { return d*2 }')\n >>> divide = crp.job('function Run (d) { return d/2 }')\n >>> numbers = range(10)\n >>> multiplied = multiply(numbers).results\n >>> divided = divide(multiplied).results\n >>> list(divided)\n [7, 2, 6, 1, 5, 9, 8, 4, 3, 0]\n\nDon't forget about the errors\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nSometimes your tasks will throw uncaught exceptions that you should know\nabout, and you can get them the same way you get results:\n\n.. code:: python\n\n >>> job = crp.job('function Run (d) { if (d === 4) { throw new Error(\"oh no, \"+d) } return d; }')\n >>> tasks = range(10)\n >>> errors = job(tasks).errors\n >>> for error in errors:\n ... print error\n\nTasks and Results, lower level\n------------------------------\n\nSubmitting tasks\n~~~~~~~~~~~~~~~~\n\nOnce again, tasks may be any iterable:\n\n.. code:: python\n\n >>> multiply = crp.job('function Run (d) { return d*2 }')\n >>> multiply.submit_tasks(range(10))\n\nGetting results\n~~~~~~~~~~~~~~~\n\n.. code:: python\n\n >>> results = multiply.get_results()\n >>> list(results)\n [18, 8, 10, 4, 6, 16, 14, 0, 2, 12]\n\nThis delivers all the job's computed results at the moment, but you\nshould in fact get every result as soon as it's computed, in a stream:\n\nStreaming results\n~~~~~~~~~~~~~~~~~\n\nYou can also iterate through every result as soon as it comes in:\n\n.. code:: python\n\n >>> expected_results = 10\n >>> results = multiply.get_results_stream()\n >>> for result in results:\n ... print(result)\n ... expected_results -= 1\n ... if expected_results == 0:\n ... break\n\nThe stream does not know if or when a result might be computed and\ndelivered, so you must count how many results you still expect to break\nthe loop.\n\nTo use this properly you should start listening for streaming results\nbefore sending tasks, probably a separate thread:\n\n.. code:: python\n\n >>> import threading\n >>> job = crp.job(\"function Run(d) { return d; }\")\n >>> def get_results():\n ... expected_results = 10\n ... for result in job.get_results_stream():\n ... print(result)\n ... expected_results -= 1\n ... if expected_results == 0:\n ... break\n ... \n >>> t = threading.Thread(target=get_results)\n >>> t.start()\n >>> job.submit_tasks(range(10))\n >>> 7\n 9\n 6\n 2\n 3\n 8\n 1\n 4\n 0\n 5\n >>> t.join()\n\nSometimes your tasks will have uncaught exceptions and those will cause\na result to not be delivered, so you must account for those as well to\ndecrease your expected\\_results counter.\n\nErrors and streaming errors\n~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nSometimes your tasks throw uncaught exceptions, and you should get them:\n\n.. code:: python\n\n >>> program = \"\"\"\n ... function Run (d) {\n ... if (d === 4) {\n ... throw new Error(\"oops, it's \"+d);\n ... } else {\n ... return d;\n ... }\n ... }\n ... \"\"\"\n >>> job = crp.job(program)\n >>> job.submit_tasks(range(10))\n >>> list(job.get_results())\n [1, 6, 9, 8, 5, 7, 2, 3, 0] # oh no, 4 is missing...\n >>> list(job.get_errors())\n [{u'message': u\"oops, it's 4\", u'type': u'program', u'name': u'Error', u'stack': u'Run@blob:9a4029f7-fff7-4da8-b552-92507e341749:5\\n[2]>> print(json.dumps(list(job.get_errors()), sort_keys=True, indent=2)) # prettier\n [\n {\n \"message\": \"oops, it's 4\", \n \"name\": \"Error\", \n \"stack\": \"Run@blob:9a4029f7-fff7-4da8-b552-92507e341749:5\\n[2]>> errors = multiply.get_errors_stream()\n >>> for error in errors:\n ... print(error)", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/CrowdProcess/crpy", "keywords": null, "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "CrowdProcess", "package_url": "https://pypi.org/project/CrowdProcess/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/CrowdProcess/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/CrowdProcess/crpy" }, "release_url": "https://pypi.org/project/CrowdProcess/0.2.0/", "requires_dist": null, "requires_python": null, "summary": "UNKNOWN", "version": "0.2.0" }, "last_serial": 1099897, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "33aa9d6fdd27296a6f02a650a3ce1dac", "sha256": "25532adfff63bdc6fde68c0fa85888bf6ccb5f45d87a8e2a2874ff827ef97a91" }, "downloads": -1, "filename": "CrowdProcess-0.1.tar.gz", "has_sig": false, "md5_digest": "33aa9d6fdd27296a6f02a650a3ce1dac", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2264, "upload_time": "2014-05-15T16:21:18", "url": "https://files.pythonhosted.org/packages/25/69/f927f0c39c87ff794fb0a1ab25b60ef44c1ee22538915f7180ee478cad8d/CrowdProcess-0.1.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "fed72bf992c4cee3ebbf994882200b8e", "sha256": "98fe7a8f8b5f32ac9825e49e379ef7d12d0532bf3a02042fec753c728da9130a" }, "downloads": -1, "filename": "CrowdProcess-0.1.1.tar.gz", "has_sig": false, "md5_digest": "fed72bf992c4cee3ebbf994882200b8e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2318, "upload_time": "2014-05-15T16:23:22", "url": "https://files.pythonhosted.org/packages/8c/36/b9b1ad2781a326852e60fda102b26aed74eb02c4829a5b99b55e16475565/CrowdProcess-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "b0dcb8d47e4a0fac45346ca1cac0b5c4", "sha256": "a38a8bf02e9745e4de23f13aef95784d3611c65fa8a0cd530a946d4ae19fe8f8" }, "downloads": -1, "filename": "CrowdProcess-0.1.2.tar.gz", "has_sig": false, "md5_digest": "b0dcb8d47e4a0fac45346ca1cac0b5c4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2322, "upload_time": "2014-05-15T16:57:04", "url": "https://files.pythonhosted.org/packages/6f/12/ff0ac942f2291db217c38faa2bb988e68db617c87500fa30d7e4a2594a99/CrowdProcess-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "3d5ab1f4c491b9ff87344381978aac3e", "sha256": "878709895fef3969ac58b2749a40ec59ccd7015deaa328683b70cbb4ec428684" }, "downloads": -1, "filename": "CrowdProcess-0.1.3.tar.gz", "has_sig": false, "md5_digest": "3d5ab1f4c491b9ff87344381978aac3e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2318, "upload_time": "2014-05-15T17:19:37", "url": "https://files.pythonhosted.org/packages/47/0d/8e35347dab8f3bba1b89dc5e594525f8d12fe542b4bead157f97ad6d712e/CrowdProcess-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "aadd4a29e906e79b16db92f13c0c54da", "sha256": "7d08cd834ab7f33a47cb7aee860e2d2e4a41605702f1da44d6ea502b189c3ab9" }, "downloads": -1, "filename": "CrowdProcess-0.1.4.tar.gz", "has_sig": false, "md5_digest": "aadd4a29e906e79b16db92f13c0c54da", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5024, "upload_time": "2014-05-16T14:39:30", "url": "https://files.pythonhosted.org/packages/0c/8d/2e6499f2392b64be66ff733d081757f33af9f7eeed7360bcb93a7209bc2a/CrowdProcess-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "9f12dabdf85b8b29aa214323304408c6", "sha256": "14389c5ed8c25eaa95a05e8c64932b0f17cd84855748bc6b9576289660b6a279" }, "downloads": -1, "filename": "CrowdProcess-0.1.5.tar.gz", "has_sig": false, "md5_digest": "9f12dabdf85b8b29aa214323304408c6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5056, "upload_time": "2014-05-16T15:11:49", "url": "https://files.pythonhosted.org/packages/6d/fb/3c32007e29dfed8bf38526b8bc0849c90ecf7e0c25594a8052e11f6e4e8d/CrowdProcess-0.1.5.tar.gz" } ], "0.1.6": [ { "comment_text": "", "digests": { "md5": "fe7355ab4531765d3eb96fa2f488a3c3", "sha256": "883648db1c32383ee0bec9be75486d8d442fb124a57184109a4bc3567c6815a0" }, "downloads": -1, "filename": "CrowdProcess-0.1.6.tar.gz", "has_sig": false, "md5_digest": "fe7355ab4531765d3eb96fa2f488a3c3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5243, "upload_time": "2014-05-16T15:28:52", "url": "https://files.pythonhosted.org/packages/07/55/6693969345e0e2f62fd7b6e67a2d02e6597db797b5d28c62056d56d20bbe/CrowdProcess-0.1.6.tar.gz" } ], "0.1.7": [ { "comment_text": "", "digests": { "md5": "e086db574ba68863cc6065fc545ebccc", "sha256": "77c8d899585d705682ab649c9bb6f10b9f1d03cabbbae2c027ecdfbe699e72d6" }, "downloads": -1, "filename": "CrowdProcess-0.1.7.tar.gz", "has_sig": false, "md5_digest": "e086db574ba68863cc6065fc545ebccc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5242, "upload_time": "2014-05-16T15:30:55", "url": "https://files.pythonhosted.org/packages/4c/0d/ce030194233dfab623744cc611f72b48ed6e5c1e74add1fe81783a4dd192/CrowdProcess-0.1.7.tar.gz" } ], "0.1.8": [ { "comment_text": "", "digests": { "md5": "5d37bcb1b4ff8efc998e9a81abd61937", "sha256": "e11e51f73ada4e1e24ddf0980335c258e7351753f4889c82292cb637b14d8a24" }, "downloads": -1, "filename": "CrowdProcess-0.1.8.tar.gz", "has_sig": false, "md5_digest": "5d37bcb1b4ff8efc998e9a81abd61937", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5234, "upload_time": "2014-05-16T15:39:35", "url": "https://files.pythonhosted.org/packages/8f/a4/596c9fad241414b20b0ce35f7cbe981d4528f70c568d683d030aad38eb1c/CrowdProcess-0.1.8.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "57217a9b247bc2fff8c12714002d4a01", "sha256": "2f987e29e494220ac3dd6157832e52c8506292892c8a3a7e1633fb79da368af4" }, "downloads": -1, "filename": "CrowdProcess-0.2.0.tar.gz", "has_sig": false, "md5_digest": "57217a9b247bc2fff8c12714002d4a01", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5123, "upload_time": "2014-05-21T16:19:23", "url": "https://files.pythonhosted.org/packages/82/13/0db24e85df4b28aca26e624612cd2886516b86a554c54105285f870c2869/CrowdProcess-0.2.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "57217a9b247bc2fff8c12714002d4a01", "sha256": "2f987e29e494220ac3dd6157832e52c8506292892c8a3a7e1633fb79da368af4" }, "downloads": -1, "filename": "CrowdProcess-0.2.0.tar.gz", "has_sig": false, "md5_digest": "57217a9b247bc2fff8c12714002d4a01", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5123, "upload_time": "2014-05-21T16:19:23", "url": "https://files.pythonhosted.org/packages/82/13/0db24e85df4b28aca26e624612cd2886516b86a554c54105285f870c2869/CrowdProcess-0.2.0.tar.gz" } ] }