{ "info": { "author": "Markus Mayr", "author_email": "markusmayr@gmx.net", "bugtrack_url": null, "classifiers": [ "Programming Language :: Python :: 3.2", "Topic :: Database" ], "description": "Multiprocess Bulk Uploading to CouchDB\n######################################\n\nStatus\n======\n\nThis is not yet finished, but can already be used.\n\nHistory\n=======\n\n\nv 0.2.7\n-------\n\n* Updated readme\n\nv 0.2.6\n-------\n\n* Introduced the ``jobsbuffersizemax`` argument to limit the amount of upload-processed buffered in working memory. Now it is possible to work on data of infinite size.\n\nv 0.2.5\n-------\n\n* only spawn one concurrent process for performing uploads\n\nv 0.2.4\n-------\n\n* working version, still spans unlimited amount of processes\n\n\nUsage\n=====\n\nCreate a new mpcouchPusher object\n\n.. code-block::\n \n myCouchPusher = mpcouch.mpcouchPusher( \"http://localhost:5984/myDatabase\", 30000 )\n\nIf the data is created faster than the upload happens, it might be necessary to let the generation of new documents pause befor allowing another batch-upload process to be buffered. The amount of bufferd upload-processes is specified by the optional ``jobsbuffersizemax`` argument which defaults to 10. This means that when there are 10 batch-upload processes already waiting in line to be executed, the module holds the main python thread until one process has finished.\nBy adjusing this value it is possible to find an equilibrium between upload-buffer (working memory) and document generation speed.\n\nUse this object every time you have one single document ready to be stored in the database:\n\n.. code-block::\n \n myCouchPusher.pushData(myNewDocument)\n\nThe module will collect all documents until the threshold is reached (in our example this would be the 30000 specified above) and upload them as a batch to the CouchDB also specified at creation time of the object (myCouchDbDatabase).\n\nSince every bulk-upload is performed by a single process, the original program continues while the upload happens in the background.\n\nTo wait for all running uploads to finish and to make sure the very last batch of documents gets pushed to the server, run\n\n.. code-block::\n \n myCouchPusher.finish()\n\nafter your final document was sent to pushData.\nThe module now waits for all the uploads to finish and uploads the final bulk of collected documents.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/scubbx/couch-bulk-multiprocess", "keywords": "couchdb multiprocessing upload bulk", "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "mpcouch", "package_url": "https://pypi.org/project/mpcouch/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/mpcouch/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/scubbx/couch-bulk-multiprocess" }, "release_url": "https://pypi.org/project/mpcouch/0.2.7/", "requires_dist": null, "requires_python": null, "summary": "A multiprocess bulk-uploading helper for CouchDB", "version": "0.2.7" }, "last_serial": 1863830, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "b06e1c0f4a447c3eeadb77ed463bdae9", "sha256": "88aec7b04fc8fc9b352654e684d90f7e5fcb2b0f5c2e6ae77f7ca324bc1116bc" }, "downloads": -1, "filename": "mpcouch-0.1.tar.gz", "has_sig": false, "md5_digest": "b06e1c0f4a447c3eeadb77ed463bdae9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1996, "upload_time": "2015-06-15T15:32:35", "url": "https://files.pythonhosted.org/packages/9a/81/4c7dc29589b8d31c44d82ab1e0e78de88f0159683aedc4c0fdac2320b015/mpcouch-0.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "8b5e046a9ad6aec98523e783c704067d", "sha256": "b7031e939b05f881455bf3672d57d86475ea03f93961f593d6a7eb348d57f28b" }, "downloads": -1, "filename": "mpcouch-0.2.tar.gz", "has_sig": false, "md5_digest": "8b5e046a9ad6aec98523e783c704067d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1951, "upload_time": "2015-06-15T15:42:27", "url": "https://files.pythonhosted.org/packages/21/d7/34870880e2aca1c09f135aac4b713a160f67d84b53e74876f3dbe49da4fd/mpcouch-0.2.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "b2ee3552b350b51f718dc1c7056ad9e3", "sha256": "913f3277dab40fd3b9dc8a8b9d2d7b76af9c802369a20b1343faa8f86042b8cb" }, "downloads": -1, "filename": "mpcouch-0.2.1.tar.gz", "has_sig": false, "md5_digest": "b2ee3552b350b51f718dc1c7056ad9e3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2765, "upload_time": "2015-06-15T15:56:16", "url": "https://files.pythonhosted.org/packages/66/c5/2b9bb5b9f20771fdf600edb62d3e1f12adc89c2f7f8900c5265b8bccc81d/mpcouch-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "713ac75506eeb6fb01594fe4398f0710", "sha256": "43731699f897663285bd741788e7159a22b68e182a2324e96ca207638e36508b" }, "downloads": -1, "filename": "mpcouch-0.2.2.tar.gz", "has_sig": false, "md5_digest": "713ac75506eeb6fb01594fe4398f0710", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2796, "upload_time": "2015-06-15T15:59:43", "url": "https://files.pythonhosted.org/packages/b9/42/a05e1d145bb53fe8140b38883dd315ae9948e63d89f25de9297452d2739f/mpcouch-0.2.2.tar.gz" } ], "0.2.3": [ { "comment_text": "", "digests": { "md5": "707d1a3c9f7cf6e784df4b2c2b068275", "sha256": "7e18906d7a8072abedfc0d5b7e5b3206b79a006eb93db252bd1aab6d6d679a8f" }, "downloads": -1, "filename": "mpcouch-0.2.3.tar.gz", "has_sig": false, "md5_digest": "707d1a3c9f7cf6e784df4b2c2b068275", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2793, "upload_time": "2015-06-15T16:03:09", "url": "https://files.pythonhosted.org/packages/95/3a/85a5b171c84f39eaa911d640e1dc9c0cc73bf12b71f4df36c869d0081ffc/mpcouch-0.2.3.tar.gz" } ], "0.2.4": [ { "comment_text": "", "digests": { "md5": "a43fca98e348d2d1e14dd68b1fe55523", "sha256": "c7fa729840181abb4c51c4d7ac2f209fc5e77b1bd6252e8261d8cd69b891702d" }, "downloads": -1, "filename": "mpcouch-0.2.4.tar.gz", "has_sig": false, "md5_digest": "a43fca98e348d2d1e14dd68b1fe55523", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2816, "upload_time": "2015-06-18T14:30:42", "url": "https://files.pythonhosted.org/packages/85/64/110dfff245cd99e744d8d07a33495df796b82fe7bb8c0db7d391e1dd446b/mpcouch-0.2.4.tar.gz" } ], "0.2.5": [ { "comment_text": "", "digests": { "md5": "36d1de7c477c90cb1a22a61b163cce5d", "sha256": "de472026128da5e17e71cdc1e94215e1439cd2421e05666f9528320a1f954917" }, "downloads": -1, "filename": "mpcouch-0.2.5.tar.gz", "has_sig": false, "md5_digest": "36d1de7c477c90cb1a22a61b163cce5d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3297, "upload_time": "2015-06-28T19:50:58", "url": "https://files.pythonhosted.org/packages/66/31/baa9b214eedd151c912bbdabfa6f2ebedc98d69c4c10c9ce75568777bc50/mpcouch-0.2.5.tar.gz" } ], "0.2.6": [ { "comment_text": "", "digests": { "md5": "93fe8a04b643654593dd089e0a2731af", "sha256": "6a6f8c61b0e9cb55e8311c9b0ba6e8887f426228aaba9e1fa0a655adfc2f46a1" }, "downloads": -1, "filename": "mpcouch-0.2.6.tar.gz", "has_sig": false, "md5_digest": "93fe8a04b643654593dd089e0a2731af", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4069, "upload_time": "2015-12-15T19:53:47", "url": "https://files.pythonhosted.org/packages/7a/7a/9d2ed0cdb4d148aeaffb284ef1e8e580c4b958a1cbff04d9dd7e5aa478da/mpcouch-0.2.6.tar.gz" } ], "0.2.6-dev": [ { "comment_text": "", "digests": { "md5": "ef16b0c8e8f6ae282844ac66cbde260c", "sha256": "10a902e911ce7b005bde22622eddd62b443bdb251d7c3e59f7313608826c0c07" }, "downloads": -1, "filename": "mpcouch-0.2.6-dev.tar.gz", "has_sig": false, "md5_digest": "ef16b0c8e8f6ae282844ac66cbde260c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4288, "upload_time": "2015-12-15T20:01:33", "url": "https://files.pythonhosted.org/packages/ef/20/b2fb707f78af3bb96817543650e9dfaea682419332fbb83093788b63da52/mpcouch-0.2.6-dev.tar.gz" } ], "0.2.6a": [ { "comment_text": "", "digests": { "md5": "adc73175f4977cd37fedaa2fc0614346", "sha256": "b40ff6b7f42fc2a02c4b325b3659e27f53e17bd5bd5c7d1c07c4682d26dd1044" }, "downloads": -1, "filename": "mpcouch-0.2.6a.tar.gz", "has_sig": false, "md5_digest": "adc73175f4977cd37fedaa2fc0614346", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4299, "upload_time": "2015-12-15T20:02:50", "url": "https://files.pythonhosted.org/packages/c8/75/80aefbe5f1f42ca0bd52e71677282c2eecce501af595334461992c76266e/mpcouch-0.2.6a.tar.gz" } ], "0.2.7": [ { "comment_text": "", "digests": { "md5": "c5f1bf70657a9975c68a0d56c6f146a8", "sha256": "509ea58735383a0aa13e51d245c93cd20a54ccf8f08661195637b0278a5f2abf" }, "downloads": -1, "filename": "mpcouch-0.2.7.tar.gz", "has_sig": false, "md5_digest": "c5f1bf70657a9975c68a0d56c6f146a8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4302, "upload_time": "2015-12-15T20:03:55", "url": "https://files.pythonhosted.org/packages/06/2a/885a9d5c79fa05b3618d3fdd4b38b3cab8336549ad1e5b1c669d2031d055/mpcouch-0.2.7.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "c5f1bf70657a9975c68a0d56c6f146a8", "sha256": "509ea58735383a0aa13e51d245c93cd20a54ccf8f08661195637b0278a5f2abf" }, "downloads": -1, "filename": "mpcouch-0.2.7.tar.gz", "has_sig": false, "md5_digest": "c5f1bf70657a9975c68a0d56c6f146a8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4302, "upload_time": "2015-12-15T20:03:55", "url": "https://files.pythonhosted.org/packages/06/2a/885a9d5c79fa05b3618d3fdd4b38b3cab8336549ad1e5b1c669d2031d055/mpcouch-0.2.7.tar.gz" } ] }