{ "info": { "author": "Like Zhang", "author_email": "zlike@yahoo.com", "bugtrack_url": null, "classifiers": [], "description": "# cocotask\n### Build task queue on either Rabbitmq, Kafka or Redis! Simple! Easy! and FAST!!!!\n\n**Why creating this framework?**\n* Most task queues using Rabbitmq/Kafka/Redis are doing the same thing, but there is no unified wrapper. It's a waste of time to write code for different MQs if all you need is a task queue to distribute jobs\n* No need to worry about using Rabbitmq, Kafka or Redis. Just pick one and go! It's simply a few lines of config changes in future if you want to switch to a different underlying system\n* Hide all details for connecting/subscribing/publishing/etc. You can just use the same API for any of Kafka/Rabbitmq/Redis!\n* Minimize the efforts for team members to handcraft the code for exchange/queue handling\n* Team members should focus on how to handle messages\n* There lacks good mananger tool/lib to handle creating multiple consumers (although it's simple)\n* Celery is the original idea, but it doesn't support Kafka and Windows platform. It's also tightly coupled with Python alone.\n
\n\n## Test\n\n1. install rabbitmq or kafka on local machine (either docker or pure rabbitmq).\n - Rabbitmq: https://www.rabbitmq.com/\n - Kafka: https://kafka.apache.org/quickstart (for kafka, you have to manually create a topic named `test_topic_1` in order to run the test. In order to try multiple consumers, you need to set partitions to 2 or above, not 1)\n - Redis: https://redis.io/download\n\n2. make sure it's running Python 3.5 above\n\n3. pip install pika kafka-python redis jsmin\n\n4. pip install cocotask\n\n5. now you have cocotask setup on your machine. To test, you can go to the ./test folder\n - Run: `python producer_test.py` (this will post a string. Code is very simple)\n - In another window, under ./test, run: **`cocotask ./config.json userworkers TestWorker 4`**\n you'll see the worker starts and process 1 message we just posted.\n\n6. So the key part is how we use cocotask command tool. the parameters are:\n - config_path: path to config file\n - module_name: module of your customized worker class\n - class_name: the class name of your own worker class (in the example above, it's userworkers.TestWorker, so the module name is userworkers, the class name is TestWorker)\n - number of workers: the total number of customer workers\n - logginglevel(optional): python logging level INFO/DEBUG/etc.\n - modulepath(optional): the relative path to find the module you defined. Default is '.'\n\n7. Develop your own worker class and try\n\n## Development\n\n**Build your own worker**\n\n```\nfrom cocotask import CocoBaseWorker\n\nclass TestWorker(CocoBaseWorker):\n\n def process(self, body):\n print(body)\n```\nCheck userworkers/test_worker.py for reference.\n\n**Post a message**\n```\nfrom cocotask import CocoProducerManager as pm\nimport json\n\nwith open('config.json', 'r') as f:\n config = json.load(f)\n\nproducer = pm.create_instance(config)\n\nproducer.connect()\nproducer.send('12345678')\nproducer.close()\n```\n**Switch from RabbitMQ to Kafka or reverse**\nIn test/config.json\n```\n{\n \"MQ_TYPE\": \"RMQ\", # change this to KAFKA if your underlying MQ is KAFKA\n\n \"RMQ\": {\n ...\n },\n\n \"KAFKA\": {\n ...\n },\n\n \"REDIS\": {\n ...\n }\n}\n\n```\n**We do support SASL_PLAINTEXT for kafka and simple auth in Redis as in the comments of the config file. Check their website to see how to setup the authentication**\n\nYou can build your own dictionary object as configuration for sure, as long as it contains the required fields.\n\n\nThat's it. Simple and straightforward.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/likexx/cocotask", "keywords": "task message queue MQ Pika kafka-python rabbitmq kafka redis rmq producer consumer celery", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "cocotask", "package_url": "https://pypi.org/project/cocotask/", "platform": "", "project_url": "https://pypi.org/project/cocotask/", "project_urls": { "Homepage": "https://github.com/likexx/cocotask" }, "release_url": "https://pypi.org/project/cocotask/0.2.1.1/", "requires_dist": [ "jsmin", "kafka-python", "pika", "redis" ], "requires_python": "", "summary": "An unified framework to build task queue on Rabbitmq | Kafka | Redis", "version": "0.2.1.1" }, "last_serial": 4516006, "releases": { "0.1.1.10": [ { "comment_text": "", "digests": { "md5": "02ca98fa4befe5a1d7b1f210dfe57c5c", "sha256": "d8098f5f8907ab4153cb5a8d669188a60f84a239241f6b8a432bd53f670aef39" }, "downloads": -1, "filename": "cocotask-0.1.1.10-py3-none-any.whl", "has_sig": false, "md5_digest": "02ca98fa4befe5a1d7b1f210dfe57c5c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 12015, "upload_time": "2018-09-20T15:20:58", "url": "https://files.pythonhosted.org/packages/7e/4c/1981d1113f7413052215f02868e75f3a8d70fbe9008f961a4a800499f6ac/cocotask-0.1.1.10-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "81d729e2c465c99454726c0882d73b3d", "sha256": "7b12aaa6be692243e1d0018dc8c3d2ba106a36f1e83aea0d49c8c02a4d0498a4" }, "downloads": -1, "filename": "cocotask-0.1.1.10.tar.gz", "has_sig": false, "md5_digest": "81d729e2c465c99454726c0882d73b3d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7263, "upload_time": "2018-09-20T15:20:59", "url": "https://files.pythonhosted.org/packages/de/be/f0858ce4177ddc1472ddecc31e39ed3ef9b95e6a62e90a2b6e3f9316af10/cocotask-0.1.1.10.tar.gz" } ], "0.1.1.11": [ { "comment_text": "", "digests": { "md5": "b3bdebd1da98e6996246ceb6e01ad0b2", "sha256": "1cba625f6329e25a6aee013bd591b38e3e5efbde14a30f3d592f544276e77b14" }, "downloads": -1, "filename": "cocotask-0.1.1.11-py3-none-any.whl", "has_sig": false, "md5_digest": "b3bdebd1da98e6996246ceb6e01ad0b2", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15323, "upload_time": "2018-10-16T12:12:01", "url": "https://files.pythonhosted.org/packages/28/53/7bfd07738665dad81ab74d15bfac85612af6e217314f2713d7a903cec368/cocotask-0.1.1.11-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b668ffe38f9d734b84ac5fbae8bc6b94", "sha256": "9767bdab6d724b688fd78555ec21f85dc0707020a186357677d076f160dcd72c" }, "downloads": -1, "filename": "cocotask-0.1.1.11.tar.gz", "has_sig": false, "md5_digest": "b668ffe38f9d734b84ac5fbae8bc6b94", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6984, "upload_time": "2018-10-16T12:12:05", "url": "https://files.pythonhosted.org/packages/ab/3f/efc1d32929d8128d91460410304306aeba0f9b947d5bbcff7c84b82bdc21/cocotask-0.1.1.11.tar.gz" } ], "0.1.1.12": [ { "comment_text": "", "digests": { "md5": "c280730327def91173a2bf6f688aa655", "sha256": "40b5e4d866b3148e22663d856cb44e4968548288bb1201483b1fd339fce9fa0c" }, "downloads": -1, "filename": "cocotask-0.1.1.12-py3-none-any.whl", "has_sig": false, "md5_digest": "c280730327def91173a2bf6f688aa655", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15399, "upload_time": "2018-10-16T12:21:29", "url": "https://files.pythonhosted.org/packages/94/17/955956df57dbcdd2d8f9d6e89bf5d6f225f467e75c821673ecbed5aa8b28/cocotask-0.1.1.12-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f618be42a726d9b0ad534c79682002c6", "sha256": "94f7418c5abf5fbd9920e7629d7f13d5e723d1df6f088ec58e972bf19194a8d3" }, "downloads": -1, "filename": "cocotask-0.1.1.12.tar.gz", "has_sig": false, "md5_digest": "f618be42a726d9b0ad534c79682002c6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7013, "upload_time": "2018-10-16T12:21:30", "url": "https://files.pythonhosted.org/packages/a2/91/61f54202e20cd047f8b558303e295ba959afbdc5af5f4a26a7bb05b349b1/cocotask-0.1.1.12.tar.gz" } ], "0.1.1.6": [ { "comment_text": "", "digests": { "md5": "92add714090000e2189fc3cb2794fa40", "sha256": "d1cae9b4101f3fb032490fdca8cf88bd87a1e538763040dd9f1b81669b508faf" }, "downloads": -1, "filename": "cocotask-0.1.1.6-py3-none-any.whl", "has_sig": false, "md5_digest": "92add714090000e2189fc3cb2794fa40", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 14274, "upload_time": "2018-05-24T08:17:40", "url": "https://files.pythonhosted.org/packages/73/3a/76aa9253b5f6eb2ab7b18d6e5c870bc97d9d3a27e31ff202c72577eb7e4b/cocotask-0.1.1.6-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "78e8105a100f4c130e4064ccaedce3ee", "sha256": "0a6523163f2c47a7f9b62a0b105f43117e96c5f6bd58ba60c765a3a94dabd8e9" }, "downloads": -1, "filename": "cocotask-0.1.1.6.tar.gz", "has_sig": false, "md5_digest": "78e8105a100f4c130e4064ccaedce3ee", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6235, "upload_time": "2018-05-24T08:17:42", "url": "https://files.pythonhosted.org/packages/e5/de/da5acf8aa4bc3d7c22fe1a9c6fdce7fcfbd3ef859175cfa110cec43b660a/cocotask-0.1.1.6.tar.gz" } ], "0.1.1.7": [ { "comment_text": "", "digests": { "md5": "074c5892a12579004584a80ae93fa6ec", "sha256": "61fc1df152347bfa53b3c55e1ce6be43a8b5b1cc3bfeee878bdc43385f54a71e" }, "downloads": -1, "filename": "cocotask-0.1.1.7-py3-none-any.whl", "has_sig": false, "md5_digest": "074c5892a12579004584a80ae93fa6ec", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 11927, "upload_time": "2018-05-28T17:33:02", "url": "https://files.pythonhosted.org/packages/79/aa/d44cd3f0ce3f4ffc929f9e4d06059d49fe029976009dcd1e124e5b38927d/cocotask-0.1.1.7-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "d183038ae08aeb8f59da3708bcfcf960", "sha256": "9bca7466d9e6003ed2eaccaa88848f7a8bc96dc7a0f5c28f1aced3513f4b0e39" }, "downloads": -1, "filename": "cocotask-0.1.1.7.tar.gz", "has_sig": false, "md5_digest": "d183038ae08aeb8f59da3708bcfcf960", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7199, "upload_time": "2018-05-28T17:33:04", "url": "https://files.pythonhosted.org/packages/8c/18/8cbdc2f97b9fcf516ca30c88ad56827198bc081788bdb43d9100741f8cdb/cocotask-0.1.1.7.tar.gz" } ], "0.1.1.8": [ { "comment_text": "", "digests": { "md5": "dd87adce5ecef1aabdef1f6282ce8ee0", "sha256": "9c3c0e1ad7bf660f0bc4c644427b43a80f6888606ea0c77108ab0f7edf870c44" }, "downloads": -1, "filename": "cocotask-0.1.1.8-py3-none-any.whl", "has_sig": false, "md5_digest": "dd87adce5ecef1aabdef1f6282ce8ee0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 14468, "upload_time": "2018-07-03T03:08:00", "url": "https://files.pythonhosted.org/packages/e2/5f/ba5e1c1be5c986b57db655fd78d71d1a4d21d0311dd525f934c2cb8e8aed/cocotask-0.1.1.8-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "6ffc5c75a7e62d1577d51f9bc7107ab0", "sha256": "9e14af57224d9681c14c159424c6ffe98c2f6a7d1ced0125b8c603ecd1048e4b" }, "downloads": -1, "filename": "cocotask-0.1.1.8.tar.gz", "has_sig": false, "md5_digest": "6ffc5c75a7e62d1577d51f9bc7107ab0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6296, "upload_time": "2018-07-03T03:08:03", "url": "https://files.pythonhosted.org/packages/0d/ff/044611f8e466c391a19a4eddcc182e608c08dba8326acae8b8c403bb940f/cocotask-0.1.1.8.tar.gz" } ], "0.1.1.9": [ { "comment_text": "", "digests": { "md5": "9bbd5f6312de1d656b38652245f6b83f", "sha256": "7e51dd6b00a0345799c2a3a1882ea8cd3972a99bf4a852761c64c7bb031512b6" }, "downloads": -1, "filename": "cocotask-0.1.1.9-py3-none-any.whl", "has_sig": false, "md5_digest": "9bbd5f6312de1d656b38652245f6b83f", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 12014, "upload_time": "2018-09-20T09:32:27", "url": "https://files.pythonhosted.org/packages/a5/28/4877b41646ab34b11a17f26fb74ac98567a2e74ca03a57201dabf901173e/cocotask-0.1.1.9-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "5bc1fe681689218634b6edddd64927a7", "sha256": "7a7804edb53b2762cd1d4c7b57d11c35699cae8ad2556145cbb899960bb35fbf" }, "downloads": -1, "filename": "cocotask-0.1.1.9.tar.gz", "has_sig": false, "md5_digest": "5bc1fe681689218634b6edddd64927a7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7280, "upload_time": "2018-09-20T09:32:29", "url": "https://files.pythonhosted.org/packages/0b/23/5a457ccf475de4e262cc5a43d3de7b806f486289686ab08aee8777e81419/cocotask-0.1.1.9.tar.gz" } ], "0.2.0.0": [ { "comment_text": "", "digests": { "md5": "b92da3eb7752e59c884443e0b96ea4f7", "sha256": "34ac702ae5fe3ad2e8f7de9cd37a1de8559055021e2aa01f9fe58a51f4c99855" }, "downloads": -1, "filename": "cocotask-0.2.0.0-py3-none-any.whl", "has_sig": false, "md5_digest": "b92da3eb7752e59c884443e0b96ea4f7", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15494, "upload_time": "2018-11-05T08:13:28", "url": "https://files.pythonhosted.org/packages/8b/1c/9a6c736e7eac909eeb2e424ab3b02e98029fdd718efd215c92a3f079fc8d/cocotask-0.2.0.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0b5e33b05b8eb6cdcd19fd1e1251ca82", "sha256": "5871973d3e7ae771f1770bcc0b81ef76fec1cec84fab0aafa070672fad85d103" }, "downloads": -1, "filename": "cocotask-0.2.0.0.tar.gz", "has_sig": false, "md5_digest": "0b5e33b05b8eb6cdcd19fd1e1251ca82", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7062, "upload_time": "2018-11-05T08:13:30", "url": "https://files.pythonhosted.org/packages/bf/2b/7e517acfa455c3ce2a7c0b49a2dee6f292912a3a7130383a7cdf8acccf15/cocotask-0.2.0.0.tar.gz" } ], "0.2.1.0": [ { "comment_text": "", "digests": { "md5": "085ec3bd88a01cae36add762c9ad8957", "sha256": "fdbdfa53fcae3d37902d490592dc5e161cd11f54b919754ec6371489dbc3412f" }, "downloads": -1, "filename": "cocotask-0.2.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "085ec3bd88a01cae36add762c9ad8957", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15628, "upload_time": "2018-11-22T04:12:19", "url": "https://files.pythonhosted.org/packages/ce/20/eff58871594c76be9b92ebee2ffdbcd358a189e087d4859099c7d28bc2ea/cocotask-0.2.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "96735411b1b0ec276eac1c9860a2ce83", "sha256": "818d1a35e4f3480c2f3e8a216db0259db8122ccd49652980e5323daad68fc251" }, "downloads": -1, "filename": "cocotask-0.2.1.0.tar.gz", "has_sig": false, "md5_digest": "96735411b1b0ec276eac1c9860a2ce83", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8857, "upload_time": "2018-11-22T04:12:21", "url": "https://files.pythonhosted.org/packages/40/96/3b00697f9b228c459337d8daaa216c6ab9dd7fd50bfe448801ea9b5ea728/cocotask-0.2.1.0.tar.gz" } ], "0.2.1.1": [ { "comment_text": "", "digests": { "md5": "1610509831cc57d5199ea3d08c3e0e04", "sha256": "c2cc64078adae8f2544cb893ee52c00ff701450062153463a2e070c0f844179c" }, "downloads": -1, "filename": "cocotask-0.2.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "1610509831cc57d5199ea3d08c3e0e04", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15626, "upload_time": "2018-11-22T09:06:52", "url": "https://files.pythonhosted.org/packages/be/be/e5a5af5cc001f25b2842ea66c8ab6c1fccefe7d289fa4556df92ecc2fc02/cocotask-0.2.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c707e6b2f232e4b45c042e701a55303c", "sha256": "cb05424cc3231b7ef54bfe4d9937ba70bd4d01627c9eb659836ff4943fc1ddd2" }, "downloads": -1, "filename": "cocotask-0.2.1.1.tar.gz", "has_sig": false, "md5_digest": "c707e6b2f232e4b45c042e701a55303c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8850, "upload_time": "2018-11-22T09:06:53", "url": "https://files.pythonhosted.org/packages/0e/fd/30e0a729eb7290db04d3d3750a855321058cc137b8b643e165578c135e8d/cocotask-0.2.1.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1610509831cc57d5199ea3d08c3e0e04", "sha256": "c2cc64078adae8f2544cb893ee52c00ff701450062153463a2e070c0f844179c" }, "downloads": -1, "filename": "cocotask-0.2.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "1610509831cc57d5199ea3d08c3e0e04", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15626, "upload_time": "2018-11-22T09:06:52", "url": "https://files.pythonhosted.org/packages/be/be/e5a5af5cc001f25b2842ea66c8ab6c1fccefe7d289fa4556df92ecc2fc02/cocotask-0.2.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c707e6b2f232e4b45c042e701a55303c", "sha256": "cb05424cc3231b7ef54bfe4d9937ba70bd4d01627c9eb659836ff4943fc1ddd2" }, "downloads": -1, "filename": "cocotask-0.2.1.1.tar.gz", "has_sig": false, "md5_digest": "c707e6b2f232e4b45c042e701a55303c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8850, "upload_time": "2018-11-22T09:06:53", "url": "https://files.pythonhosted.org/packages/0e/fd/30e0a729eb7290db04d3d3750a855321058cc137b8b643e165578c135e8d/cocotask-0.2.1.1.tar.gz" } ] }