{
    "info": {
        "author": "Jay Johnson",
        "author_email": "jay.p.h.johnson@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: Apache Software License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: Implementation :: PyPy",
            "Topic :: Software Development :: Libraries :: Python Modules"
        ],
        "description": "Celery Connectors\n=================\n\nCelery_ is a great framework for processing messages from a message queue broker like Redis or RabbitMQ. If you have a queue with json or pickled messages that you need to consume and process, then hopefully this repository will help you out.\n\nIt has multiple examples on setting up working publisher-subscriber messaging workflows using Celery, Celery Bootsteps, Kombu, and Kombu mixins. These examples are focused on finding a starting ground to tune for high availability + performance + reduce the risk of message loss (the dockerized celery bootstep rabbitmq subscriber can process around **100,000 messages in 90 seconds** with 3 workers). By using the included docker containers combined with the included load tests, you can start to vet your solution won't wake you up in the middle of the night during an outage.\n\nEach example below can run as a docker container with the included docker-compose files in the `compose directory`_. Please note these docker-compose steps are optional and the consumer counts in the documentation below will only refer to the non-dockerized, repository versions.\n\nHere's the JSON-to-Celery ecomm relay example in action. By using docker-compose you can use container monitoring tools to benchmark resources and throughput to figure out your deployment footprint and address bottlenecks. \n\n.. image:: https://raw.githubusercontent.com/jay-johnson/celery-connectors/master/_images/celery-connectors-json-to-celery-relay-with-existing-ecomm-celery-app.gif\n    :align: center\n\n.. _Celery: http://docs.celeryproject.org/en/latest/\n.. _compose directory: https://github.com/jay-johnson/celery-connectors/tree/master/compose\n\nWhy do I care?\n--------------\n\n- Do you want to read json or pickled messages out of a queue and have a framework handle the scaling and deployment aspects all out of the box? \n\n- Do you want a simple way to read out of queues without setting up a task result backend (mongo)?\n\n- Do you want to connect a windows python client to a backend linux system or cluster?\n\n- Do you want to communicate with all your AWS VPC backends over SQS?\n\n- Do you want to glue python and non-python technologies together through a message queue backend?\n\n- Do you want something that works with python 2 and 3?\n\nHow do I get started?\n---------------------\n\n#.  Setup the virtualenv \n\n    If you want to use python 2:\n\n    ::\n\n        virtualenv venv && source venv/bin/activate && pip install celery-connectors\n\n    If you want to use python 3:\n\n    ::\n\n        virtualenv -p python3 venv && source venv/bin/activate && pip install celery-connectors\n\n#.  Confirm the pip is installed\n\n    ::\n\n        pip list | grep celery-connectors\n\n#.  Start the containers\n\n    ::\n\n        # if you do not have docker compose installed, you can try installing it with:\n        # pip install docker-compose\n        start-redis-and-rabbitmq.sh\n\n    Or if your docker version and OS support container volume-mounting, then you can persist Redis and RabbitMQ messages and data to disk with:\n\n    ::\n\n        ./start-persistence-containers.sh\n\n#.  Check the Redis and RabbitMQ containers are running\n\n    ::\n\n        docker ps\n        CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS                                                                                                       NAMES\n        913e8092dbde        mher/flower:latest          \"/usr/local/bin/py...\"   35 seconds ago      Up 35 seconds                                                                                                                   celflowerredis\n        b6983a1316ba        rabbitmq:3.6.6-management   \"docker-entrypoint...\"   35 seconds ago      Up 34 seconds       4369/tcp, 5671/tcp, 0.0.0.0:5672->5672/tcp, 0.0.0.0:15672->15672/tcp, 15671/tcp, 0.0.0.0:25672->25672/tcp   celrabbit1\n        52cb4c511d61        redis:4.0.5-alpine          \"docker-entrypoint...\"   35 seconds ago      Up 34 seconds       0.0.0.0:6379->6379/tcp, 0.0.0.0:16379->16379/tcp                                                            celredis1\n        202bdaf70784        mher/flower:latest          \"/usr/local/bin/py...\"   35 seconds ago      Up 35 seconds                                                                                                                   celflowerrabbit\n\nRunning a Payments JSON-to-JSON Relay Service\n=============================================\n\nThis will simulate a json->json relay using kombu mixins:\n\nhttp://docs.celeryproject.org/projects/kombu/en/latest/reference/kombu.mixins.html\n\nKombu mixins are a great way to process messages without Celery, and they are resilient to multiple HA scenarios including a complete broker failures. While building this I would load up messages to process, simulate lag before an ``ack`` and then start/stop the RabbitMQ docker container to see how things reacted. As long as the subscribers can ``declare`` their consuming queues on a fresh broker start-up case, these mixins seem capable of surviving these types of DR events. By default these builds are going to only read one message out of the queue at a time.\n\nStart JSON Relay\n----------------\n\nThis process will consume JSON dictionary messages on the ``ecomm.api.west`` RabbitMQ queue and pass the message to the ``reporting.payments`` queue.\n\nPlease start this in a new terminal that has sourced the virtual env: ``source venv/bin/activate``\n\n::\n\n    start-mixin-json-relay.py \n    INFO:mixin_relay:Consuming queues=1\n    INFO:relay:consuming queues=[<unbound Queue ecomm.api.west -> <unbound Exchange ecomm.api(topic)> -> ecomm.api.west>]\n    INFO:kombu.mixins:Connected to amqp://rabbitmq:**@127.0.0.1:5672//\n    INFO:relay-wrk:creating consumer for queues=1 callback=handle_message relay_ex=Exchange ''(direct) relay_rk=reporting.payments prefetch=1\n\nOr with docker compose\n\n::\n\n    docker-compose -f compose-start-mixin-json-relay.yml up\n    Starting jtojrelay ... \n    Starting jtojrelay ... done\n    Attaching to jtojrelay\n    jtojrelay    | 2017-12-15 06:37:39,458 - jtoj_relay - INFO - Consuming queues=1\n    jtojrelay    | 2017-12-15 06:37:39,462 - jtoj_relay - INFO - consuming queues=[<unbound Queue ecomm.api.west -> <unbound Exchange \"\"(topic)> -> ecomm.api.west>]\n    jtojrelay    | 2017-12-15 06:37:39,478 - kombu.mixins - INFO - Connected to amqp://rabbitmq:**@127.0.0.1:5672//\n\nList the Queues\n---------------\n\nIn a new terminal that has the virtual env loaded, checkout the RabbitMQ queues:\n\n::\n\n    list-queues.sh \n\n    Listing Queues broker=localhost:15672\n\n+-----------------------------------------------+-----------+----------+----------------+-------------------------+\n|                     name                      | consumers | messages | messages_ready | messages_unacknowledged |\n+-----------------------------------------------+-----------+----------+----------------+-------------------------+\n| celeryev.ea44162e-7224-4167-be30-4be614c33fc9 | 1         | 0        | 0              | 0                       |\n+-----------------------------------------------+-----------+----------+----------------+-------------------------+\n| ecomm.api.west                                | 1         | 0        | 0              | 0                       |\n+-----------------------------------------------+-----------+----------+----------------+-------------------------+\n\nStart the Kombu Mixin Subscriber\n--------------------------------\n\nIn a new terminal that has the virtual env loaded, start the subscriber for relayed messags in the ``reporting.payments`` queue:\n\n::\n\n    kombu_mixin_subscriber.py \n    INFO:kombu-mixin-subscriber:Start - kombu-mixin-subscriber\n    INFO:kombu-subscriber:setup routing\n    INFO:kombu-subscriber:kombu-mixin-subscriber - kombu.subscriber queues=reporting.payments consuming with callback=handle_message\n\nOr with docker compose:\n\n::\n\n    docker-compose -f compose-kombu-mixin-subscriber.yml  up\n    WARNING: Found orphan containers (jtojrelay) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.\n    Creating kombumixinsubrmq ... done\n    Attaching to kombumixinsubrmq\n    kombumixinsubrmq    | 2017-12-15 06:41:15,135 - kombu-mixin-subscriber - INFO - Start - kombu-mixin-subscriber\n    kombumixinsubrmq    | 2017-12-15 06:41:15,135 - kombu-subscriber - INFO - setup routing\n\nList the Bindings\n-----------------\n\nWith the relay and the subscrbier online the bindings should show two separate queues for these two processes.\n\n::\n\n    list-bindings.sh \n\n    Listing Bindings broker=localhost:15672\n\n+--------------------+-----------------------------------------------+-----------------------------------------------+\n|       source       |                  destination                  |                  routing_key                  |\n+--------------------+-----------------------------------------------+-----------------------------------------------+\n|                    | celeryev.ea44162e-7224-4167-be30-4be614c33fc9 | celeryev.ea44162e-7224-4167-be30-4be614c33fc9 |\n+--------------------+-----------------------------------------------+-----------------------------------------------+\n|                    | ecomm.api.west                                | ecomm.api.west                                |\n+--------------------+-----------------------------------------------+-----------------------------------------------+\n|                    | reporting.payments                            | reporting.payments                            |\n+--------------------+-----------------------------------------------+-----------------------------------------------+\n| celeryev           | celeryev.ea44162e-7224-4167-be30-4be614c33fc9 | #                                             |\n+--------------------+-----------------------------------------------+-----------------------------------------------+\n| ecomm.api          | ecomm.api.west                                | ecomm.api.west                                |\n+--------------------+-----------------------------------------------+-----------------------------------------------+\n| reporting.payments | reporting.payments                            | reporting.payments                            |\n+--------------------+-----------------------------------------------+-----------------------------------------------+\n\nPublish Ecomm messages to the Relay\n-----------------------------------\n\nIn a new terminal that has the virtual env loaded, start the mixin publisher that will send JSON messages to the ``ecomm.api.west`` queue:\n\n::\n\n    start-mixin-publisher.py \n    INFO:robopub:Generating messages=10\n    INFO:robopub:Publishing messages=10\n    INFO:pub_send:pub_send publish - ex=Exchange ecomm.api(topic) rk=ecomm.api.west sz=json\n    INFO:pub_send:pub_send publish - ex=Exchange ecomm.api(topic) rk=ecomm.api.west sz=json\n    INFO:pub_send:pub_send publish - ex=Exchange ecomm.api(topic) rk=ecomm.api.west sz=json\n    INFO:pub_send:pub_send publish - ex=Exchange ecomm.api(topic) rk=ecomm.api.west sz=json\n    INFO:pub_send:pub_send publish - ex=Exchange ecomm.api(topic) rk=ecomm.api.west sz=json\n    INFO:pub_send:pub_send publish - ex=Exchange ecomm.api(topic) rk=ecomm.api.west sz=json\n    INFO:pub_send:pub_send publish - ex=Exchange ecomm.api(topic) rk=ecomm.api.west sz=json\n    INFO:pub_send:pub_send publish - ex=Exchange ecomm.api(topic) rk=ecomm.api.west sz=json\n    INFO:pub_send:pub_send publish - ex=Exchange ecomm.api(topic) rk=ecomm.api.west sz=json\n    INFO:pub_send:pub_send publish - ex=Exchange ecomm.api(topic) rk=ecomm.api.west sz=json\n    INFO:robopub:Done Publishing\n\nOr with docker compose:\n\n::\n\n    docker-compose -f compose-start-mixin-publisher.yml up\n    WARNING: Found orphan containers (kombumixinsubrmq, jtojrelay) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.\n    Starting mixinpubrmq ... done\n    Attaching to mixinpubrmq\n\nVerify the Relay Handled the Messages\n-------------------------------------\n\nVerify the terminal logs in the relay look similar to:\n\n::\n\n    INFO:relay-wrk:default handle_message - acking - msg={'data': {'simulated_lag': 1.0}, 'msg_id': '35e8546f-f757-4764-9a25-12b867f61957_1', 'created': '2017-12-13T01:30:35.401399'}\n    INFO:relay-wrk:send start - relay_ex=Exchange ''(direct) relay_rk=reporting.payments id=95c93115-2041-424b-b37e-0e8dff1b6336_1\n    INFO:pub_send:pub_send publish - ex=Exchange ''(direct) rk=reporting.payments sz=json\n    INFO:relay-wrk:send done - id=95c93115-2041-424b-b37e-0e8dff1b6336_1\n    INFO:relay-wrk:default handle_message - acking - msg={'data': {'simulated_lag': 1.0}, 'msg_id': '989641cc-cd2b-4041-81aa-bdd27393646a_1', 'created': '2017-12-13T01:30:35.401529'}\n    INFO:relay-wrk:send start - relay_ex=Exchange ''(direct) relay_rk=reporting.payments id=7d8b473a-1f7e-4d04-8e8a-234536b0a8fb_1\n    INFO:pub_send:pub_send publish - ex=Exchange ''(direct) rk=reporting.payments sz=json\n    INFO:relay-wrk:send done - id=7d8b473a-1f7e-4d04-8e8a-234536b0a8fb_1\n    INFO:relay-wrk:default handle_message - acking - msg={'data': {'simulated_lag': 1.0}, 'msg_id': '68eb6ab0-2e41-4838-a088-927709c4d595_1', 'created': '2017-12-13T01:30:35.401554'}\n    INFO:relay-wrk:send start - relay_ex=Exchange ''(direct) relay_rk=reporting.payments id=4ca34760-db69-4c06-97c9-0355c38bd158_1\n    INFO:pub_send:pub_send publish - ex=Exchange ''(direct) rk=reporting.payments sz=json\n    INFO:relay-wrk:send done - id=4ca34760-db69-4c06-97c9-0355c38bd158_1\n    INFO:relay-wrk:default handle_message - acking - msg={'data': {'simulated_lag': 1.0}, 'msg_id': 'f906ab52-27f1-4ea7-bd68-2956da232258_1', 'created': '2017-12-13T01:30:35.401618'}\n    INFO:relay-wrk:send start - relay_ex=Exchange ''(direct) relay_rk=reporting.payments id=8a584a99-b35d-4e18-acd8-45d32871ba0a_1\n\nVerify the Subscriber Handled the Relayed Messages\n--------------------------------------------------\n\n::\n\n    INFO:kombu-mixin-subscriber:callback received msg body={'msg_id': '95c93115-2041-424b-b37e-0e8dff1b6336_1', 'data': {'org_msg': {'msg_id': '35e8546f-f757-4764-9a25-12b867f61957_1', 'data': {'simulated_lag': 1.0}, 'created': '2017-12-13T01:30:35.401399'}, 'relay_name': 'json-to-json-relay'}, 'created': '2017-12-13T01:30:35.423314'}\n    INFO:kombu-subscriber:kombu-mixin-subscriber - kombu.subscriber queues=reporting.payments consuming with callback=handle_message\n    INFO:kombu-mixin-subscriber:callback received msg body={'msg_id': '7d8b473a-1f7e-4d04-8e8a-234536b0a8fb_1', 'data': {'org_msg': {'msg_id': '989641cc-cd2b-4041-81aa-bdd27393646a_1', 'data': {'simulated_lag': 1.0}, 'created': '2017-12-13T01:30:35.401529'}, 'relay_name': 'json-to-json-relay'}, 'created': '2017-12-13T01:30:35.445645'}\n    INFO:kombu-subscriber:kombu-mixin-subscriber - kombu.subscriber queues=reporting.payments consuming with callback=handle_message\n    INFO:kombu-mixin-subscriber:callback received msg body={'msg_id': '4ca34760-db69-4c06-97c9-0355c38bd158_1', 'data': {'org_msg': {'msg_id': '68eb6ab0-2e41-4838-a088-927709c4d595_1', 'data': {'simulated_lag': 1.0}, 'created': '2017-12-13T01:30:35.401554'}, 'relay_name': 'json-to-json-relay'}, 'created': '2017-12-13T01:30:35.453077'}\n    INFO:kombu-subscriber:kombu-mixin-subscriber - kombu.subscriber queues=reporting.payments consuming with callback=handle_message\n    INFO:kombu-mixin-subscriber:callback received msg body={'msg_id': '8a584a99-b35d-4e18-acd8-45d32871ba0a_1', 'data': {'org_msg': {'msg_id': 'f906ab52-27f1-4ea7-bd68-2956da232258_1', 'data': {'simulated_lag': 1.0}, 'created': '2017-12-13T01:30:35.401618'}, 'relay_name': 'json-to-json-relay'}, 'created': '2017-12-13T01:30:35.458601'}\n\nConfirm the Queues are empty\n----------------------------\n\n::\n\n    list-queues.sh \n\n    Listing Queues broker=localhost:15672\n\n+-----------------------------------------------+-----------+----------+----------------+-------------------------+\n|                     name                      | consumers | messages | messages_ready | messages_unacknowledged |\n+-----------------------------------------------+-----------+----------+----------------+-------------------------+\n| celeryev.ea44162e-7224-4167-be30-4be614c33fc9 | 1         | 0        | 0              | 0                       |\n+-----------------------------------------------+-----------+----------+----------------+-------------------------+\n| ecomm.api.west                                | 1         | 0        | 0              | 0                       |\n+-----------------------------------------------+-----------+----------+----------------+-------------------------+\n| reporting.payments                            | 1         | 0        | 0              | 0                       |\n+-----------------------------------------------+-----------+----------+----------------+-------------------------+\n\nStop the the JSON Relay Demo\n-----------------------------\n\nIn the mixin relay and mixin subscriber terminal sessions use: ``ctrl + c`` to stop the processes.\n\nRestart the docker containers to a good, clean state.\n\nStop:\n\n::\n\n    stop-redis-and-rabbitmq.sh \n    Stopping redis and rabbitmq\n    Stopping celrabbit1      ... done\n    Stopping celredis1       ... done\n    Stopping celflowerredis  ... done\n    Stopping celflowerrabbit ... done\n\nStart:\n\n::\n\n    start-redis-and-rabbitmq.sh \n    Starting redis and rabbitmq\n    Creating celrabbit1 ... done\n    Creating celrabbit1 ... done\n    Creating celredis1 ... done\n    Creating celflowerredis ... done\n\nRunning an Ecommerce JSON-to-Celery Relay Service\n=================================================\n\nThis will simulate hooking up an existing Celery application to start processing Celery tasks from JSON messages in a RabbitMQ queue. This is useful because it allows reusing existing Celery application tasks over a JSON messaging layer for mapping payloads to specific, existing Celery tasks. With this approach you can glue python and non-python services together provided that they can publish JSON messages to Redis, RabbitMQ or AWS SQS (please refer to the `fix SQS`_ section). Each of the components below can scale horizontally for redundancy. Each one also utilizes native RabbitMQ acks (https://www.rabbitmq.com/confirms.html) to ensure messages are never deleted or lost until propagation to the next queue or component has been confirmed.\n\n.. _fix SQS: https://github.com/jay-johnson/celery-connectors#temporary-fix-for-kombu-sqs\n\nNote: Please run this demo with three separate terminal sessions and a browser to view the Celery application's task progress in Flower.\n\nStart Ecommerce Celery Worker\n-----------------------------\n\nStart a Celery worker for an existing ecommerce application from a hypothetical Django or Flask server.\n\nNote: Please run this from the base directory for the repository and source the virtual env: ``source venv/bin/activate``\n\n::\n\n    ./start-ecomm-worker.sh\n\n    -------------- celery@ecommerce_subscriber v4.1.0 (latentcall)\n    ---- **** ----- \n    --- * ***  * -- Linux-4.7.4-200.fc24.x86_64-x86_64-with-fedora-24-Twenty_Four 2017-12-14 00:33:02\n    -- * - **** --- \n    - ** ---------- [config]\n    - ** ---------- .> app:         ecommerce-worker:0x7f0c23f1c550\n    - ** ---------- .> transport:   amqp://rabbitmq:**@localhost:5672//\n    - ** ---------- .> results:     redis://localhost:6379/10\n    - *** --- * --- .> concurrency: 3 (prefork)\n    -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)\n    --- ***** ----- \n    -------------- [queues]\n                    .> celery           exchange=celery(direct) key=celery\n\n\n    [tasks]\n    . ecomm_app.ecommerce.tasks.handle_user_conversion_events\n\n    [2017-12-14 00:33:02,243: INFO/MainProcess] Connected to amqp://rabbitmq:**@127.0.0.1:5672//\n    [2017-12-14 00:33:02,260: INFO/MainProcess] mingle: searching for neighbors\n    [2017-12-14 00:33:03,293: INFO/MainProcess] mingle: all alone\n    [2017-12-14 00:33:03,337: INFO/MainProcess] celery@ecommerce_subscriber ready.\n    [2017-12-14 00:33:05,275: INFO/MainProcess] Events of group {task} enabled by remote.\n\nOr with docker compose:\n\n::\n\n    docker-compose -f compose-start-ecomm-worker.yml up\n    Recreating ecommworker ... done\n    Attaching to ecommworker\n\nNotice the worker is named ``celery@ecommerce_subscriber`` this is the identifier for viewing the Celery application in Flower:\n\nhttp://localhost:5555/worker/celery@ecommerce_subscriber (login: admin/admin)\n\nStart Ecomm Relay\n-----------------\n\nThis process will consume JSON dictionary messages on the ``ecomm.api.west`` RabbitMQ queue and pass the message to the ecomm Celery app as a ``ecomm_app.ecommerce.tasks.handle_user_conversion_events`` Celery task.\n\nPlease start this in a new terminal that has sourced the virtual env: ``source venv/bin/activate``\n\n::\n\n    ./start-mixin-celery-relay.py \n    2017-12-14 00:36:47,339 - jtoc_relay - INFO - Consuming queues=1\n    2017-12-14 00:36:47,342 - jtoc - INFO - consuming queues=[<unbound Queue ecomm.api.west -> <unbound Exchange ecomm.api(topic)> -> ecomm.api.west>]\n    2017-12-14 00:36:47,353 - kombu.mixins - INFO - Connected to amqp://rabbitmq:**@127.0.0.1:5672//\n    2017-12-14 00:36:47,355 - jtoc - INFO - creating consumer for queues=1 callback=handle_message relay_ex=Exchange ''(direct) relay_rk=reporting.payments prefetch=1\n\nOr with docker compose:\n\n::\n\n    docker-compose -f compose-start-mixin-celery-relay.yml up\n    Creating jtocrelay ... done\n    Attaching to jtocrelay\n    jtocrelay    | 2017-12-15 06:56:07,689 - jtoc_relay - INFO - Consuming queues=1\n    jtocrelay    | 2017-12-15 06:56:07,703 - jtoc_relay - INFO - consuming queues=[<unbound Queue ecomm.api.west -> <unbound Exchange ecomm.api(topic)> -> ecomm.api.west>]\n    jtocrelay    | 2017-12-15 06:56:07,720 - kombu.mixins - INFO - Connected to amqp://rabbitmq:**@127.0.0.1:5672//\n\nPublish a User Conversion Event to the Ecomm Relay\n--------------------------------------------------\n\nThis will use Kombu to publish a JSON dictionary message to the ``ecomm.api.west`` RabbitMQ queue which is monitored by the mixin JSON to Celery relay. This test tool is configured to simulate hypothetical worst-cast lag during the relay + message processing. This is a functional test to ensure everything stays connected and ready for more messages to process.\n\nPlease start this in a new terminal that has sourced the virtual env: ``source venv/bin/activate``\n\n::\n\n    ./start-mixin-publisher.py \n    2017-12-14 00:42:16,849 - robopub - INFO - Generating messages=10\n    2017-12-14 00:42:16,850 - robopub - INFO - Publishing messages=10\n    2017-12-14 00:42:16,866 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=46cb24f0d0_1\n    2017-12-14 00:42:16,867 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=d2724b75fa_1\n    2017-12-14 00:42:16,867 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=e72e09da34_1\n    2017-12-14 00:42:16,869 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=f5ec3f0c9d_1\n    2017-12-14 00:42:16,870 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=222094db10_1\n    2017-12-14 00:42:16,871 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=9bed4cc0e5_1\n    2017-12-14 00:42:16,871 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=f66139a9cf_1\n    2017-12-14 00:42:16,872 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=94d3a2c7ed_1\n    2017-12-14 00:42:16,873 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=b517f87ff4_1\n    2017-12-14 00:42:16,873 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=822ef4142c_1\n    2017-12-14 00:42:16,874 - robopub - INFO - Done Publishing\n\n\nOr with docker compose:\n\n::\n\n    docker-compose -f compose-start-mixin-publisher.yml up\n    WARNING: Found orphan containers (jtocrelay) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.\n    Recreating mixinpubrmq ... done\n    Attaching to mixinpubrmq\n    mixinpubrmq    | 2017-12-15 06:56:43,517 - robopub - INFO - Generating messages=10\n\nVerify the Ecomm Relay Processed the Conversion Message\n-------------------------------------------------------\n\nAfter the simulated lag finishes, the logs for the ecomm relay should show something similar to:\n\n::\n\n    2017-12-14 00:42:16,869 - jtoc - INFO - hd msg=46cb24f0d0_1 from_ex=ecomm.api from_rk=ecomm.api.west\n    2017-12-14 00:42:16,870 - jtoc - INFO - relay msg_id=46cb24f0d0_1 body={'msg_id': '46cb24f0d0_1', 've\n    2017-12-14 00:42:16,937 - jtoc - INFO - relay done with msg_id=46cb24f0d0_1\n    2017-12-14 00:42:16,937 - jtoc - INFO - task - ecomm_app.ecommerce.tasks.handle_user_conversion_events - simulating processing lag sleep=8.0 seconds\n    2017-12-14 00:42:24,947 - jtoc - INFO - hd msg=d2724b75fa_1 from_ex=ecomm.api from_rk=ecomm.api.west\n    2017-12-14 00:42:24,947 - jtoc - INFO - relay msg_id=d2724b75fa_1 body={'msg_id': 'd2724b75fa_1', 've\n    2017-12-14 00:42:24,953 - jtoc - INFO - relay done with msg_id=d2724b75fa_1\n    2017-12-14 00:42:24,953 - jtoc - INFO - task - ecomm_app.ecommerce.tasks.handle_user_conversion_events - simulating processing lag sleep=8.0 seconds\n    2017-12-14 00:42:32,962 - jtoc - INFO - hd msg=e72e09da34_1 from_ex=ecomm.api from_rk=ecomm.api.west\n    2017-12-14 00:42:32,963 - jtoc - INFO - relay msg_id=e72e09da34_1 body={'msg_id': 'e72e09da34_1', 've\n    2017-12-14 00:42:32,968 - jtoc - INFO - relay done with msg_id=e72e09da34_1\n    2017-12-14 00:42:32,968 - jtoc - INFO - task - ecomm_app.ecommerce.tasks.handle_user_conversion_events - simulating processing lag sleep=8.0 seconds\n    2017-12-14 00:42:40,982 - jtoc - INFO - hd msg=f5ec3f0c9d_1 from_ex=ecomm.api from_rk=ecomm.api.west\n    2017-12-14 00:42:40,983 - jtoc - INFO - relay msg_id=f5ec3f0c9d_1 body={'msg_id': 'f5ec3f0c9d_1', 've\n    2017-12-14 00:42:41,005 - jtoc - INFO - relay done with msg_id=f5ec3f0c9d_1\n    2017-12-14 00:42:41,006 - jtoc - INFO - task - ecomm_app.ecommerce.tasks.handle_user_conversion_events - simulating processing lag sleep=8.0 seconds\n    2017-12-14 00:42:49,014 - jtoc - INFO - hd msg=222094db10_1 from_ex=ecomm.api from_rk=ecomm.api.west\n    2017-12-14 00:42:49,015 - jtoc - INFO - relay msg_id=222094db10_1 body={'msg_id': '222094db10_1', 've\n    2017-12-14 00:42:49,024 - jtoc - INFO - relay done with msg_id=222094db10_1\n    2017-12-14 00:42:49,024 - jtoc - INFO - task - ecomm_app.ecommerce.tasks.handle_user_conversion_events - simulating processing lag sleep=8.0 seconds\n    2017-12-14 00:42:57,034 - jtoc - INFO - hd msg=9bed4cc0e5_1 from_ex=ecomm.api from_rk=ecomm.api.west\n    2017-12-14 00:42:57,035 - jtoc - INFO - relay msg_id=9bed4cc0e5_1 body={'msg_id': '9bed4cc0e5_1', 've\n    2017-12-14 00:42:57,045 - jtoc - INFO - relay done with msg_id=9bed4cc0e5_1\n    2017-12-14 00:42:57,045 - jtoc - INFO - task - ecomm_app.ecommerce.tasks.handle_user_conversion_events - simulating processing lag sleep=8.0 seconds\n    2017-12-14 00:43:05,052 - jtoc - INFO - hd msg=f66139a9cf_1 from_ex=ecomm.api from_rk=ecomm.api.west\n    2017-12-14 00:43:05,053 - jtoc - INFO - relay msg_id=f66139a9cf_1 body={'msg_id': 'f66139a9cf_1', 've\n    2017-12-14 00:43:05,061 - jtoc - INFO - relay done with msg_id=f66139a9cf_1\n    2017-12-14 00:43:05,061 - jtoc - INFO - task - ecomm_app.ecommerce.tasks.handle_user_conversion_events - simulating processing lag sleep=8.0 seconds\n    2017-12-14 00:43:13,073 - jtoc - INFO - hd msg=94d3a2c7ed_1 from_ex=ecomm.api from_rk=ecomm.api.west\n    2017-12-14 00:43:13,074 - jtoc - INFO - relay msg_id=94d3a2c7ed_1 body={'msg_id': '94d3a2c7ed_1', 've\n    2017-12-14 00:43:13,095 - jtoc - INFO - relay done with msg_id=94d3a2c7ed_1\n    2017-12-14 00:43:13,098 - jtoc - INFO - task - ecomm_app.ecommerce.tasks.handle_user_conversion_events - simulating processing lag sleep=8.0 seconds\n    2017-12-14 00:43:21,105 - jtoc - INFO - hd msg=b517f87ff4_1 from_ex=ecomm.api from_rk=ecomm.api.west\n    2017-12-14 00:43:21,106 - jtoc - INFO - relay msg_id=b517f87ff4_1 body={'msg_id': 'b517f87ff4_1', 've\n    2017-12-14 00:43:21,123 - jtoc - INFO - relay done with msg_id=b517f87ff4_1\n    2017-12-14 00:43:21,124 - jtoc - INFO - task - ecomm_app.ecommerce.tasks.handle_user_conversion_events - simulating processing lag sleep=8.0 seconds\n    2017-12-14 00:43:29,140 - jtoc - INFO - hd msg=822ef4142c_1 from_ex=ecomm.api from_rk=ecomm.api.west\n    2017-12-14 00:43:29,140 - jtoc - INFO - relay msg_id=822ef4142c_1 body={'msg_id': '822ef4142c_1', 've\n    2017-12-14 00:43:29,147 - jtoc - INFO - relay done with msg_id=822ef4142c_1\n    2017-12-14 00:43:29,147 - jtoc - INFO - task - ecomm_app.ecommerce.tasks.handle_user_conversion_events - simulating processing lag sleep=8.0 seconds\n\nVerify the Ecomm Celery Application Processed the Task\n------------------------------------------------------\n\nThe logs for the ecomm Celery worker should show something similar to:\n\n::\n\n    [2017-12-14 00:42:16,938: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[7848f13c-00e1-47d1-b5a5-a8e0dea1dc04]   expires:[2017-12-14 08:47:16.881373+00:00]\n    [2017-12-14 00:42:16,940: INFO/ForkPoolWorker-1] task - user_conversion_events - start body={'subscription_id': 321, 'r_id': '2483467dad_1', 'stripe_id': 876, 'version': 1, 'created': '2017-12-14T00:42:16.870156', 'product_id': 'JJJ', 'account_id': 999, 'msg_id': '46cb24f0d0_1'}\n    [2017-12-14 00:42:16,940: INFO/ForkPoolWorker-1] task - user_conversion_events - done\n    [2017-12-14 00:42:16,942: INFO/ForkPoolWorker-1] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[7848f13c-00e1-47d1-b5a5-a8e0dea1dc04] succeeded in 0.002363318002608139s: True\n    [2017-12-14 00:42:24,954: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[4ea2b08f-efa1-46f8-a522-7e2ccde37f4e]   expires:[2017-12-14 08:47:24.950295+00:00]\n    [2017-12-14 00:42:24,955: INFO/ForkPoolWorker-2] task - user_conversion_events - start body={'subscription_id': 321, 'r_id': '88daa66cac_1', 'stripe_id': 876, 'version': 1, 'created': '2017-12-14T00:42:24.947811', 'product_id': 'JJJ', 'account_id': 999, 'msg_id': 'd2724b75fa_1'}\n    [2017-12-14 00:42:24,955: INFO/ForkPoolWorker-2] task - user_conversion_events - done\n    [2017-12-14 00:42:24,960: INFO/ForkPoolWorker-2] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[4ea2b08f-efa1-46f8-a522-7e2ccde37f4e] succeeded in 0.005053305001638364s: True\n    [2017-12-14 00:42:32,979: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[496e6c89-e725-433d-8bfa-a0d0decc8e0d]   expires:[2017-12-14 08:47:32.965396+00:00]\n    [2017-12-14 00:42:32,981: INFO/ForkPoolWorker-3] task - user_conversion_events - start body={'subscription_id': 321, 'r_id': '2bb5cdd264_1', 'stripe_id': 876, 'version': 1, 'created': '2017-12-14T00:42:32.963186', 'product_id': 'JJJ', 'account_id': 999, 'msg_id': 'e72e09da34_1'}\n    [2017-12-14 00:42:32,981: INFO/ForkPoolWorker-3] task - user_conversion_events - done\n    [2017-12-14 00:42:32,987: INFO/ForkPoolWorker-3] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[496e6c89-e725-433d-8bfa-a0d0decc8e0d] succeeded in 0.00654161800048314s: True\n    [2017-12-14 00:42:41,008: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[f4f7681e-1bca-4798-a73a-89f62317651d]   expires:[2017-12-14 08:47:40.991378+00:00]\n    [2017-12-14 00:42:41,012: INFO/ForkPoolWorker-1] task - user_conversion_events - start body={'subscription_id': 321, 'r_id': '5365dc6b70_1', 'stripe_id': 876, 'version': 1, 'created': '2017-12-14T00:42:40.983174', 'product_id': 'JJJ', 'account_id': 999, 'msg_id': 'f5ec3f0c9d_1'}\n    [2017-12-14 00:42:41,012: INFO/ForkPoolWorker-1] task - user_conversion_events - done\n    [2017-12-14 00:42:41,014: INFO/ForkPoolWorker-1] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[f4f7681e-1bca-4798-a73a-89f62317651d] succeeded in 0.002192696003476158s: True\n    [2017-12-14 00:42:49,026: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[35d5ed9e-aacf-4b05-bae0-f74b8df83ad2]   expires:[2017-12-14 08:47:49.017937+00:00]\n    [2017-12-14 00:42:49,028: INFO/ForkPoolWorker-2] task - user_conversion_events - start body={'subscription_id': 321, 'r_id': 'f369b4c0e0_1', 'stripe_id': 876, 'version': 1, 'created': '2017-12-14T00:42:49.015218', 'product_id': 'JJJ', 'account_id': 999, 'msg_id': '222094db10_1'}\n    [2017-12-14 00:42:49,028: INFO/ForkPoolWorker-2] task - user_conversion_events - done\n    [2017-12-14 00:42:49,031: INFO/ForkPoolWorker-2] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[35d5ed9e-aacf-4b05-bae0-f74b8df83ad2] succeeded in 0.00297039799625054s: True\n    [2017-12-14 00:42:57,047: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[4c9137a1-e4c0-44f2-852d-b96e8004cf52]   expires:[2017-12-14 08:47:57.040272+00:00]\n    [2017-12-14 00:42:57,050: INFO/ForkPoolWorker-3] task - user_conversion_events - start body={'subscription_id': 321, 'r_id': '81646a1d3e_1', 'stripe_id': 876, 'version': 1, 'created': '2017-12-14T00:42:57.035385', 'product_id': 'JJJ', 'account_id': 999, 'msg_id': '9bed4cc0e5_1'}\n    [2017-12-14 00:42:57,051: INFO/ForkPoolWorker-3] task - user_conversion_events - done\n    [2017-12-14 00:42:57,053: INFO/ForkPoolWorker-3] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[4c9137a1-e4c0-44f2-852d-b96e8004cf52] succeeded in 0.0024162650006473996s: True\n    [2017-12-14 00:43:05,061: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[c0eafd5b-803a-4550-9bba-961d7ab7f4cc]   expires:[2017-12-14 08:48:05.056204+00:00]\n    [2017-12-14 00:43:05,064: INFO/ForkPoolWorker-1] task - user_conversion_events - start body={'subscription_id': 321, 'r_id': '360be5bb5d_1', 'stripe_id': 876, 'version': 1, 'created': '2017-12-14T00:43:05.052968', 'product_id': 'JJJ', 'account_id': 999, 'msg_id': 'f66139a9cf_1'}\n    [2017-12-14 00:43:05,065: INFO/ForkPoolWorker-1] task - user_conversion_events - done\n    [2017-12-14 00:43:05,067: INFO/ForkPoolWorker-1] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[c0eafd5b-803a-4550-9bba-961d7ab7f4cc] succeeded in 0.003034861001651734s: True\n    [2017-12-14 00:43:13,100: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[b402c99b-b998-48b8-9cb8-bb49b1289032]   expires:[2017-12-14 08:48:13.081228+00:00]\n    [2017-12-14 00:43:13,106: INFO/ForkPoolWorker-2] task - user_conversion_events - start body={'subscription_id': 321, 'r_id': '5d4d3f1277_1', 'stripe_id': 876, 'version': 1, 'created': '2017-12-14T00:43:13.074799', 'product_id': 'JJJ', 'account_id': 999, 'msg_id': '94d3a2c7ed_1'}\n    [2017-12-14 00:43:13,107: INFO/ForkPoolWorker-2] task - user_conversion_events - done\n    [2017-12-14 00:43:13,110: INFO/ForkPoolWorker-2] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[b402c99b-b998-48b8-9cb8-bb49b1289032] succeeded in 0.004359455000667367s: True\n    [2017-12-14 00:43:21,127: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[a57b8b49-349a-44d3-99fc-12f96d69d489]   expires:[2017-12-14 08:48:21.114216+00:00]\n    [2017-12-14 00:43:21,129: INFO/ForkPoolWorker-3] task - user_conversion_events - start body={'subscription_id': 321, 'r_id': '97ec19ac8d_1', 'stripe_id': 876, 'version': 1, 'created': '2017-12-14T00:43:21.106783', 'product_id': 'JJJ', 'account_id': 999, 'msg_id': 'b517f87ff4_1'}\n    [2017-12-14 00:43:21,130: INFO/ForkPoolWorker-3] task - user_conversion_events - done\n    [2017-12-14 00:43:21,133: INFO/ForkPoolWorker-3] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[a57b8b49-349a-44d3-99fc-12f96d69d489] succeeded in 0.003475217003142461s: True\n    [2017-12-14 00:43:29,150: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[da0c3a78-cac7-4b78-8a32-568d8a5c7362]   expires:[2017-12-14 08:48:29.143188+00:00]\n    [2017-12-14 00:43:29,152: INFO/ForkPoolWorker-1] task - user_conversion_events - start body={'subscription_id': 321, 'r_id': '6f3fe96baf_1', 'stripe_id': 876, 'version': 1, 'created': '2017-12-14T00:43:29.140647', 'product_id': 'JJJ', 'account_id': 999, 'msg_id': '822ef4142c_1'}\n    [2017-12-14 00:43:29,152: INFO/ForkPoolWorker-1] task - user_conversion_events - done\n    [2017-12-14 00:43:29,155: INFO/ForkPoolWorker-1] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[da0c3a78-cac7-4b78-8a32-568d8a5c7362] succeeded in 0.0034106169987353496s: True\n\nBenchmark the JSON to Celery Relay Service\n------------------------------------------\n\nThe ``start-mixin-load-test.py`` load test will send in 20,000 messages with no simulated lag. This may take a few moments to finish so you might want to open a new terminal and source the virtual env to run ``watch -n5 list-queues.sh`` for tracking the test's progress.\n\n::\n\n    ./start-mixin-load-test.py \n    2017-12-14 00:48:06,217 - robopub - INFO - Generating messages=20000\n    2017-12-14 00:48:06,694 - robopub - INFO - Publishing messages=20000\n    2017-12-14 00:48:06,821 - pub - INFO - 1.00 send done msg=200/20000 ex=ecomm.api rk=ecomm.api.west\n    2017-12-14 00:48:06,821 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=69ae9e80bf_1\n    2017-12-14 00:48:06,916 - pub - INFO - 2.00 send done msg=400/20000 ex=ecomm.api rk=ecomm.api.west\n    2017-12-14 00:48:06,917 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=43c153a155_1\n    2017-12-14 00:48:07,015 - pub - INFO - 3.00 send done msg=600/20000 ex=ecomm.api rk=ecomm.api.west\n    2017-12-14 00:48:07,016 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=1978ebf438_1\n    2017-12-14 00:48:07,075 - pub - INFO - 4.00 send done msg=800/20000 ex=ecomm.api rk=ecomm.api.west\n    2017-12-14 00:48:07,075 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=3dbae69bb2_1\n    2017-12-14 00:48:07,157 - pub - INFO - 5.00 send done msg=1000/20000 ex=ecomm.api rk=ecomm.api.west\n    2017-12-14 00:48:07,158 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=a5dda8b23a_1\n    2017-12-14 00:48:07,240 - pub - INFO - 6.00 send done msg=1200/20000 ex=ecomm.api rk=ecomm.api.west\n    2017-12-14 00:48:07,241 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=138a5c7939_1\n    2017-12-14 00:48:07,310 - pub - INFO - 7.00 send done msg=1400/20000 ex=ecomm.api rk=ecomm.api.west\n    2017-12-14 00:48:07,311 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=a1c6315380_1\n    2017-12-14 00:48:07,374 - pub - INFO - 8.00 send done msg=1600/20000 ex=ecomm.api rk=ecomm.api.west\n    2017-12-14 00:48:07,374 - pub - INFO - ex=ecomm.api rk=ecomm.api.west msg=f1cf343847_1\n\nOr with docker compose:\n\n::\n\n    docker-compose -f compose-start-mixin-load-test.yml up\n    WARNING: Found orphan containers (ecommworker, jtocrelay) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.\n    Starting mixinloadtest ... done\n    Attaching to mixinloadtest\n\n\nSample output during that load test:\n\n::\n\n    list-queues.sh \n\n    Listing Queues broker=localhost:15672\n\n+-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n|                     name                      | durable | auto_delete | consumers | messages | messages_ready | messages_unacknowledged |\n+-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n| celery                                        | True    | False       | 1         | 0        | 0              | 0                       |\n+-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n| celery@ecommerce_subscriber.celery.pidbox     | False   | True        | 1         | 0        | 0              | 0                       |\n+-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n| celeryev.28b0b3a0-2e82-4e16-b829-a2835763b3cb | False   | True        | 1         | 0        | 0              | 0                       |\n+-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n| celeryev.b019122d-0dd3-48c0-8c0a-b82f4fb8d4d7 | False   | True        | 1         | 0        | 0              | 0                       |\n+-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n| ecomm.api.west                                | True    | False       | 1         | 17810    | 17809          | 1                       |\n+-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n\nView the Ecomm Celery Worker Tasks in Flower\n--------------------------------------------\n\nYou can also watch progress using the Flower Celery monitoring application that's included in the docker compose file.\n\nHere's a snapshot of my 20,000 + 10 messages using the ``celery@ecommerce_subscriber`` Celery worker.\n\n.. image:: https://raw.githubusercontent.com/jay-johnson/celery-connectors/master/_images/flower-jtoc-relay-results.png\n    :align: center\n\nThe ``Processed`` and ``Succeeded`` task counts for the ``celery@ecommerce_subscriber`` should increment each time a User Conversion Event is published by the ecomm relay to the ecomm worker.\n\nhttp://localhost:5555/dashboard (login: admin/admin)\n\nView specific task details:\n\nhttp://localhost:5555/tasks\n\nStop the Ecomm Demo\n-------------------\n\nRestart the docker containers to a good, clean state.\n\nStop:\n\n::\n\n    stop-redis-and-rabbitmq.sh \n    Stopping redis and rabbitmq\n    Stopping celrabbit1      ... done\n    Stopping celredis1       ... done\n    Stopping celflowerredis  ... done\n    Stopping celflowerrabbit ... done\n\nStart:\n\n::\n\n    start-redis-and-rabbitmq.sh \n    Starting redis and rabbitmq\n    Creating celrabbit1 ... done\n\nVerify the Relay Service Automatically Healed\n---------------------------------------------\n\nWant to try the load test again now that we just simulated a broker outage for all of the messaging and monitoring containers?\n\n::\n\n    ./start-mixin-load-test.py \n\nor\n\n::\n\n    docker-compose -f compose-start-mixin-load-test.yml up\n\nIf not, then stop the ecomm relay and ecomm worker terminal sessions using: ``ctrl + c``\n\nRunning an Ecommerce JSON-to-Celery Relay Service - Example 2\n=============================================================\n\nThis example uses just kombu producers and consumers instead of the kombu.ConsumerProducerMixin to run the same relay as the example above.\n\nStart Ecommerce Celery Worker\n-----------------------------\n\nStart a Celery worker for an existing ecommerce application from a hypothetical Django or Flask server.\n\nNote: Please run this from the base directory for the repository and source the virtual env: ``source venv/bin/activate``\n\n::\n\n    ./start-ecomm-worker.sh\n\n    -------------- celery@ecommerce_subscriber v4.1.0 (latentcall)\n    ---- **** ----- \n    --- * ***  * -- Linux-4.7.4-200.fc24.x86_64-x86_64-with-fedora-24-Twenty_Four 2017-12-14 00:33:02\n    -- * - **** --- \n    - ** ---------- [config]\n    - ** ---------- .> app:         ecommerce-worker:0x7f0c23f1c550\n    - ** ---------- .> transport:   amqp://rabbitmq:**@localhost:5672//\n    - ** ---------- .> results:     redis://localhost:6379/10\n    - *** --- * --- .> concurrency: 3 (prefork)\n    -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)\n    --- ***** ----- \n    -------------- [queues]\n                    .> celery           exchange=celery(direct) key=celery\n\n\n    [tasks]\n    . ecomm_app.ecommerce.tasks.handle_user_conversion_events\n\n    [2017-12-14 00:33:02,243: INFO/MainProcess] Connected to amqp://rabbitmq:**@127.0.0.1:5672//\n    [2017-12-14 00:33:02,260: INFO/MainProcess] mingle: searching for neighbors\n    [2017-12-14 00:33:03,293: INFO/MainProcess] mingle: all alone\n    [2017-12-14 00:33:03,337: INFO/MainProcess] celery@ecommerce_subscriber ready.\n    [2017-12-14 00:33:05,275: INFO/MainProcess] Events of group {task} enabled by remote.\n\nOr with docker compose:\n\n::\n\n    docker-compose -f compose-start-ecomm-worker.yml up\n    Starting ecommworker ... done\n    Attaching to ecommworker\n\nNotice the worker is named ``celery@ecommerce_subscriber`` this is the identifier for viewing the Celery application in Flower:\n\nhttp://localhost:5555/worker/celery@ecommerce_subscriber (login: admin/admin)\n\nStart Ecomm Relay\n-----------------\n\nThis process will consume JSON dictionary messages on the ``user.events.conversions`` RabbitMQ queue and pass the message to the ecomm Celery app as a ``ecomm_app.ecommerce.tasks.handle_user_conversion_events`` Celery task.\n\nPlease start this in a new terminal that has sourced the virtual env: ``source venv/bin/activate``\n\n::\n\n    ./start-ecomm-relay.py\n    2017-12-14 00:33:36,943 - ecomm-relay-loader - INFO - Start - ecomm-relay\n    2017-12-14 00:33:36,944 - message-processor - INFO - ecomm-relay START - consume_queue=user.events.conversions rk=reporting.accounts callback=relay_callback\n    2017-12-14 00:33:36,944 - kombu-subscriber - INFO - setup routing\n\nOr with docker compose:\n\n::\n\n    docker-compose -f compose-start-ecomm-relay.yml up\n    WARNING: Found orphan containers (ecommworker) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.\n    Creating ecommrelay ... done\n    Attaching to ecommrelay\n\nPublish a User Conversion Event to the Ecomm Relay\n--------------------------------------------------\n\nThis will use Kombu to publish a JSON dictionary message to the ``user.events.conversions`` RabbitMQ queue which is monitored by the ecomm relay.\n\nPlease start this in a new terminal that has sourced the virtual env: ``source venv/bin/activate``\n\n::\n\n    publish-user-conversion-events-rabbitmq.py \n    INFO:publish-user-conversion-events:Start - publish-user-conversion-events\n    INFO:publish-user-conversion-events:Sending user conversion event msg={'product_id': 'XYZ', 'stripe_id': 999, 'account_id': 777, 'created': '2017-12-14T00:33:55.826534', 'subscription_id': 888} ex=user.events rk=user.events.conversions\n    INFO:kombu-publisher:SEND - exch=user.events rk=user.events.conversions\n    INFO:publish-user-conversion-events:End - publish-user-conversion-events sent=True\n\nOr with docker compose:\n\n::\n\n    docker-compose -f compose-publish-user-conversion-events-rabbitmq.yml up\n    WARNING: Found orphan containers (ecommrelay, ecommworker) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.\n    Starting ucepubrmq ... done\n    Attaching to ucepubrmq\n\nVerify the Ecomm Relay Processed the Conversion Message\n-------------------------------------------------------\n\nThe logs for the ecomm relay should show something similar to:\n\n::\n\n    2017-12-14 00:33:55,865 - ecomm-relay-loader - INFO - Sending broker=amqp://rabbitmq:rabbitmq@localhost:5672// body={'org_msg': {'stripe_id': 999, 'created': '2017-12-14T00:33:55.826534', 'product_id': 'XYZ', 'subscription_id': 888, 'account_id': 777}, 'stripe_id': 876, 'version': 1, 'account_id': 999, 'msg_id': '7a73a74d-f539-4634-8a03-2aa2a5fd8d5e', 'created': '2017-12-14T00:33:55.863870', 'product_id': 'JJJ', 'subscription_id': 321}\n    2017-12-14 00:33:55,928 - ecomm-relay-loader - INFO - Done with msg_id=7a73a74d-f539-4634-8a03-2aa2a5fd8d5e result=True\n\nVerify the Ecomm Celery Application Processed the Task\n------------------------------------------------------\n\nThe logs for the ecomm Celery worker should show something similar to:\n\n::\n\n    [2017-12-14 00:33:55,919: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[9ee85235-0ffb-4c46-9cd7-0bd2c153bd9b]  \n    [2017-12-14 00:33:55,921: INFO/ForkPoolWorker-1] task - user_conversion_events - start body={'subscription_id': 321, 'stripe_id': 876, 'org_msg': {'subscription_id': 888, 'stripe_id': 999, 'created': '2017-12-14T00:33:55.826534', 'product_id': 'XYZ', 'account_id': 777}, 'version': 1, 'created': '2017-12-14T00:33:55.863870', 'product_id': 'JJJ', 'account_id': 999, 'msg_id': '7a73a74d-f539-4634-8a03-2aa2a5fd8d5e'}\n    [2017-12-14 00:33:55,921: INFO/ForkPoolWorker-1] task - user_conversion_events - done\n    [2017-12-14 00:33:55,926: INFO/ForkPoolWorker-1] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[9ee85235-0ffb-4c46-9cd7-0bd2c153bd9b] succeeded in 0.0055257950007217005s: True\n\nView the Ecomm Celery Worker Tasks in Flower\n--------------------------------------------\n\nThe ``Processed`` and ``Succeeded`` task counts for the ``celery@ecommerce_subscriber`` should increment each time a User Conversion Event is published by the ecomm relay to the ecomm worker.\n\nhttp://localhost:5555/dashboard\n\nView specific task details:\n\nhttp://localhost:5555/tasks\n\nStop the Ecomm Demo\n-------------------\n\nIn the ecomm relay and ecomm worker terminal sessions use: ``ctrl + c`` to stop the processes.\n\nRestart the docker containers to a good, clean state.\n\nStop:\n\n::\n\n    stop-redis-and-rabbitmq.sh \n    Stopping redis and rabbitmq\n    Stopping celrabbit1      ... done\n    Stopping celredis1       ... done\n    Stopping celflowerredis  ... done\n    Stopping celflowerrabbit ... done\n\nStart:\n\n::\n\n    start-redis-and-rabbitmq.sh \n    Starting redis and rabbitmq\n    Creating celrabbit1 ... done\n\nView the Ecomm Celery Worker Tasks in Flower\n--------------------------------------------\n\nThe ``Processed`` and ``Succeeded`` task counts for the ``celery@ecommerce_subscriber`` should increment each time a User Conversion Event is published by the ecomm relay to the ecomm worker.\n\nhttp://localhost:5555/dashboard\n\nView specific task details:\n\nhttp://localhost:5555/tasks\n\nStop the Ecomm Demo\n-------------------\n\nIn the ecomm relay and ecomm worker terminal sessions use: ``ctrl + c`` to stop the processes.\n\nRestart the docker containers to a good, clean state.\n\nStop:\n\n::\n\n    stop-redis-and-rabbitmq.sh \n    Stopping redis and rabbitmq\n    Stopping celrabbit1      ... done\n    Stopping celredis1       ... done\n    Stopping celflowerredis  ... done\n    Stopping celflowerrabbit ... done\n\nStart:\n\n::\n\n    start-redis-and-rabbitmq.sh \n    Starting redis and rabbitmq\n    Creating celrabbit1 ... done\n    Creating celrabbit1 ... done\n    Creating celredis1 ... done\n    Creating celflowerredis ... done\n\nCelery Bootstep with RabbitMQ Outage Example\n============================================\n\nThis example uses Celery bootsteps (http://docs.celeryproject.org/en/latest/userguide/extending.html) to run a standalone, headless subscriber that consumes routed messages to two queues. It will set up a RabbitMQ topic exchange with a queue that is bound using a routing key and a separate direct queue for additional messages to process. Once the entities are available in RabbitMQ, Kombu publishes the message to the exchanges and RabbitMQ provides the messaging facility to route the messages to the subscribed Celery workers' queues. Once messages are being processed we will simulate a broker failure and see how resilient Celery bootsteps are to this type of disaster.\n\n#.  Stop and Start the docker containers\n\n    ::\n\n        ./stop-redis-and-rabbitmq.sh \n        Stopping redis and rabbitmq\n        Stopping celredis1       ... done\n        Stopping celflowerrabbit ... done\n        Stopping celflowerredis  ... done\n        Stopping celrabbit1      ... done\n\n    ::\n\n        ./start-redis-and-rabbitmq.sh \n        Starting redis and rabbitmq\n        Creating celrabbit1 ... done\n        Creating celredis1 ... \n        Creating celflowerredis ... \n        Creating celrabbit1 ... \n\n#.  List the Queues\n\n    ::\n\n        list-queues.sh \n\n    Listing Queues broker=localhost:15672\n\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    |                     name                      | durable | auto_delete | consumers | messages | messages_ready | messages_unacknowledged |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celeryev.a1ccb5f7-4f76-4e26-9cdc-bf5438ba5362 | False   | True        | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n\n#.  Publish a message\n\n    ::\n\n        run_rabbitmq_publisher.py \n        INFO:run-rabbitmq-publisher:Start - run-rabbitmq-publisher\n        INFO:run-rabbitmq-publisher:Sending msg={'created': '2017-12-14T18:08:29.481313', 'account_id': 456} ex=reporting rk=reporting.accounts\n        INFO:kombu-publisher:SEND - exch=reporting rk=reporting.accounts\n        INFO:run-rabbitmq-publisher:End - run-rabbitmq-publisher sent=True\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-run-rabbitmq-publisher.yml up\n        Creating kombupubrmq ... done\n        Attaching to kombupubrmq\n        kombupubrmq    | 2017-12-15 07:31:23,802 - run-rabbitmq-publisher - INFO - Start - run-rabbitmq-publisher\n        kombupubrmq    | 2017-12-15 07:31:23,802 - run-rabbitmq-publisher - INFO - Sending msg={'account_id': 456, 'created': '2017-12-15T07:31:23.802616'} ex=reporting rk=reporting.accounts\n        kombupubrmq    | 2017-12-15 07:31:23,899 - kombu-publisher - INFO - SEND - exch=reporting rk=reporting.accounts\n        kombupubrmq    | 2017-12-15 07:31:23,903 - run-rabbitmq-publisher - INFO - End - run-rabbitmq-publisher sent=True\n        kombupubrmq exited with code 0\n\n#.  Confirm the message is ready in the RabbitMQ Queue\n\n    Note the ``messages`` and ``messages_ready`` count increased while the ``messages_unacknowledged`` did not. Which is because we have not started the subscriber to process ready messages in the ``reporting.accounts`` queue.\n\n    ::\n\n        list-queues.sh \n\n    Listing Queues broker=localhost:15672\n\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    |                     name                      | durable | auto_delete | consumers | messages | messages_ready | messages_unacknowledged |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celeryev.a1ccb5f7-4f76-4e26-9cdc-bf5438ba5362 | False   | True        | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | reporting.accounts                            | True    | False       | 0         | 1        | 1              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n\n#.  List the Exchanges\n\n    ::\n\n        list-exchanges.sh \n\n    Listing Exchanges broker=localhost:15672\n\n    +---------------------+---------+---------+-------------+\n    |        name         |  type   | durable | auto_delete |\n    +---------------------+---------+---------+-------------+\n    |                     | direct  | True    | False       |\n    +---------------------+---------+---------+-------------+\n    | amq.direct          | direct  | True    | False       |\n    +---------------------+---------+---------+-------------+\n    | amq.fanout          | fanout  | True    | False       |\n    +---------------------+---------+---------+-------------+\n    | amq.headers         | headers | True    | False       |\n    +---------------------+---------+---------+-------------+\n    | amq.match           | headers | True    | False       |\n    +---------------------+---------+---------+-------------+\n    | amq.rabbitmq.log    | topic   | True    | False       |\n    +---------------------+---------+---------+-------------+\n    | amq.rabbitmq.trace  | topic   | True    | False       |\n    +---------------------+---------+---------+-------------+\n    | amq.topic           | topic   | True    | False       |\n    +---------------------+---------+---------+-------------+\n    | celery.pidbox       | fanout  | False   | False       |\n    +---------------------+---------+---------+-------------+\n    | celeryev            | topic   | True    | False       |\n    +---------------------+---------+---------+-------------+\n    | reply.celery.pidbox | direct  | False   | False       |\n    +---------------------+---------+---------+-------------+\n    | reporting           | topic   | True    | False       |\n    +---------------------+---------+---------+-------------+\n\n#.  Consume that message by starting up the Celery Rabbitmq subscriber module\n\n    This will consume messages from the ``reporting.accounts`` and ``reporting.subscriptions`` queues.\n\n    ::\n\n        celery worker -A run_rabbitmq_subscriber -n rabbitmq_bootstep -c 3 --loglevel=INFO -Ofair\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-run-celery-rabbitmq-subscriber.yml up\n        Creating celeryrabbitmqsubscriber ... done\n        Attaching to celeryrabbitmqsubscriber\n\n#.  Confirm the worker's logs show the message was received\n\n    ::\n\n        2017-12-14 10:10:25,832: INFO callback received msg body={'account_id': 456, 'created': '2017-12-14T18:08:29.481313'} from_ex=reporting from_rk=reporting.accounts\n\n#.  View the Rabbit Subscriber ``celery@rabbitmq_bootstep`` in Flower\n\n    Rabbit Flower server (login admin/admin)\n\n    http://localhost:5555/\n\n#.  Verify the message is no longer in the Queue and Celery is connected as a consumer\n\n    With the Celery RabbitMQ worker still running, in a new terminal list the queues. Verify there is a consumer on the ``reporting.accounts`` and ``reporting.subscriptions`` queues.\n\n    ::\n\n        list-queues.sh\n\n    Listing Queues broker=localhost:15672\n\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    |                     name                      | durable | auto_delete | consumers | messages | messages_ready | messages_unacknowledged |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celery.rabbit.sub                             | True    | False       | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celery@rabbitmq_bootstep.celery.pidbox        | False   | True        | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celeryev.a1ccb5f7-4f76-4e26-9cdc-bf5438ba5362 | False   | True        | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celeryev.f85fe29a-b729-48fa-a17d-b7e12c14dba8 | False   | True        | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | reporting.accounts                            | True    | False       | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | reporting.subscriptions                       | True    | False       | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n\n#.  Start the Queue watcher\n\n    In a new terminal activate the virtual env ``source venv/bin/activate``. \n\n    ::\n\n        watch-queues.sh\n\n    The watch will poll RabbitMQ for the queues every second and before the load tests start should look empty:\n\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    |                     name                      | durable | auto_delete | consumers | messages | messages_ready | messages_unacknowledged |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celery.rabbit.sub                             | True    | False       | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celery@rabbitmq_bootstep.celery.pidbox        | False   | True        | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celeryev.a1ccb5f7-4f76-4e26-9cdc-bf5438ba5362 | False   | True        | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celeryev.f85fe29a-b729-48fa-a17d-b7e12c14dba8 | False   | True        | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | reporting.accounts                            | True    | False       | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | reporting.subscriptions                       | True    | False       | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n\n#.  Start the Accounts and Subscriptions Load Tests\n\n    This will require two separate terminal sessions with the virtual env activated ``source venv/bin/activate``. \n\n    In terminal 1 start the Accounts load test\n\n    ::\n\n        start-load-test-rabbitmq.py\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-start-load-test-rabbitmq.yml up\n        WARNING: Found orphan containers (subsloadtest, celeryrabbitmqsubscriber) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.\n        Creating loadtestrmq ... done\n        Attaching to loadtestrmq\n\n    In terminal 2 start the Subscriptions load test\n\n    ::\n\n        start-subscriptions-rabbitmq-test.py\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-start-subscriptions-rabbitmq-test.yml up\n        WARNING: Found orphan containers (celeryrabbitmqsubscriber) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.\n        Creating subsloadtest ... done\n        Attaching to subsloadtest\n\n#.  Verify the Queues are filling up\n\n    After a few seconds, the queues should be filling up with Account and Subscription messages that are being actively processed.\n\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    |                     name                      | durable | auto_delete | consumers | messages | messages_ready | messages_unacknowledged |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celery.rabbit.sub                             | True    | False       | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celery@rabbitmq_bootstep.celery.pidbox        | False   | True        | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celeryev.a1ccb5f7-4f76-4e26-9cdc-bf5438ba5362 | False   | True        | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | celeryev.f85fe29a-b729-48fa-a17d-b7e12c14dba8 | False   | True        | 1         | 0        | 0              | 0                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | reporting.accounts                            | True    | False       | 1         | 31157    | 31154          | 3                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n    | reporting.subscriptions                       | True    | False       | 1         | 30280    | 30277          | 3                       |\n    +-----------------------------------------------+---------+-------------+-----------+----------+----------------+-------------------------+\n\n#.  Verify the Celery Bootstep Subscriber is processing messages\n\n    By default the Celery subscriber workers are processing 1 message at a time per consumer. In this example we started 3 workers so there are 3 messages that are unacknowledged at a time. Confirm messages are being processed ``from_rk=reporting.subscriptions`` and ``from_rk=reporting.accounts``. This means the Celery workers are processing messages that have routing keys from the different queues.\n\n    ::\n\n        2017-12-14 10:24:12,168: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.178164', 'msg_id': '66e0d69aa0_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:24:12,168: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.845445', 'msg_id': 'd64e18e7be_1'} from_ex= from_rk=reporting.accounts\n        2017-12-14 10:24:12,169: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.178278', 'msg_id': '712132669a_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:24:12,170: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.845478', 'msg_id': '2174427099_1'} from_ex= from_rk=reporting.accounts\n        2017-12-14 10:24:12,182: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.178345', 'msg_id': '1d4a251145_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:24:12,183: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.178380', 'msg_id': 'b62922b333_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:24:12,184: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.178404', 'msg_id': 'adc1b1988e_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:24:12,184: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.845491', 'msg_id': '91ac6c413c_1'} from_ex= from_rk=reporting.accounts\n        2017-12-14 10:24:12,184: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.845505', 'msg_id': '0ffd4abf90_1'} from_ex= from_rk=reporting.accounts\n        2017-12-14 10:24:12,185: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.845519', 'msg_id': '5a11d2aa97_1'} from_ex= from_rk=reporting.accounts\n        2017-12-14 10:24:12,185: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.178843', 'msg_id': '77dd35ade4_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:24:12,186: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.178944', 'msg_id': '2317ff179d_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:24:12,186: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.179021', 'msg_id': 'acce2d2672_1'} from_ex= from_rk=reporting.subscriptions\n\n#.  Stop the Docker containers\n\n    Note: you can stop the docker containers while the tests are still publishing messages if you want. They should gracefully reconnect once the broker is restored.\n\n    ::\n\n        ./stop-redis-and-rabbitmq.sh \n        Stopping redis and rabbitmq\n        Stopping celflowerredis  ... done\n        Stopping celflowerrabbit ... done\n        Stopping celrabbit1      ... done\n        Stopping celredis1       ... done\n\n#.  Confirm Celery was disconnected\n\n    ::\n\n        2017-12-14 10:27:00,213: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.995821', 'msg_id': 'a138cf8d8c_1'} from_ex= from_rk=reporting.accounts\n        2017-12-14 10:27:00,213: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.461333', 'msg_id': '406df22df7_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:27:00,214: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.461346', 'msg_id': 'a473232ee4_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:27:00,214: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:21:46.461361', 'msg_id': '12219ca1fd_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:27:00,330: WARNING consumer: Connection to broker lost. Trying to re-establish the connection...\n        Traceback (most recent call last):\n        File \"/home/driver/dev/celery-connectors/venv/lib/python3.5/site-packages/celery/worker/consumer/consumer.py\", line 320, in start\n            blueprint.start(self)\n        File \"/home/driver/dev/celery-connectors/venv/lib/python3.5/site-packages/celery/bootsteps.py\", line 119, in start\n            step.start(parent)\n        File \"/home/driver/dev/celery-connectors/venv/lib/python3.5/site-packages/celery/worker/consumer/consumer.py\", line 596, in start\n            c.loop(*c.loop_args())\n        File \"/home/driver/dev/celery-connectors/venv/lib/python3.5/site-packages/celery/worker/loops.py\", line 88, in asynloop\n            next(loop)\n        File \"/home/driver/dev/celery-connectors/venv/lib/python3.5/site-packages/kombu-4.1.0-py3.5.egg/kombu/async/hub.py\", line 354, in create_loop\n            cb(*cbargs)\n        File \"/home/driver/dev/celery-connectors/venv/lib/python3.5/site-packages/kombu-4.1.0-py3.5.egg/kombu/transport/base.py\", line 236, in on_readable\n            reader(loop)\n        File \"/home/driver/dev/celery-connectors/venv/lib/python3.5/site-packages/kombu-4.1.0-py3.5.egg/kombu/transport/base.py\", line 218, in _read\n            drain_events(timeout=0)\n        File \"/home/driver/dev/celery-connectors/venv/lib/python3.5/site-packages/amqp-2.2.2-py3.5.egg/amqp/connection.py\", line 471, in drain_events\n            while not self.blocking_read(timeout):\n        File \"/home/driver/dev/celery-connectors/venv/lib/python3.5/site-packages/amqp-2.2.2-py3.5.egg/amqp/connection.py\", line 476, in blocking_read\n            frame = self.transport.read_frame()\n        File \"/home/driver/dev/celery-connectors/venv/lib/python3.5/site-packages/amqp-2.2.2-py3.5.egg/amqp/transport.py\", line 226, in read_frame\n            frame_header = read(7, True)\n        File \"/home/driver/dev/celery-connectors/venv/lib/python3.5/site-packages/amqp-2.2.2-py3.5.egg/amqp/transport.py\", line 409, in _read\n            raise IOError('Socket closed')\n        OSError: Socket closed\n        2017-12-14 10:27:00,341: ERROR consumer: Cannot connect to amqp://rabbitmq:**@127.0.0.1:5672//: [Errno 104] Connection reset by peer.\n        Trying again in 2.00 seconds...\n\n        2017-12-14 10:27:02,369: ERROR consumer: Cannot connect to amqp://rabbitmq:**@127.0.0.1:5672//: [Errno 111] Connection refused.\n        Trying again in 4.00 seconds...\n\n#.  Start the Docker containers\n\n    ::\n\n        ./start-redis-and-rabbitmq.sh \n        Starting redis and rabbitmq\n        Creating celrabbit1 ... done\n        Creating celredis1 ... \n        Creating celflowerrabbit ... \n        Creating celflowerredis ... \n\n#.  Verify the Celery workers reconnected\n\n    ::\n\n        2017-12-14 10:28:50,841: INFO Connected to amqp://rabbitmq:**@127.0.0.1:5672//\n        2017-12-14 10:28:50,872: INFO mingle: searching for neighbors\n        2017-12-14 10:28:51,925: INFO mingle: all alone\n\n#.  Start the multi-queue load test publishers again\n\n    In terminal 1:\n\n    ::\n\n        ./start-subscriptions-rabbitmq-test.py\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-start-load-test-rabbitmq.yml up\n\n    In terminal 2:\n\n    ::\n\n        ./start-load-test-rabbitmq.py\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-start-subscriptions-rabbitmq-test.yml up\n\n#.  Verify Celery is processing messages from both queues again\n\n    ::\n\n        2017-12-14 10:32:19,325: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:31:07.315190', 'msg_id': '22ede22ba6_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:32:19,326: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:31:07.315213', 'msg_id': '26f1103534_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:32:19,329: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:31:05.232153', 'msg_id': '10d7a731ca_1'} from_ex= from_rk=reporting.accounts\n        2017-12-14 10:32:19,333: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:31:05.232174', 'msg_id': 'ae75ede630_1'} from_ex= from_rk=reporting.accounts\n        2017-12-14 10:32:19,336: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:31:07.315225', 'msg_id': '0e86894ae3_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:32:19,337: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:31:05.232186', 'msg_id': '2066f80569_1'} from_ex= from_rk=reporting.accounts\n        2017-12-14 10:32:19,337: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:31:07.315240', 'msg_id': 'ea82241224_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:32:19,337: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:31:07.315264', 'msg_id': 'accbebead8_1'} from_ex= from_rk=reporting.subscriptions\n        2017-12-14 10:32:19,339: INFO callback received msg body={'data': {}, 'created': '2017-12-14T18:31:05.232198', 'msg_id': '8788b7fa97_1'} from_ex= from_rk=reporting.accounts\n\nStop the Celery Bootstep example\n--------------------------------\n\nIn all example terminal sessions, use: ``ctrl + c`` to stop any processes you no longer want to run.\n\nRestart the docker containers to a good, clean state for the next example.\n\nStop:\n\n::\n\n    stop-redis-and-rabbitmq.sh \n    Stopping redis and rabbitmq\n    Stopping celrabbit1      ... done\n    Stopping celredis1       ... done\n    Stopping celflowerredis  ... done\n    Stopping celflowerrabbit ... done\n\nStart:\n\n::\n\n    start-redis-and-rabbitmq.sh \n    Starting redis and rabbitmq\n    Creating celrabbit1 ... done\n\nRedis Message Processing Example\n================================\n\nThis example uses Celery bootsteps (http://docs.celeryproject.org/en/latest/userguide/extending.html) to run a standalone, headless subscriber that consumes messages from a Redis key which emulates a RabbitMQ queue. Kombu publishes the message to the Redis key.\n\n#.  Check that the Redis has no keys\n\n    ::\n\n        redis-cli\n        127.0.0.1:6379> keys *\n        (empty list or set)\n        127.0.0.1:6379> \n\n#.  Publish a message\n\n    ::\n\n        run_redis_publisher.py \n        2017-12-09 08:20:04,026 - run-redis-publisher - INFO - Start - run-redis-publisher\n        2017-12-09 08:20:04,027 - run-redis-publisher - INFO - Sending msg={'account_id': 123, 'created': '2017-12-09T08:20:04.027159'} ex=reporting.accounts rk=reporting.accounts\n        2017-12-09 08:20:04,050 - redis-publisher - INFO - SEND - exch=reporting.accounts rk=reporting.accounts\n        2017-12-09 08:20:04,052 - run-redis-publisher - INFO - End - run-redis-publisher sent=True\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-run-redis-publisher.yml up\n        Creating kombupubredis ... done\n        Attaching to kombupubredis\n        kombupubredis    | 2017-12-15 07:44:47,047 - run-redis-publisher - INFO - Start - run-redis-publisher\n        kombupubredis    | 2017-12-15 07:44:47,047 - run-redis-publisher - INFO - Sending msg={'account_id': 123, 'created': '2017-12-15T07:44:47.047355'} ex=reporting.accounts rk=reporting.accounts\n        kombupubredis    | 2017-12-15 07:44:47,127 - kombu-publisher - INFO - SEND - exch=reporting.accounts rk=reporting.accounts\n        kombupubredis    | 2017-12-15 07:44:47,132 - run-redis-publisher - INFO - End - run-redis-publisher sent=True\n        kombupubredis exited with code 0\n\n#.  Consume messages using the subscriber module\n\n    ::\n\n        celery worker -A run_redis_subscriber --loglevel=INFO -Ofair\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-run-celery-redis-subscriber.yml up\n        WARNING: Found orphan containers (kombupubredis) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.\n        Creating celeryredissubscriber ... done\n        Attaching to celeryredissubscriber\n\n\n#.  Confirm the Celery worker received the message\n\n    ::\n\n        2017-12-09 08:20:08,221: INFO callback received msg body={u'account_id': 123, u'created': u'2017-12-09T08:20:04.027159'}\n\n#.  View the Redis Subscriber in Flower\n\n    Redis Flower server (login admin/admin)\n\n    http://localhost:5556/\n\n#.  Look at the Redis keys\n\n    ::\n\n        redis-cli\n        127.0.0.1:6379> keys *\n        1) \"_kombu.binding.celeryev\"\n        2) \"_kombu.binding.celery\"\n        3) \"_kombu.binding.celery.pidbox\"\n        4) \"_kombu.binding.reporting.accounts\"\n        5) \"unacked_mutex\"\n        127.0.0.1:6379> \n\nRedis Kombu Subscriber\n======================\n\nIf you do not want to use Celery, you can use the ``KombuSubscriber`` class to process messages. This class will wait for a configurable amount of seconds to consume a single message from the subscribed queue and then stop processing.\n\n#.  Check the Redis keys\n\n    ::\n\n        redis-cli\n        127.0.0.1:6379> keys *\n        1) \"_kombu.binding.reporting.accounts\"\n        2) \"_kombu.binding.celery.redis.sub\"\n        127.0.0.1:6379> \n\n#.  Run the Redis Publisher\n\n    ::\n\n        run_redis_publisher.py \n        2017-12-09 11:46:39,743 - run-redis-publisher - INFO - Start - run-redis-publisher\n        2017-12-09 11:46:39,743 - run-redis-publisher - INFO - Sending msg={'account_id': 123, 'created': '2017-12-09T11:46:39.743636'} ex=reporting.accounts rk=reporting.accounts\n        2017-12-09 11:46:39,767 - redis-publisher - INFO - SEND - exch=reporting.accounts rk=reporting.accounts\n        2017-12-09 11:46:39,770 - run-redis-publisher - INFO - End - run-redis-publisher sent=True\n\n#.  Run the Redis Kombu Subscriber\n\n    By default, this will wait for a single message to be delivered within 10 seconds.\n\n    ::\n\n        kombu_redis_subscriber.py \n        2017-12-09 11:47:58,798 - kombu-redis-subscriber - INFO - Start - kombu-redis-subscriber\n        2017-12-09 11:47:58,798 - kombu-redis-subscriber - INFO - setup routing\n        2017-12-09 11:47:58,822 - kombu-redis-subscriber - INFO - kombu-redis-subscriber - kombu.subscriber queues=reporting.accounts consuming with callback=handle_message\n        2017-12-09 11:47:58,823 - kombu-redis-subscriber - INFO - callback received msg body={u'account_id': 123, u'created': u'2017-12-09T11:46:39.743636'}\n        2017-12-09 11:47:58,824 - kombu-redis-subscriber - INFO - End - kombu-redis-subscriber\n\n#.  Check the Redis keys\n\n    Nothing should have changed:\n\n    ::\n\n        127.0.0.1:6379> keys *\n        1) \"_kombu.binding.reporting.accounts\"\n        2) \"_kombu.binding.celery.redis.sub\"\n        127.0.0.1:6379> \n\nRabbitMQ Kombu Subscriber\n=========================\n\nIf you do not want to use Celery, you can use the ``KombuSubscriber`` class to process messages. This class will wait for a configurable amount of seconds to consume a single message from the subscribed queue and then stop processing.\n\n#.  List the Queues\n\n    If the docker containers are still running the previous RabbitMQ pub/sub test will still have the queues, exchanges and bindings still left over. If not then skip this step.\n\n    ::\n\n        list-queues.sh \n\n    Listing Queues broker=localhost:15672\n\n    +--------------------+-----------+----------+----------------+-------------------------+\n    |        name        | consumers | messages | messages_ready | messages_unacknowledged |\n    +--------------------+-----------+----------+----------------+-------------------------+\n    | celery.rabbit.sub  | 0         | 0        | 0              | 0                       |\n    +--------------------+-----------+----------+----------------+-------------------------+\n    | reporting.accounts | 0         | 0        | 0              | 0                       |\n    +--------------------+-----------+----------+----------------+-------------------------+\n\n#.  Run the RabbitMQ Subscriber\n\n    Please note this output assumes there are no messages in the queue already from a previous test. It will wait for 10 seconds before stopping.\n\n    ::\n\n        kombu_rabbitmq_subscriber.py \n        2017-12-09 11:53:56,948 - kombu-rabbitmq-subscriber - INFO - Start - kombu-rabbitmq-subscriber\n        2017-12-09 11:53:56,948 - kombu-rabbitmq-subscriber - INFO - setup routing\n        2017-12-09 11:53:56,973 - kombu-rabbitmq-subscriber - INFO - kombu-rabbitmq-subscriber - kombu.subscriber queues=reporting.accounts consuming with callback=handle_message\n        2017-12-09 11:54:06,975 - kombu-rabbitmq-subscriber - INFO - End - kombu-rabbitmq-subscriber\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-kombu-rabbitmq-subscriber.yml up\n        Recreating kombusubrmq ... done\n        Attaching to kombusubrmq\n        kombusubrmq    | 2017-12-15 07:51:35,444 - kombu-rabbitmq-subscriber - INFO - Start - kombu-rabbitmq-subscriber\n        kombusubrmq    | 2017-12-15 07:51:35,445 - kombu-subscriber - INFO - setup routing\n        kombusubrmq    | 2017-12-15 07:51:35,479 - kombu-subscriber - INFO - kombu-rabbitmq-subscriber - kombu.subscriber queues=reporting.accounts consuming with callback=handle_message\n        kombusubrmq    | 2017-12-15 07:51:45,489 - kombu-rabbitmq-subscriber - INFO - End - kombu-rabbitmq-subscriber\n        kombusubrmq exited with code 0\n\n#.  Run the RabbitMQ Publisher\n\n    ::\n\n        run_rabbitmq_publisher.py \n        2017-12-09 11:56:42,793 - run-rabbitmq-publisher - INFO - Start - run-rabbitmq-publisher\n        2017-12-09 11:56:42,793 - run-rabbitmq-publisher - INFO - Sending msg={'account_id': 456, 'created': '2017-12-09T11:56:42.793819'} ex=reporting rk=reporting.accounts\n        2017-12-09 11:56:42,812 - rabbitmq-publisher - INFO - SEND - exch=reporting rk=reporting.accounts\n        2017-12-09 11:56:42,814 - run-rabbitmq-publisher - INFO - End - run-rabbitmq-publisher sent=True\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-run-rabbitmq-publisher.yml up\n        Starting kombupubrmq ... done\n        Attaching to kombupubrmq\n        kombupubrmq    | 2017-12-15 07:51:50,931 - run-rabbitmq-publisher - INFO - Start - run-rabbitmq-publisher\n        kombupubrmq    | 2017-12-15 07:51:50,932 - run-rabbitmq-publisher - INFO - Sending msg={'account_id': 456, 'created': '2017-12-15T07:51:50.932501'} ex=reporting rk=reporting.accounts\n        kombupubrmq    | 2017-12-15 07:51:50,958 - kombu-publisher - INFO - SEND - exch=reporting rk=reporting.accounts\n        kombupubrmq    | 2017-12-15 07:51:50,960 - run-rabbitmq-publisher - INFO - End - run-rabbitmq-publisher sent=True\n        kombupubrmq exited with code 0\n\n#.  Run the RabbitMQ Kombu Subscriber\n\n    By default, this will wait for a single message to be delivered within 10 seconds.\n\n    ::\n\n        kombu_rabbitmq_subscriber.py \n        2017-12-09 11:57:07,047 - kombu-rabbitmq-subscriber - INFO - Start - kombu-rabbitmq-subscriber\n        2017-12-09 11:57:07,047 - kombu-rabbitmq-subscriber - INFO - setup routing\n        2017-12-09 11:57:07,103 - kombu-rabbitmq-subscriber - INFO - kombu-rabbitmq-subscriber - kombu.subscriber queues=reporting.accounts consuming with callback=handle_message\n        2017-12-09 11:57:07,104 - kombu-rabbitmq-subscriber - INFO - callback received msg body={u'account_id': 456, u'created': u'2017-12-09T11:56:42.793819'}\n        2017-12-09 11:57:07,104 - kombu-rabbitmq-subscriber - INFO - End - kombu-rabbitmq-subscriber\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-kombu-rabbitmq-subscriber.yml up\n        Starting kombusubrmq ... done\n        Attaching to kombusubrmq\n        kombusubrmq    | 2017-12-15 07:51:55,366 - kombu-rabbitmq-subscriber - INFO - Start - kombu-rabbitmq-subscriber\n        kombusubrmq    | 2017-12-15 07:51:55,367 - kombu-subscriber - INFO - setup routing\n        kombusubrmq    | 2017-12-15 07:51:55,422 - kombu-subscriber - INFO - kombu-rabbitmq-subscriber - kombu.subscriber queues=reporting.accounts consuming with callback=handle_message\n        kombusubrmq    | 2017-12-15 07:51:55,423 - kombu-rabbitmq-subscriber - INFO - callback received msg body={'account_id': 456, 'created': '2017-12-15T07:51:50.932501'}\n        kombusubrmq    | 2017-12-15 07:51:55,424 - kombu-rabbitmq-subscriber - INFO - End - kombu-rabbitmq-subscriber\n        kombusubrmq exited with code 0\n\nRunning a Redis Message Processor\n=================================\n\nThis will simulate setting up a processor that handles user conversion events using a Redis server.\n\n#.  Start the User Conversion Event Processor\n\n    ::\n\n        start-kombu-message-processor-redis.py \n        2017-12-09 12:09:14,329 - loader-name - INFO - Start - msg-proc\n        2017-12-09 12:09:14,329 - msg-proc - INFO - msg-proc START - consume_queue=user.events.conversions rk=None\n        2017-12-09 12:09:14,329 - msg-sub - INFO - setup routing\n        2017-12-09 12:09:14,351 - msg-sub - INFO - msg-sub - kombu.subscriber queues=user.events.conversions consuming with callback=process_message\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-kombu-message-processor-redis.yml up\n        Creating kombumsgprocredis ... done\n        Attaching to kombumsgprocredis\n        kombumsgprocredis    | 2017-12-15 07:54:24,167 - loader-name - INFO - Start - msg-proc\n        kombumsgprocredis    | 2017-12-15 07:54:24,168 - message-processor - INFO - msg-proc START - consume_queue=user.events.conversions rk=None callback=process_message\n\n#.  Publish a User Conversion Event\n\n    From another terminal, publish a user conversion event\n\n    ::\n\n        publish-user-conversion-events-redis.py \n        2017-12-09 12:09:16,557 - publish-user-conversion-events - INFO - Start - publish-user-conversion-events\n        2017-12-09 12:09:16,558 - publish-user-conversion-events - INFO - Sending user conversion event msg={'subscription_id': 456, 'created': '2017-12-09T12:09:16.558462', 'stripe_id': 789, 'account_id': 123, 'product_id': 'ABC'} ex=user.events rk=user.events.conversions\n        2017-12-09 12:09:16,582 - publish-uce-redis - INFO - SEND - exch=user.events rk=user.events.conversions\n        2017-12-09 12:09:16,585 - publish-user-conversion-events - INFO - End - publish-user-conversion-events sent=True\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-publish-user-conversion-events-redis.yml up\n        WARNING: Found orphan containers (kombumsgprocredis) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.\n        Creating ucepubredis ... done\n        Attaching to ucepubredis\n        ucepubredis    | 2017-12-15 07:54:40,539 - publish-user-conversion-events - INFO - Start - publish-user-conversion-events\n        ucepubredis    | 2017-12-15 07:54:40,539 - publish-user-conversion-events - INFO - Sending user conversion event msg={'account_id': 123, 'subscription_id': 456, 'stripe_id': 789, 'product_id': 'ABC', 'created': '2017-12-15T07:54:40.539324'} ex=user.events rk=user.events.conversions\n        ucepubredis    | 2017-12-15 07:54:40,619 - kombu-publisher - INFO - SEND - exch=user.events rk=user.events.conversions\n        ucepubredis    | 2017-12-15 07:54:40,623 - publish-user-conversion-events - INFO - End - publish-user-conversion-events sent=True\n        ucepubredis exited with code 0\n\n#.  Confirm the Processor handled the conversion event\n\n    ::\n\n        2017-12-09 12:09:16,587 - msg-proc - INFO - msg-proc proc start - msg body={u'subscription_id': 456, u'product_id': u'ABC', u'stripe_id': 789, u'account_id': 123, u'created': u'2017-12-09T12:09:16.558462'}\n        2017-12-09 12:09:16,587 - msg-proc - INFO - No auto-caching or pub-hook set exchange=None\n        2017-12-09 12:09:16,588 - msg-proc - INFO - msg-proc proc done - msg\n\n    Or with the docker compose version should log:\n\n    ::\n\n        kombumsgprocredis    | 2017-12-15 07:54:24,167 - loader-name - INFO - Start - msg-proc\n        kombumsgprocredis    | 2017-12-15 07:54:24,168 - message-processor - INFO - msg-proc START - consume_queue=user.events.conversions rk=None callback=process_message\n        kombumsgprocredis    | 2017-12-15 07:54:24,168 - kombu-subscriber - INFO - setup routing\n        kombumsgprocredis    | 2017-12-15 07:54:40,625 - message-processor - INFO - msg-proc proc start - msg body={'account_id': 123, 'subscription_id': 456, 'stripe_id': 789, 'product_id': 'ABC', 'created': '2017-12-15T07:54:40.539324'}\n        kombumsgprocredis    | 2017-12-15 07:54:40,627 - message-processor - INFO - No auto-caching or pub-hook set exchange=None\n\n#.  Check the Redis keys for the new User Conversion Events key\n\n    ::\n\n        redis-cli \n        127.0.0.1:6379> keys *\n        1) \"_kombu.binding.reporting.accounts\"\n        2) \"_kombu.binding.user.events\"\n        3) \"_kombu.binding.celery.redis.sub\"\n        4) \"_kombu.binding.user.events.conversions\"\n        127.0.0.1:6379> \n\nRun a Message Processor from RabbitMQ with Relay Publish Hook to Redis\n======================================================================\n\nThis could also be set up for auto-caching instead of this pub-sub flow because this delivers a post-processing json dictionary into a Redis key (publish hook), and let's be honest Redis is great at caching all the datas.\n\n#.  Clear out the ``reporting.accounts`` Redis key\n\n    Either run ``kombu_redis_subscriber.py`` until there's no more messages being consumed or you can restart the docker containers with the ``stop-redis-and-rabbitmq.sh`` and ``start-redis-and-rabbitmq.sh``, but the point is verify there's nothing in the ``reporting.accounts`` key (could just delete it with the ``redis-cli``).\n\n#.  Start the Kombu RabbitMQ Message Processor\n\n    ::\n\n        start-kombu-message-processor-rabbitmq.py\n        2017-12-09 12:25:09,962 - loader-name - INFO - Start - msg-proc\n        2017-12-09 12:25:09,962 - msg-proc - INFO - msg-proc START - consume_queue=user.events.conversions rk=reporting.accounts\n        2017-12-09 12:25:09,962 - msg-sub - INFO - setup routing\n        2017-12-09 12:25:09,987 - msg-sub - INFO - msg-sub - kombu.subscriber queues=user.events.conversions consuming with callback=process_message\n\n    Docker compose can start this too:\n\n    ::\n\n        docker stop worker;docker rm worker;\n        docker-compose -f compose-kombu-message-processor-rabbitmq.yml up\n\n#.  Send a User Conversion Event to RabbitMQ\n\n    ::\n\n        publish-user-conversion-events-rabbitmq.py\n        2017-12-09 12:25:35,167 - publish-user-conversion-events - INFO - Start - publish-user-conversion-events\n        2017-12-09 12:25:35,167 - publish-user-conversion-events - INFO - Sending user conversion event msg={'subscription_id': 888, 'created': '2017-12-09T12:25:35.167891', 'stripe_id': 999, 'account_id': 777, 'product_id': 'XYZ'} ex=user.events rk=user.events.conversions\n        2017-12-09 12:25:35,185 - publish-uce-rabbitmq - INFO - SEND - exch=user.events rk=user.events.conversions\n        2017-12-09 12:25:35,187 - publish-user-conversion-events - INFO - End - publish-user-conversion-events sent=True\n\n#.  Verify the Kombu RabbitMQ Message Processor Handled the Message\n\n    Notice the ``pub-hook`` shows the relay-specific log lines\n\n    ::\n\n        2017-12-09 12:25:35,188 - msg-proc - INFO - msg-proc proc start - msg body={u'subscription_id': 888, u'product_id': u'XYZ', u'stripe_id': 999, u'account_id': 777, u'created': u'2017-12-09T12:25:35.167891'}\n        2017-12-09 12:25:35,188 - msg-proc - INFO - msg-proc pub-hook - build - hook msg body\n        2017-12-09 12:25:35,188 - msg-proc - INFO - msg-proc pub-hook - send - exchange=reporting.accounts rk=reporting.accounts sz=json\n        2017-12-09 12:25:35,210 - msg-pub - INFO - SEND - exch=reporting.accounts rk=reporting.accounts\n        2017-12-09 12:25:35,212 - msg-proc - INFO - msg-proc pub-hook - send - done exchange=reporting.accounts rk=reporting.accounts res=True\n        2017-12-09 12:25:35,212 - msg-proc - INFO - msg-proc proc done - msg\n\n#.  Process the Redis ``reporting.accounts`` queue\n\n    This could also be cached data about the user that made this purchase like a write-through-cache.\n\n    ::\n\n        kombu_redis_subscriber.py \n        2017-12-09 12:26:21,846 - kombu-redis-subscriber - INFO - Start - kombu-redis-subscriber\n        2017-12-09 12:26:21,846 - kombu-redis-subscriber - INFO - setup routing\n        2017-12-09 12:26:21,867 - kombu-redis-subscriber - INFO - kombu-redis-subscriber - kombu.subscriber queues=reporting.accounts consuming with callback=handle_message\n        2017-12-09 12:26:21,869 - kombu-redis-subscriber - INFO - callback received msg body={u'data': {}, u'org_msg': {u'subscription_id': 888, u'created': u'2017-12-09T12:25:35.167891', u'stripe_id': 999, u'product_id': u'XYZ', u'account_id': 777}, u'hook_created': u'2017-12-09T12:25:35.188420', u'version': 1, u'source': u'msg-proc'}\n        2017-12-09 12:26:21,870 - kombu-redis-subscriber - INFO - End - kombu-redis-subscriber\n\nSQS - Experimental\n==================\n\nI have opened a PR for fixing the kombu http client.\n\n#.  Export your AWS Key and Secret Key\n\n    ::\n\n        export SQS_AWS_ACCESS_KEY=<ACCESS KEY>\n        export SQS_AWS_SECRET_KEY=<SECRET KEY>\n\n#.  Publish to SQS\n\n    ::\n\n        kombu_sqs_publisher.py \n        2017-12-09 12:49:24,900 - kombu-sqs-publisher - INFO - Start - kombu-sqs-publisher\n        2017-12-09 12:49:24,901 - kombu-sqs-publisher - INFO - Sending user conversion event msg={'subscription_id': 222, 'product_id': 'DEF', 'stripe_id': 333, 'account_id': 111, 'created': '2017-12-09T12:49:24.901513'} ex=test1 rk=test1\n        2017-12-09 12:49:25,007 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): queue.amazonaws.com\n        2017-12-09 12:49:25,538 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): queue.amazonaws.com\n        2017-12-09 12:49:26,237 - kombu-sqs-publisher - INFO - SEND - exch=test1 rk=test1\n        2017-12-09 12:49:26,352 - kombu-sqs-publisher - INFO - End - kombu-sqs-publisher sent=True\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-kombu-sqs-publisher.yml up\n\n#.  Subscribe to SQS\n\n    Please see the debugging section for getting this to function with kombu 4.1.0 \n\n    https://github.com/jay-johnson/celery-connectors#temporary-fix-for-kombu-sqs\n\n    ::\n\n        kombu_sqs_subscriber.py \n        2017-12-09 12:49:41,232 - kombu-sqs-subscriber - INFO - Start - kombu-sqs-subscriber\n        2017-12-09 12:49:41,232 - kombu-sqs-subscriber - INFO - setup routing\n        2017-12-09 12:49:41,333 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): queue.amazonaws.com\n        2017-12-09 12:49:41,801 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): queue.amazonaws.com\n        2017-12-09 12:49:42,517 - kombu-sqs-subscriber - INFO - kombu-sqs-subscriber - kombu.subscriber queues=test1 consuming with callback=handle_message\n        2017-12-09 12:49:42,671 - kombu-sqs-subscriber - INFO - callback received msg body={u'subscription_id': 222, u'created': u'2017-12-09T12:49:24.901513', u'stripe_id': 333, u'product_id': u'DEF', u'account_id': 111}\n        2017-12-09 12:49:42,773 - kombu-sqs-subscriber - INFO - End - kombu-sqs-subscriber\n\n    Or with docker compose:\n\n    ::\n\n        docker-compose -f compose-kombu-sqs-subscriber.yml up\n\n#.  Verify the SQS Queue ``test1`` is empty\n\n    ::\n\n        aws sqs receive-message --queue-url https://queue.amazonaws.com/<YOUR QUEUE ID>/test1\n        echo $?\n        0\n\nSimple Pub Sub with an Existing Celery Task\n===========================================\n\nStart the Celery Worker as an Ecommerce Subscriber\n--------------------------------------------------\n\nPlease run this from the base directory of the repository in a terminal that has sourced the virtual env: ``source venv/bin/activate``.\n\n::\n\n    ./start-ecomm-worker.sh \n\n    -------------- celery@ecommerce_subscriber v4.1.0 (latentcall)\n    ---- **** ----- \n    --- * ***  * -- Linux-4.7.4-200.fc24.x86_64-x86_64-with-fedora-24-Twenty_Four 2017-12-10 15:12:11\n    -- * - **** --- \n    - ** ---------- [config]\n    - ** ---------- .> app:         ecommerce-worker:0x7fae3cfa2198\n    - ** ---------- .> transport:   amqp://rabbitmq:**@localhost:5672//\n    - ** ---------- .> results:     rpc://\n    - *** --- * --- .> concurrency: 4 (prefork)\n    -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)\n    --- ***** ----- \n    -------------- [queues]\n                    .> celery           exchange=celery(direct) key=celery\n\n\n    [tasks]\n    . ecomm_app.ecommerce.tasks.handle_user_conversion_events\n\n    [2017-12-10 15:12:11,727: INFO/MainProcess] Connected to amqp://rabbitmq:**@127.0.0.1:5672//\n    [2017-12-10 15:12:11,740: INFO/MainProcess] mingle: searching for neighbors\n    [2017-12-10 15:12:12,776: INFO/MainProcess] mingle: all alone\n    [2017-12-10 15:12:12,828: INFO/MainProcess] celery@ecommerce_subscriber ready.\n    [2017-12-10 15:12:13,633: INFO/MainProcess] Events of group {task} enabled by remote.\n\n\nPublish User Conversion Events to the Celery Ecommerce Subscriber\n-----------------------------------------------------------------\n\nPlease run this from a separate terminal that has sourced the virtual env: ``source venv/bin/activate``.\n\n#.  Change to the ``ecomm_app`` directory\n\n    ::\n\n        cd ecomm_app\n\n#.  Publish a task\n\n    This will use the Celery ``send_task`` method to publish the Celery task: ``ecomm_app.ecommerce.tasks.handle_user_conversion_events`` to RabbitMQ which is monitored by the Celery ecommerce worker.\n\n    ::\n\n        ./publish_task.py \n        INFO:celery-task-publisher:Sending broker=amqp://rabbitmq:rabbitmq@localhost:5672// body={'subscription_id': 321, 'msg_id': '6d7ab602-f7cd-4d90-a0c5-5eb0cdcb41d9', 'version': 1, 'product_id': 'JJJ', 'account_id': 999, 'stripe_id': 876, 'created': '2017-12-10T15:16:08.557804'}\n        INFO:celery-task-publisher:Done with msg_id=6d7ab602-f7cd-4d90-a0c5-5eb0cdcb41d9 result=True\n\nConfirm the Celery Worker Processed the Conversion Message\n----------------------------------------------------------\n\n::\n\n    [2017-12-10 15:16:08,593: INFO/MainProcess] Received task: ecomm_app.ecommerce.tasks.handle_user_conversion_events[9349e1be-fca5-40b5-86d3-0661fdd9fd06]  \n    [2017-12-10 15:16:08,594: INFO/ForkPoolWorker-4] task - user_conversion_events - start body={'stripe_id': 876, 'version': 1, 'subscription_id': 321, 'created': '2017-12-10T15:16:08.557804', 'account_id': 999, 'product_id': 'JJJ', 'msg_id': '6d7ab602-f7cd-4d90-a0c5-5eb0cdcb41d9'}\n    [2017-12-10 15:16:08,595: INFO/ForkPoolWorker-4] task - user_conversion_events - done\n    [2017-12-10 15:16:08,619: INFO/ForkPoolWorker-4] Task ecomm_app.ecommerce.tasks.handle_user_conversion_events[9349e1be-fca5-40b5-86d3-0661fdd9fd06] succeeded in 0.025004414000250108s: True\n\nCheck the Ecommerce Subscriber in Flower\n----------------------------------------\n\nThe Ecommerce Publisher and Subscriber are using RabbitMQ which is registered under the Flower url:\n\nhttp://localhost:5555/ - (login: admin/admin)\n\nThere should be a Worker named:\n\n::\n\n    celery@ecommerce_subscriber\n\nThere are also additional worker details available at:\n\nhttp://localhost:5555/worker/celery@ecommerce_subscriber\n\nView the registered ecommerce tasks for the worker:\n\nhttp://localhost:5555/worker/celery@ecommerce_subscriber#tab-tasks\n\nDebugging with rabbitmqadmin\n=============================\n\nThe pip and development build will install ``rabbitmqadmin`` (https://raw.githubusercontent.com/rabbitmq/rabbitmq-management/v3.7.0/bin/rabbitmqadmin) version 3.7.0. It is a great utility for verifying RabbitMQ messaging and does not require having access to the RabbitMQ cluster's host nodes (or a machine with rabbitmqctl on it).\n\nPlease note: ``rabbitmqadmin`` uses the management HTTP port (not the amqp port 5672) which requires a broker to have the management plugin enabled to work if you're using this with an external RabbitMQ cluster.\n\nChecking queues\n---------------\n\nScript in pip\n\n::\n\n    list-queues.sh \n\n    Listing Queues broker=localhost:15672\n\n    +--------------------+-----------+----------+----------------+-------------------------+\n    |        name        | consumers | messages | messages_ready | messages_unacknowledged |\n    +--------------------+-----------+----------+----------------+-------------------------+\n    | celery             | 0         | 0        | 0              | 0                       |\n    | reporting.accounts | 0         | 0        | 0              | 0                       |\n    +--------------------+-----------+----------+----------------+-------------------------+\n\nManual way\n\n::\n\n    rabbitmqadmin.py --host=localhost --port=15672 --username=rabbitmq --password=rabbitmq list queues\n    +--------------------+-----------+----------+----------------+-------------------------+\n    |        name        | consumers | messages | messages_ready | messages_unacknowledged |\n    +--------------------+-----------+----------+----------------+-------------------------+\n    | celery             | 0         | 0        | 0              | 0                       |\n    | reporting.accounts | 0         | 0        | 0              | 0                       |\n    +--------------------+-----------+----------+----------------+-------------------------+\n\nChecking exchanges\n------------------\n\nScript in pip\n\n::\n\n    list-exchanges.sh \n\n    Listing Exchanges broker=localhost:15672\n\n    +---------------------+---------+\n    |        name         |  type   |\n    +---------------------+---------+\n    |                     | direct  |\n    | amq.direct          | direct  |\n    | amq.fanout          | fanout  |\n    | amq.headers         | headers |\n    | amq.match           | headers |\n    | amq.rabbitmq.log    | topic   |\n    | amq.rabbitmq.trace  | topic   |\n    | amq.topic           | topic   |\n    | celery              | direct  |\n    | celery.pidbox       | fanout  |\n    | celeryev            | topic   |\n    | reply.celery.pidbox | direct  |\n    | reporting.accounts  | topic   |\n    +---------------------+---------+\n\nManual way\n\n::\n\n    rabbitmqadmin.py --host=localhost --port=15672 --username=rabbitmq --password=rabbitmq list exchanges name typa\n    +---------------------+---------+\n    |        name         |  type   |\n    +---------------------+---------+\n    |                     | direct  |\n    | amq.direct          | direct  |\n    | amq.fanout          | fanout  |\n    | amq.headers         | headers |\n    | amq.match           | headers |\n    | amq.rabbitmq.log    | topic   |\n    | amq.rabbitmq.trace  | topic   |\n    | amq.topic           | topic   |\n    | celery              | direct  |\n    | celery.pidbox       | fanout  |\n    | celeryev            | topic   |\n    | reply.celery.pidbox | direct  |\n    | reporting.accounts  | topic   |\n    +---------------------+---------+\n\nList Bindings\n=============\n\nScript in pip\n\n::\n\n    list-bindings.sh \n\n    Listing Bindings broker=localhost:15672\n\n    +--------------------+--------------------+--------------------+\n    |       source       |    destination     |    routing_key     |\n    +--------------------+--------------------+--------------------+\n    |                    | celery             | celery             |\n    |                    | reporting.accounts | reporting.accounts |\n    | celery             | celery             | celery             |\n    | reporting          | reporting.accounts | reporting.accounts |\n    +--------------------+--------------------+--------------------+\n\nManual way\n\n::\n\n    rabbitmqadmin.py --host=localhost --port=15672 --username=rabbitmq --password=rabbitmq list bindings source destination routing_key\n    +--------------------+--------------------+--------------------+\n    |       source       |    destination     |    routing_key     |\n    +--------------------+--------------------+--------------------+\n    |                    | celery             | celery             |\n    |                    | reporting.accounts | reporting.accounts |\n    | celery             | celery             | celery             |\n    | reporting          | reporting.accounts | reporting.accounts |\n    +--------------------+--------------------+--------------------+\n\nDevelopment Guide\n=================\n\n#.  Install the development environment\n\n    ::\n\n        virtualenv -p python3 venv && source venv/bin/activate && pip install -e .\n\n#.  Run tests\n\n    The tests require the docker containers to be running prior to starting.\n\n    ::\n\n        python setup.py test\n\nDebugging\n=========\n\npycURL Reinstall with NSS\n-------------------------\n\nFor anyone wanting to use kombu SQS, I had to uninstall ``pycurl`` and install it with ``nss``.\n\nThe error looked like this in the logs:\n\n::\n\n    2017-12-09 12:28:46,811 - kombu-sqs-subscriber - INFO - kombu-sqs-subscriber - kombu.subscriber consume hit exception=The curl client requires the pycurl library. queue=test1\n\n\nSo I opened up a python shell\n\nPython 2:\n\n::\n\n    $ python\n    Python 2.7.12 (default, Sep 29 2016, 13:30:34) \n    [GCC 6.2.1 20160916 (Red Hat 6.2.1-2)] on linux2\n    Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n    >>> import pycurl\n    Traceback (most recent call last):\n    File \"<stdin>\", line 1, in <module>\n    File \"build/bdist.linux-x86_64/egg/pycurl.py\", line 7, in <module>\n    File \"build/bdist.linux-x86_64/egg/pycurl.py\", line 6, in __bootstrap__\n    ImportError: pycurl: libcurl link-time ssl backend (nss) is different from compile-time ssl backend (none/other)\n    >>> \n\nPython 3:\n\n::\n\n    $ python\n    Python 3.5.3 (default, May 11 2017, 09:10:41) \n    [GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux\n    Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n    >>> import pycurl\n    Traceback (most recent call last):\n        File \"<stdin>\", line 1, in <module>\n    ImportError: pycurl: libcurl link-time ssl backend (nss) is different from compile-time ssl backend (none/other)\n    >>> \n\nUninstalled and Reinstalled pycurl with nss\n\n::\n\n    pip uninstall -y pycurl; pip install pycurl --compile --global-option=\"--with-nss\" pycurl\n\nTemporary fix for Kombu SQS\n---------------------------\n\nSQS Kombu Subscriber ``'NoneType' object has no attribute 'call_repeatedly'``\n\nUntil Kombu fixes the SQS transport and publishes it to pypi, the SQS subscriber will throw exceptions like below.\n\n::\n\n    kombu_sqs_subscriber.py \n    2017-12-09 12:30:45,493 - kombu-sqs-subscriber - INFO - Start - kombu-sqs-subscriber\n    2017-12-09 12:30:45,493 - kombu-sqs-subscriber - INFO - setup routing\n    2017-12-09 12:30:45,602 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): queue.amazonaws.com\n    2017-12-09 12:30:46,046 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): queue.amazonaws.com\n    2017-12-09 12:30:46,832 - kombu-sqs-subscriber - INFO - kombu-sqs-subscriber - kombu.subscriber queues=test1 consuming with callback=handle_message\n    2017-12-09 12:30:46,989 - kombu-sqs-subscriber - INFO - callback received msg body={u'subscription_id': 222, u'created': u'2017-12-09T12:28:28.093582', u'stripe_id': 333, u'product_id': u'DEF', u'account_id': 111}\n    2017-12-09 12:30:46,994 - kombu-sqs-subscriber - INFO - kombu-sqs-subscriber - kombu.subscriber consume hit exception='NoneType' object has no attribute 'call_repeatedly' queue=test1\n    2017-12-09 12:30:46,994 - kombu-sqs-subscriber - INFO - End - kombu-sqs-subscriber\n    Restoring 1 unacknowledged message(s)\n\nNotice the last line has put the message into SQS in-flight which means it has not been acknowledged or deleted.\n\nYou can verify this message is still there with the aws cli:\n\n::\n\n    aws sqs receive-message --queue-url https://queue.amazonaws.com/<YOUR QUEUE ID>/test1\n    {\n        \"Messages\": [\n            {\n                \"Body\": \"eyJib2R5IjogImV5SnpkV0p6WTNKcGNIUnBiMjVmYVdRaU9pQXlNaklzSUNKd2NtOWtkV04wWDJsa0lqb2dJa1JGUmlJc0lDSnpkSEpwY0dWZmFXUWlPaUF6TXpNc0lDSmhZMk52ZFc1MFgybGtJam9nTVRFeExDQWlZM0psWVhSbFpDSTZJQ0l5TURFM0xURXlMVEE1VkRFeU9qVXpPakEwTGpjME9UY3lOaUo5IiwgImhlYWRlcnMiOiB7fSwgImNvbnRlbnQtdHlwZSI6ICJhcHBsaWNhdGlvbi9qc29uIiwgInByb3BlcnRpZXMiOiB7InByaW9yaXR5IjogMCwgImJvZHlfZW5jb2RpbmciOiAiYmFzZTY0IiwgImRlbGl2ZXJ5X2luZm8iOiB7InJvdXRpbmdfa2V5IjogInRlc3QxIiwgImV4Y2hhbmdlIjogInRlc3QxIn0sICJkZWxpdmVyeV9tb2RlIjogMiwgImRlbGl2ZXJ5X3RhZyI6ICJkOGI3MjNiMi05MDVkLTQxZTEtODVlNy00NjUwZGY2NWU2MTgifSwgImNvbnRlbnQtZW5jb2RpbmciOiAidXRmLTgifQ==\", \n                \"ReceiptHandle\": \"AQEBDnxqT1+SOam1ZtMKPgh77a8bapLbcrI3PZRTqVZJokz0h7oMusuJPAB9jksH3BQHQyg3TyZXasBblpMcin3HTzh7ykTgAgawhMreOoWGGiaeEoOekaChn2yFpKDbVP1ZENRVcpAzeDXzCd52TITZbyLk8FY1PJB3XpAiih9SH/R0FPj3JnU0WTxjTAWtBnSlUUGXFc3CczJi61YsJS+bTZs8JIgDaICMF+zMhnV+rV4zXDObTVFM3OaMdf/puqZ9yRd3fM1GsOxZaDNRDGYKml/UK0tn32gtqPSuUW905YamwnWQYB9mF338Jgx11rv78b5lLogpU/0t6E+0tD1Lkr/UR/M64NZI2eTwp6ZHNtqTNbkjd5VsBgB39b+wXFFn\", \n                \"MD5OfBody\": \"e72609877b90ad86df2f161c6303eaf0\", \n                \"MessageId\": \"684328b4-a38c-4868-8550-e0d46599a0c2\"\n            }\n        ]\n    }\n\nIf you're feeling bold, you can run off my PR fix branch as well:\n\n::\n\n    pip uninstall -y kombu ; rm -rf /tmp/sqs-pr-fix-with-kombu; git clone https://github.com/jay-johnson/kombu.git /tmp/sqs-pr-fix-with-kombu && pushd /tmp/sqs-pr-fix-with-kombu && git checkout sqs-http-get-client && python setup.py develop && popd\n\nWith the SQS fix applied locally (works on python 2 and 3 on my fedora 24 vm):\n\n::\n\n    2017-12-09 12:47:12,177 - kombu-sqs-subscriber - INFO - Start - kombu-sqs-subscriber\n    2017-12-09 12:47:12,177 - kombu-sqs-subscriber - INFO - setup routing\n    2017-12-09 12:47:12,295 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): queue.amazonaws.com\n    2017-12-09 12:47:12,736 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): queue.amazonaws.com\n    2017-12-09 12:47:13,454 - kombu-sqs-subscriber - INFO - kombu-sqs-subscriber - kombu.subscriber queues=test1 consuming with callback=handle_message\n    2017-12-09 12:47:13,592 - kombu-sqs-subscriber - INFO - callback received msg body={u'subscription_id': 222, u'created': u'2017-12-09T12:28:28.093582', u'stripe_id': 333, u'product_id': u'DEF', u'account_id': 111}\n    2017-12-09 12:47:13,689 - kombu-sqs-subscriber - INFO - End - kombu-sqs-subscriber\n\nAfter running it you can confirm the message has been deleted and acknowledged with the aws cli:\n\n::\n\n    aws sqs receive-message --queue-url https://queue.amazonaws.com/<YOUR QUEUE ID>/test1\n    echo $?\n    0\n\nTesting\n=======\n\nStart the Relay\n---------------\n\n::\n\n    ./start-ecomm-relay.py\n\nStart the Celery Worker\n-----------------------\n\n::\n\n    ./start-ecomm-worker.sh\n\nLoad Test Celery Worker over RabbitMQ\n-------------------------------------\n\nThis will send 50,000 messages over with the Celery ``send_task`` method. As long as the ecomm Celery worker is running the messages will be sent over.\n\n::\n\n    python -m unittest tests/load_test_worker_rabbitmq.py\n\nLoad Test Relay\n---------------\n\nThis will send 50,000 messages over the ``user.events.conversions`` RabbitMQ queue for the ecomm relay to process and then send to the ecomm worker.\n\n::\n\n    python -m unittest tests/load_test_relay_rabbitmq.py\n\nCleanup Persistence\n-------------------\n\nDocker compose creates files and directories as the host's ``root`` user. This makes cleaning up test runs annoying. So here's a tool to clean the persistence data and logs but it requires providing ``sudo`` or running as ``root``.\n\n**Please, please, please be careful!**\n\n#.  Shut them down to prevent writes to the volumes\n\n    ::\n\n        stop-redis-and-rabbitmq.sh\n\n#.  Clean them up\n\n    ::\n\n        sudo ./clean-persistence-data.sh \n\n        Using root to delete persistence directories: ./docker/data/rabbitmq/ ./docker/data/redis and logs: ./docker/logs/rabbitmq ./docker/logs/redis and files: ./docker/data/rabbitmq/.erlang.cookie\n\n        - deleting=./docker/data/rabbitmq\n        - deleting=./docker/logs/rabbitmq\n        - deleting=./docker/data/redis\n        - deleting=./docker/logs/redis\n        - deleting=./docker/data/rabbitmq/.erlang.cookie\n\n#.  Start them up again\n\n    ::\n\n        start-persistence-containers.sh\n\nCreate your own self-signed Keys, Certs and Certificate Authority with Ansible\n------------------------------------------------------------------------------\n\nIf you have openssl installed you can use this ansible playbook to create your own certificate authority (CA), keys and certs.\n\n#.  Create the CA, Keys and Certificates\n\n    ::\n\n        cd ansible\n        ansible-playbook -i inventory_dev create-x509s.yml\n\n#.  Verify the Jupyter Client Cert\n\n    ::\n\n        openssl x509 -in ../compose/ssl/client_cert.pem -text -noout\n\n#.  Verify the Jupyter Server Cert\n\n    ::\n\n        openssl x509 -in ../compose/ssl/jupyter_server_cert.pem -text -noout\n\n#.  Using the certs\n\n    Docker makes testing ssl easier so the certs are created under the ``compose/ssl`` directory:\n\n    ::\n\n        tree ../compose/ssl\n        \u251c\u2500\u2500 ca.pem\n        \u251c\u2500\u2500 ca_private_key.pem\n        \u251c\u2500\u2500 client_cert.pem\n        \u251c\u2500\u2500 client.csr\n        \u251c\u2500\u2500 client_key.pem\n        \u251c\u2500\u2500 database_server_cert.pem\n        \u251c\u2500\u2500 database_server.csr\n        \u251c\u2500\u2500 database_server_key.pem\n        \u251c\u2500\u2500 docker_server_cert.pem\n        \u251c\u2500\u2500 docker_server.csr\n        \u251c\u2500\u2500 docker_server_key.pem\n        \u251c\u2500\u2500 extfile.cnf\n        \u251c\u2500\u2500 jenkins_server_cert.pem\n        \u251c\u2500\u2500 jenkins_server.csr\n        \u251c\u2500\u2500 jenkins_server_key.pem\n        \u251c\u2500\u2500 jupyter_server_cert.pem\n        \u251c\u2500\u2500 jupyter_server.csr\n        \u251c\u2500\u2500 jupyter_server_key.pem\n        \u251c\u2500\u2500 kibana_server_cert.pem\n        \u251c\u2500\u2500 kibana_server.csr\n        \u251c\u2500\u2500 kibana_server_key.pem\n        \u251c\u2500\u2500 nginx_server_cert.pem\n        \u251c\u2500\u2500 nginx_server.csr\n        \u251c\u2500\u2500 nginx_server_key.pem\n        \u251c\u2500\u2500 pgadmin_server_cert.pem\n        \u251c\u2500\u2500 pgadmin_server.csr\n        \u251c\u2500\u2500 pgadmin_server_key.pem\n        \u251c\u2500\u2500 phpmyadmin_server_cert.pem\n        \u251c\u2500\u2500 phpmyadmin_server.csr\n        \u251c\u2500\u2500 phpmyadmin_server_key.pem\n        \u251c\u2500\u2500 rabbitmq_server_cert.pem\n        \u251c\u2500\u2500 rabbitmq_server.csr\n        \u251c\u2500\u2500 rabbitmq_server_key.pem\n        \u251c\u2500\u2500 redis_server_cert.pem\n        \u251c\u2500\u2500 redis_server.csr\n        \u251c\u2500\u2500 redis_server_key.pem\n        \u251c\u2500\u2500 restapi_server_cert.pem\n        \u251c\u2500\u2500 restapi_server.csr\n        \u251c\u2500\u2500 restapi_server_key.pem\n        \u251c\u2500\u2500 webserver_server_cert.pem\n        \u251c\u2500\u2500 webserver_server.csr\n        \u2514\u2500\u2500 webserver_server_key.pem\n\n\n#.  Set up your own extfile.cnf - Optional\n\n    You can change the source ``extfile.cnf`` which is copied over to the ``compose/ssl`` directory when the playbook runs as needed.\n\n    ::\n\n        cat ./configs/extfile.cnf \n        subjectAltName = DNS:*.localdev.com, DNS:rabbitmq.localdev.com, DNS:redis.localdev.com, DNS:jupyter.localdev.com, DNS:jenkins.localdev.com, DNS:www.localdev.com, DNS:api.localdev.com, DNS:db.localdev.com, DNS:pgadmin.localdev.com, DNS:phpmyadmin.localdev.com, DNS:kibana.localdev.com, DNS:lb.localdev.com, DNS:docker.localdev.com, IP:127.0.0.1\n        extendedKeyUsage = serverAuth\n\n\n#.  Customizing your own openssl.cnf and cert_openssl.cnf - Optional\n\n    You can change the source ``openssl.cnf`` before creating the certs.\n\n    ::\n\n        cat ./configs/openssl.cnf \n        [ req ]\n        prompt              = no\n        default_bits        = 2048\n        distinguished_name  = req_distinguished_name # where to get DN for reqs\n\n        [ req_distinguished_name ]\n        C  = US\n        ST = WA\n        L  = Redmond\n        O  = SecureEverything\n        OU = SecureEverythingOrgUnit\n        CN = LocalDev\n\n    You can change the source ``cert_openssl.cnf`` before creating the certs.\n\n    ::\n\n        cat ./configs/cert_openssl.cnf \n        [req]\n        days                   = 2000\n        serial                 = 1\n        distinguished_name     = req_distinguished_name\n        x509_extensions        = v3_ca\n\n\n        [req_distinguished_name]\n        countryName            = US\n        stateOrProvinceName    = WA\n        localityName           = Redmond\n        organizationName       = SecureEverything\n        organizationalUnitName = SecureEverythingOrgUnit\n        commonName             = SecureEverything\n\n        [ v3_ca ]\n        subjectKeyIdentifier   = hash\n        authorityKeyIdentifier = keyid:always,issuer:always\n        basicConstraints       = CA:TRUE\n        keyUsage               = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment, keyAgreement, keyCertSign\n        subjectAltName         = DNS:*.localdev.com, DNS:redis.localdev.com, DNS:rabbitmq.localdev.com, DNS:jupyter.localdev.com, DNS:jenkins.localdev.com, DNS:www.localdev.com, DNS:api.localdev.com, DNS:db.localdev.com, DNS:pgadmin.localdev.com, DNS:phpmyadmin.localdev.com, DNS:kibana.localdev.com, DNS:lb.localdev.com, DNS:docker.localdev.com, email:admin@localdev.com\n        issuerAltName          = issuer:copy\n\n    I found this link helpful for understanding all the different configurable options:\n    https://www.ibm.com/support/knowledgecenter/en/SSB23S_1.1.0.13/gtps7/cfgcert.html\n\nRunning JupyterHub with Postgres and SSL\n----------------------------------------\n\n#.  Pull the default Jupyter image\n\n    All users will share this large 4.4 gb image\n\n    ::\n\n        docker pull jupyter/scipy-notebook:latest\n\n#.  Append the following entries to your ``/etc/hosts`` row with ``127.0.0.1``\n\n    ``jupyter.localdev.com rabbitmq.localdev.com redis.localdev.com jenkins.localdev.com``\n\n#.  Verify ``/etc/hosts`` has the entries\n\n    ::\n\n        cat /etc/hosts | grep localdev\n        127.0.0.1      localhost localhost.localdomain localhost4 localhost4.localdomain4 jupyter.localdev.com rabbitmq.localdev.com redis.localdev.com jenkins.localdev.com\n\n#.  From the base repository directory, change to the ``compose`` directory\n\n    ::\n\n        cd compose\n\n#.  Create the JupyterHub docker network\n\n    This should only be required if the ``jupyterhub-network`` does not already exist.\n\n    ::\n\n        docker network create jupyterhub-network\n\n#.  Create the JupyterHub docker data volume\n\n    This should only be required if the ``jupyterhub-data`` does not already exist.\n\n    ::\n\n        docker volume create --name jupyterhub-data\n\n    Each user will need a data volume if they are not already created as well with naming scheme:\n\n    ``jupyterhub-user-<username>`` to persist notebooks.\n\n    ::\n\n        docker volume create --name jupyterhub-user-admin\n\n#.  Start JupyterHub\n\n    ::\n\n        docker stop jupyterhub ; docker rm jupyterhub; docker-compose -f compose-jupyter.yml up\n\n#.  Login to JupyterHub\n\n    Please change these defaults before deploying to production:\n\n    - username: ``admin``\n    - password: ``admin``\n\n    Please accept to \"Proceed\" passed the self-signed certificate warning.\n\n    https://jupyter.localdev.com/hub/login\n\n    .. image:: https://raw.githubusercontent.com/jay-johnson/celery-connectors/master/_images/jupyterhub-step-1-login-as-admin-admin.png\n        :align: center\n\n#.  Start the Admin user Jupyter instance\n\n    Click on **Start My Server**\n\n    .. image:: https://raw.githubusercontent.com/jay-johnson/celery-connectors/master/_images/jupyterhub-step-2-start-server.png\n        :align: center\n\n#.  Clone some great notebooks into the Admin Jupyter workspace\n\n    From a terminal with access to ``docker`` clone a repository with some amazing ipython notebooks:\n\n    https://github.com/donnemartin/data-science-ipython-notebooks\n\n    ::\n\n        docker exec -it jupyter-admin git clone https://github.com/donnemartin/data-science-ipython-notebooks.git /home/jovyan/work/data-science-ipython-notebooks\n        Cloning into '/home/jovyan/work/data-science-ipython-notebooks'...\n        remote: Counting objects: 2344, done.\n        remote: Total 2344 (delta 0), reused 0 (delta 0), pack-reused 2344\n        Receiving objects: 100% (2344/2344), 47.76 MiB | 16.95 MiB/s, done.\n        Resolving deltas: 100% (1317/1317), done.\n        Checking connectivity... done.\n\n#.  Browse the cloned notebooks\n\n    https://jupyter.localdev.com/user/admin/tree/work/data-science-ipython-notebooks\n\n    .. image:: https://raw.githubusercontent.com/jay-johnson/celery-connectors/master/_images/jupyterhub-step-3-browse-ipython-notebooks.png\n        :align: center\n\n#.  Open the one of the cloned notebooks\n\n    https://jupyter.localdev.com/user/admin/notebooks/work/data-science-ipython-notebooks/scikit-learn/scikit-learn-intro.ipynb\n\n#.  Select Kernel -> Restart & Run All\n\n    Confirm you can run all the cells in the notebook\n\n#.  Verify the notebook ran all the cells without any errors\n\n    Save the output and changes to the notebook with ``ctrl + s``. At the bottom of the notebook you should see the updated chart for the ``sepal width`` and ``sepal-length`` similar to:\n\n    .. image:: https://raw.githubusercontent.com/jay-johnson/celery-connectors/master/_images/jupyterhub-step-4-run-all-notebook-cells.png\n        :align: center\n\n#.  Verify the notebook was changed and updated\n\n    Browse to:\n\n    https://jupyter.localdev.com/user/admin/tree/work/data-science-ipython-notebooks/scikit-learn\n\n    The ``scikit-learn-intro.ipynb`` should be running and updated.\n\n    .. image:: https://raw.githubusercontent.com/jay-johnson/celery-connectors/master/_images/jupyterhub-step-5-confirm-notebook-was-saved.png\n        :align: center\n\n#.  Stop the Admin Jupyter instance\n\n    The notebooks should persist a stop and start of a user's Jupyter container instance.\n\n    https://jupyter.localdev.com/hub/admin\n\n    It should look something like this:\n\n    .. image:: https://raw.githubusercontent.com/jay-johnson/celery-connectors/master/_images/jupyterhub-step-6-stop-server.png\n        :align: center\n\n#.  Start the Admin Jupyter instance again\n\n    Click ``start server``\n\n#.  Browse to the ``scikit-learn`` directory and confirm the files were not lost on the restart\n\n    https://jupyter.localdev.com/user/admin/tree/work/data-science-ipython-notebooks/scikit-learn\n\n    .. image:: https://raw.githubusercontent.com/jay-johnson/celery-connectors/master/_images/jupyterhub-step-7-jupyterhub-user-notebook-persistence.png\n        :align: center\n\nLinting\n-------\n\n::\n\n    pycodestyle --max-line-length=160 --exclude=venv,build,.tox,celery_connectors/rabbitmq/rabbitmqadmin.py\n\nLicense\n-------\n\nApache 2.0 - Please refer to the LICENSE_ for more details\n\n.. _License: https://github.com/jay-johnson/celery-connectors/blob/master/LICENSE\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/jay-johnson/celery-connectors",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "celery-connectors",
        "package_url": "https://pypi.org/project/celery-connectors/",
        "platform": "",
        "project_url": "https://pypi.org/project/celery-connectors/",
        "project_urls": {
            "Homepage": "https://github.com/jay-johnson/celery-connectors"
        },
        "release_url": "https://pypi.org/project/celery-connectors/1.0.30/",
        "requires_dist": [
            "ansible (>=1.9)",
            "pep8 (>=1.7.1)",
            "flake8 (>=3.4.1)",
            "boto3",
            "pycurl",
            "redis",
            "celery (>=4.1.0)",
            "kombu (>=4.1.0)",
            "logstash-formatter",
            "python-logstash",
            "coverage",
            "future",
            "pylint",
            "spylunking",
            "unittest2",
            "mock"
        ],
        "requires_python": "",
        "summary": "Celery Connectors",
        "version": "1.0.30"
    },
    "last_serial": 4445922,
    "releases": {
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2eb892b4d4a0dde7e231cd9ebc54c110",
                    "sha256": "63be3db036989ef4e14ff4e75190633b3449aca82976bf865bff90726234ea1d"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "2eb892b4d4a0dde7e231cd9ebc54c110",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 26775,
                "upload_time": "2017-12-03T09:26:51",
                "url": "https://files.pythonhosted.org/packages/64/b6/21bac0366b7a9e6feaa15804fbe8b38e0c3810f99882c86dd53bc3d5c278/celery_connectors-1.0.0-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "145cf869eb910a4861e59207966e4d56",
                    "sha256": "bda79280153cccfdcbb9c3fddae01452c244bb802774011dabe5e7f3cfbe4b84"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "145cf869eb910a4861e59207966e4d56",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 22400,
                "upload_time": "2017-12-03T09:26:49",
                "url": "https://files.pythonhosted.org/packages/97/71/0af7bbd33f3eed58e6b4e5748eb1f6972839c45b466439ab7b308b7a176e/celery-connectors-1.0.0.tar.gz"
            }
        ],
        "1.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ad2c28cdd20a7c7e59bb8259a54450ed",
                    "sha256": "80c245e239d1c740d1074eb9fda5fdbe7ef82855fe16792bac61f3aca9ac1abd"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "ad2c28cdd20a7c7e59bb8259a54450ed",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 26774,
                "upload_time": "2017-12-03T09:49:32",
                "url": "https://files.pythonhosted.org/packages/5c/5a/c3bac293f1f30300edd95a4b5b251ecd261365343adf1a61ce9c7210c7aa/celery_connectors-1.0.1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "687e315b45f0063ff2c1436cd1f92810",
                    "sha256": "1558f78963e65c5cf05c04e2652b019feaa0408c35c9d32218ab4a672df8ef4f"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "687e315b45f0063ff2c1436cd1f92810",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 22381,
                "upload_time": "2017-12-03T09:49:31",
                "url": "https://files.pythonhosted.org/packages/e2/3e/8a2f5337141797ddeaf1e943d73fbf089ad2274f3e9a1d888ad12adc714c/celery-connectors-1.0.1.tar.gz"
            }
        ],
        "1.0.10": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d443483c305873bb75d7701d8117886d",
                    "sha256": "aa8f8cdb9b37fa91d1da8b7c02561cd0734755e657dbefeea21dcac11ba9815f"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.10-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "d443483c305873bb75d7701d8117886d",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 47831,
                "upload_time": "2017-12-08T10:46:25",
                "url": "https://files.pythonhosted.org/packages/22/f3/66138468a82de1ed501ef12d7b271c38b29d1eb52100c80440358dbae335/celery_connectors-1.0.10-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "83bb204b14b0530c2068ecae09d29e6f",
                    "sha256": "7227737d2a3fcaab36b150bc04e9ca1e9f5de776a8bf84b24a4c3a9f0ea0f3b2"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.10.tar.gz",
                "has_sig": false,
                "md5_digest": "83bb204b14b0530c2068ecae09d29e6f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 31667,
                "upload_time": "2017-12-08T10:46:23",
                "url": "https://files.pythonhosted.org/packages/62/a3/4c3e41295c40d2b03ab7aed122c4217921d224f65893dd6cb3b81f38d098/celery-connectors-1.0.10.tar.gz"
            }
        ],
        "1.0.11": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6c60c12e8d19b566f026485ace88ed98",
                    "sha256": "6370352c8baa6090861b13b2fdc8c77f17674bea530845c1735c5030f8e14824"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.11-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "6c60c12e8d19b566f026485ace88ed98",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 41878,
                "upload_time": "2017-12-09T06:02:31",
                "url": "https://files.pythonhosted.org/packages/44/13/28a250c471f209658af6a2982f1e368c07838e2c8888b0fc3db866ab79e9/celery_connectors-1.0.11-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "78770c3c94f3b2d739a138fece1153e9",
                    "sha256": "6efe7a6258fa28284e71fbfd04c15754d38194e95e96b3acae7774094ccdb5fc"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.11.tar.gz",
                "has_sig": false,
                "md5_digest": "78770c3c94f3b2d739a138fece1153e9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 31790,
                "upload_time": "2017-12-09T06:02:30",
                "url": "https://files.pythonhosted.org/packages/6f/85/04071b1459d1bb81d9dd476d6465a1ecf3b8256f3838715b1845e17be6e6/celery-connectors-1.0.11.tar.gz"
            }
        ],
        "1.0.12": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "856da780ad521fd6d1bddc64d35a103b",
                    "sha256": "1a81b8032b11e3df9c7a841ec44536b77a0681b279aae94d4fb85cf4df1f62b7"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.12-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "856da780ad521fd6d1bddc64d35a103b",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 39529,
                "upload_time": "2017-12-09T16:34:46",
                "url": "https://files.pythonhosted.org/packages/69/76/fafb7aa9897b7f228761cfee7a825f09ec111050eda88b8217dbdcba3700/celery_connectors-1.0.12-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "2c2fae4eb4771194606ad47450eabb40",
                    "sha256": "414bd6f8badb2bb2ac544e9a21685f5c65449df366982ead1db10dbc1fcb7078"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.12.tar.gz",
                "has_sig": false,
                "md5_digest": "2c2fae4eb4771194606ad47450eabb40",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 31829,
                "upload_time": "2017-12-09T16:34:44",
                "url": "https://files.pythonhosted.org/packages/de/5c/3b45d6217da73741f86daa71497492ad5be2b2be7b99126a21ca28206ddb/celery-connectors-1.0.12.tar.gz"
            }
        ],
        "1.0.13": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "77b9e676c84442e07bbcfe0509aa020c",
                    "sha256": "0700d4f97ea76428820ff74b3e34064657743ce5bd6c603868dd78e671bcc61a"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.13-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "77b9e676c84442e07bbcfe0509aa020c",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 54577,
                "upload_time": "2017-12-09T22:13:04",
                "url": "https://files.pythonhosted.org/packages/d4/c8/0abf465abce213f1f763de2a185b344f8fe7ee96328e924e9bad5997456b/celery_connectors-1.0.13-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "fc5ef9d645783208204ce30574c4a34e",
                    "sha256": "df3b99412d9b70dcf8ef6e452b17b273943f428eb1a3d05db2334e4936de8808"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.13.tar.gz",
                "has_sig": false,
                "md5_digest": "fc5ef9d645783208204ce30574c4a34e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 34020,
                "upload_time": "2017-12-09T22:13:02",
                "url": "https://files.pythonhosted.org/packages/f6/da/6b55e77a234a7a4bb754791de1536eaf0c0d831af6de7efebee2d8c0cbbb/celery-connectors-1.0.13.tar.gz"
            }
        ],
        "1.0.14": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1ca22ba4007f4a28be907aa2e41f7677",
                    "sha256": "bc787805016ad56544f3db94a65a4216d4926bd53bedd4c5881df1de6694e1de"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.14-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1ca22ba4007f4a28be907aa2e41f7677",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 52468,
                "upload_time": "2017-12-10T23:37:20",
                "url": "https://files.pythonhosted.org/packages/28/b0/f46e830676b4bf57887bef8ab60cf333fb125e442e751bd90af8ca5463c6/celery_connectors-1.0.14-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "be404f4e74ac81240ccc9a38c3e8d399",
                    "sha256": "bb173ae1acce7fcb1300e8cf4092c6d8843c76d47277904ed3e7759f88d34e2a"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.14.tar.gz",
                "has_sig": false,
                "md5_digest": "be404f4e74ac81240ccc9a38c3e8d399",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 36308,
                "upload_time": "2017-12-10T23:37:18",
                "url": "https://files.pythonhosted.org/packages/2e/57/7921a4b923c1cb396f764fa18ae90f233a53a6aab33c473d0b00069ac35d/celery-connectors-1.0.14.tar.gz"
            }
        ],
        "1.0.15": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "10dd06bde66e502dfc7cb871075d721f",
                    "sha256": "c988b117ae5517914400631599668e0e6b20f18216bd0d140b6b2472dd69dcea"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.15-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "10dd06bde66e502dfc7cb871075d721f",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 53961,
                "upload_time": "2017-12-12T09:10:11",
                "url": "https://files.pythonhosted.org/packages/a6/ed/7b1b6f17aa66e597ee3553eccb9ec075ecb1a2483c52cfb0abe97f16efbe/celery_connectors-1.0.15-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "f020fce18837d0687b09f81a5e9a1c77",
                    "sha256": "0858d9352fa4716764d143a4d41cab816b2bbb4397584881c0cacd4c4e61bc83"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.15.tar.gz",
                "has_sig": false,
                "md5_digest": "f020fce18837d0687b09f81a5e9a1c77",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 37624,
                "upload_time": "2017-12-12T09:10:09",
                "url": "https://files.pythonhosted.org/packages/35/02/a1b9845eaf6ae143685f4465c9128f1bbfe7b65d2daff97888adc1817b90/celery-connectors-1.0.15.tar.gz"
            }
        ],
        "1.0.16": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f5b57513104a24ed76eeac8eed7ae90a",
                    "sha256": "42b5acbba2a2461c8c6ad0ea46dffd4debb74e3e147db5d6299088bf97689f9b"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.16-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f5b57513104a24ed76eeac8eed7ae90a",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 74441,
                "upload_time": "2017-12-13T09:52:25",
                "url": "https://files.pythonhosted.org/packages/71/f9/6b59fbccc27a13d3299203b449fbf77d2240a8940d23b47d633d48764335/celery_connectors-1.0.16-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "cb877c5958f7c76117d0f6b916646257",
                    "sha256": "f12b386c1e2a71cb4cc12f0aca4e33a09e3ce4b775def8307f5181b13440128b"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.16.tar.gz",
                "has_sig": false,
                "md5_digest": "cb877c5958f7c76117d0f6b916646257",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 44478,
                "upload_time": "2017-12-13T09:52:23",
                "url": "https://files.pythonhosted.org/packages/9d/b8/66f603bab6d254a8748295ef66e0666cee21d727752bf57c479c2c8b7491/celery-connectors-1.0.16.tar.gz"
            }
        ],
        "1.0.17": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4265f25cef47dc17532a8a1678852bef",
                    "sha256": "4b93dad4a96279acfc425993e045acb170f5398b911e47af3144b85aa2d1a26a"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.17-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "4265f25cef47dc17532a8a1678852bef",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 95290,
                "upload_time": "2017-12-14T09:37:03",
                "url": "https://files.pythonhosted.org/packages/52/fb/e628b3c39a3152318976659bba7e5be25be9961e09a4d336cb1f9bb51a1d/celery_connectors-1.0.17-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "00fec89c76e170dfa03125811a81d957",
                    "sha256": "10daa2203f0b98f1c887a3def31456b9f6b969d1dad198ba09a321171ce03214"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.17.tar.gz",
                "has_sig": false,
                "md5_digest": "00fec89c76e170dfa03125811a81d957",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 52463,
                "upload_time": "2017-12-14T09:36:59",
                "url": "https://files.pythonhosted.org/packages/29/a6/fa9fc269484df2bf7b466f494612e3d6867c92d22504791336acf6c75e36/celery-connectors-1.0.17.tar.gz"
            }
        ],
        "1.0.18": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1f11619023b1a219b8a25ecbc977db4a",
                    "sha256": "19ccff814fef5e3e05a656f300435f6cb09f37eed4581cf4ed4c031cdd110cd4"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.18-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1f11619023b1a219b8a25ecbc977db4a",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 95298,
                "upload_time": "2017-12-14T09:39:02",
                "url": "https://files.pythonhosted.org/packages/39/3c/2eec29ef08a9f077b0634e9a5db815bc69280dc8d4595f1a86b8c523706b/celery_connectors-1.0.18-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "2d9f0465a12976844f6e5b387c256509",
                    "sha256": "5303d892d7e116446b62c822ed7d34c9e70d08a8611f0cdff16a4c4a144243cc"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.18.tar.gz",
                "has_sig": false,
                "md5_digest": "2d9f0465a12976844f6e5b387c256509",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 52472,
                "upload_time": "2017-12-14T09:39:00",
                "url": "https://files.pythonhosted.org/packages/a3/17/fe374f31326ee006f1847978c2c3acaf3b1c3456134c2a9cb56bf83b2aa5/celery-connectors-1.0.18.tar.gz"
            }
        ],
        "1.0.19": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9bc93de2ec1c73246172dfed3ebdb2f5",
                    "sha256": "cc9769cf2ff3440470795a647004acdf6cb2f535b5467a76c63eb4f8d3473421"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.19-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "9bc93de2ec1c73246172dfed3ebdb2f5",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 95306,
                "upload_time": "2017-12-14T09:40:53",
                "url": "https://files.pythonhosted.org/packages/c8/d6/f127e8fc922711686fd46bb54f94a64290c75038532c9ef327af0ac6bfdd/celery_connectors-1.0.19-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "ee90b063d607d3a0137b1ff828ce6959",
                    "sha256": "0bbadf9009fc5ec9a2d68e72d739a0719bae9c05efcc5d9f8223fc89c8a4c6bb"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.19.tar.gz",
                "has_sig": false,
                "md5_digest": "ee90b063d607d3a0137b1ff828ce6959",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 52474,
                "upload_time": "2017-12-14T09:40:49",
                "url": "https://files.pythonhosted.org/packages/ea/84/5cc63df26046de7f7ac3a62dd982e2b92c9b58bd85198016b9b344f69d0c/celery-connectors-1.0.19.tar.gz"
            }
        ],
        "1.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9a88a487c6566f7c2879ed10b837f87d",
                    "sha256": "0c41885e05cf78e974d668880b23ec47e1f0ebedf0595805ec98f185950e63e3"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.2-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "9a88a487c6566f7c2879ed10b837f87d",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 41309,
                "upload_time": "2017-12-03T20:55:36",
                "url": "https://files.pythonhosted.org/packages/f0/d1/ce23e96f65cf95a1dd6db103f7c31adf8491f1ccbe21dcc929a045855d91/celery_connectors-1.0.2-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "f4d6d072bed2414d7ec8ad90d95d9fd0",
                    "sha256": "ca77d50907d9c2c23484aaf0925bce371022e48cccb6b9cd678e82f34d1d7f00"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "f4d6d072bed2414d7ec8ad90d95d9fd0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 23720,
                "upload_time": "2017-12-03T20:55:33",
                "url": "https://files.pythonhosted.org/packages/a3/41/2317f10e0f2ba671733bfb5067fe1b2fb7c6543ee5649a46582850a23072/celery-connectors-1.0.2.tar.gz"
            }
        ],
        "1.0.20": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a141c8dc6fba9bc58ab5df474b8ca07c",
                    "sha256": "3fb69c378f5c2b3b15b3e204cebb09c5ca137d967c47b6869801388043fccc30"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.20-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a141c8dc6fba9bc58ab5df474b8ca07c",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 96640,
                "upload_time": "2017-12-15T08:26:13",
                "url": "https://files.pythonhosted.org/packages/6d/0d/8041c704d0570294a9dc2b8a268be81c6481392fec14095dc4a1247ba83a/celery_connectors-1.0.20-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "b09433600cd3d7fa9e6a58c26f287459",
                    "sha256": "208d965e80a692baf84c7eef09d2aa0c4a3724fb7cb9ccb0ebdfa0745e22a9d9"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.20.tar.gz",
                "has_sig": false,
                "md5_digest": "b09433600cd3d7fa9e6a58c26f287459",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 57345,
                "upload_time": "2017-12-15T08:26:10",
                "url": "https://files.pythonhosted.org/packages/5c/0a/494105798495057b42eebb0ceff66caccbac3f298cd7216acbdda810c04e/celery-connectors-1.0.20.tar.gz"
            }
        ],
        "1.0.21": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ec242ff4e645caed8f2b20475094d215",
                    "sha256": "ac3e8feb40b9c488348c1e6d81e4e4c5bf70da77d207691498a7a5dcebde7a60"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.21-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "ec242ff4e645caed8f2b20475094d215",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 97125,
                "upload_time": "2017-12-16T08:17:01",
                "url": "https://files.pythonhosted.org/packages/65/50/02b00c652251309baa1adb5224cbb0f2d44d3e07e08df2f89e8ba5784c96/celery_connectors-1.0.21-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "2f7565b662e96255d926107182a77467",
                    "sha256": "a8d67f9b48b6229b30758403fb38ef7dd13d8c8451c2ec81afff4f2ac7fa142c"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.21.tar.gz",
                "has_sig": false,
                "md5_digest": "2f7565b662e96255d926107182a77467",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 58144,
                "upload_time": "2017-12-16T08:16:59",
                "url": "https://files.pythonhosted.org/packages/72/06/38eee5af12c32a6f5ce5d34c695787d2c5e6630d976a0e3779913a8d8f49/celery-connectors-1.0.21.tar.gz"
            }
        ],
        "1.0.22": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "081daeb6f0eadeaa265d00175ec6eb87",
                    "sha256": "9f3e7adebe7b2efa1ba258c05622191dd209c6ecf5cd996b2b6c19a5f1084673"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.22-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "081daeb6f0eadeaa265d00175ec6eb87",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 97324,
                "upload_time": "2017-12-19T09:47:09",
                "url": "https://files.pythonhosted.org/packages/a5/aa/17cbb9b86d359a2ea10731f1c364ac4dff5d250c2750d93faa798de28ff1/celery_connectors-1.0.22-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "e3b6cda605a6e1b3ede85a5a2247d0ba",
                    "sha256": "6c25c062f9f75b8bd704d989e81bb68187cd96e33eaa81f1055cb78bd12882c1"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.22.tar.gz",
                "has_sig": false,
                "md5_digest": "e3b6cda605a6e1b3ede85a5a2247d0ba",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 58806,
                "upload_time": "2017-12-19T09:47:06",
                "url": "https://files.pythonhosted.org/packages/eb/b9/7c1b96cc99db683f8f8bca6f3243637f2f23c788c63f26504f3af8889335/celery-connectors-1.0.22.tar.gz"
            }
        ],
        "1.0.23": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6ace53052ac9c10b249b0c907626c9db",
                    "sha256": "addc1e5d3192387e36e02c016cd3e09a1b9d10d0b836a6a23d371dc107c05552"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.23-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "6ace53052ac9c10b249b0c907626c9db",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 97458,
                "upload_time": "2017-12-20T09:07:32",
                "url": "https://files.pythonhosted.org/packages/4e/42/40c0325ef20b101c0ec5d339f9930b4e5f2b2654b1a650e3bf90edf53c0e/celery_connectors-1.0.23-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "0bccb1f5078a158f52aa9d1a95706676",
                    "sha256": "88f28ad230e7f93d67a58cd4104035f9c7620e8e2d3ec2f62b98b2ed5b89136e"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.23.tar.gz",
                "has_sig": false,
                "md5_digest": "0bccb1f5078a158f52aa9d1a95706676",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 59485,
                "upload_time": "2017-12-20T09:07:30",
                "url": "https://files.pythonhosted.org/packages/97/39/06f5c18660fcff229c930cd6c419798ece454a4687ce7b8dc3b79d0487e1/celery-connectors-1.0.23.tar.gz"
            }
        ],
        "1.0.24": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "47f159c11744c3c72eea569eff3bb767",
                    "sha256": "f81d182a4d06e6d7c33994dd1e212a299bfd5b4ac6a30c9b488f31bc57a12124"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.24-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "47f159c11744c3c72eea569eff3bb767",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 97283,
                "upload_time": "2017-12-21T06:46:16",
                "url": "https://files.pythonhosted.org/packages/df/fc/985da2ce2af7d0f7d6b7580cfcf842bb4201f81fa0ffb7565b258fdf2ec4/celery_connectors-1.0.24-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "028fad150455fea5e74466449acf5f43",
                    "sha256": "56642e76a5e4056dfb26f36fe91801c6e9230359c72378248e21e686bf349e9d"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.24.tar.gz",
                "has_sig": false,
                "md5_digest": "028fad150455fea5e74466449acf5f43",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 60263,
                "upload_time": "2017-12-21T06:46:10",
                "url": "https://files.pythonhosted.org/packages/ce/9f/7487ced85dac864c47f5f80235fc91792338133a378c31a18c5c95d9677a/celery-connectors-1.0.24.tar.gz"
            }
        ],
        "1.0.26": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "70eb08dc2b19c1270daece93324e9c5c",
                    "sha256": "f6286f670d359caf07c9f0acf03202342f034c6cdd53593546900ffa6cb51360"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.26-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "70eb08dc2b19c1270daece93324e9c5c",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 72570,
                "upload_time": "2018-06-13T04:35:55",
                "url": "https://files.pythonhosted.org/packages/5a/1a/1825f9eb54241bc4c10f60b53e5999e6d481ae6b4428bff3e4759ebb5a79/celery_connectors-1.0.26-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "59d8cd9bfb1142a63e70b9885ca0ddc4",
                    "sha256": "a3e9762b3630240bd1aed7b48e667dc9ac611eca1675c0a4e9f46092992f8d3f"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.26.tar.gz",
                "has_sig": false,
                "md5_digest": "59d8cd9bfb1142a63e70b9885ca0ddc4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 63051,
                "upload_time": "2018-06-13T04:35:53",
                "url": "https://files.pythonhosted.org/packages/15/39/72e1f59c935126d0e6ab67e58906d329f9262af571fcde0b04f3a954467c/celery-connectors-1.0.26.tar.gz"
            }
        ],
        "1.0.27": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "15949b71b65037206ab2d4c02eed60c1",
                    "sha256": "95bf1843a17d33f5bbb3460992e61a08369837dce8419ad0311563335f732f04"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.27-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "15949b71b65037206ab2d4c02eed60c1",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 72570,
                "upload_time": "2018-06-13T04:37:38",
                "url": "https://files.pythonhosted.org/packages/08/b1/bc0b001d00e96b109882abd987e24567b56ecba3423dad2a73dda3ea6262/celery_connectors-1.0.27-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7154990a4749ff922d5dcf0805d48a40",
                    "sha256": "c864240675439c95ae3eac9e8aa9bdd8c7142da55ecc0c376fddda6a099270ea"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.27.tar.gz",
                "has_sig": false,
                "md5_digest": "7154990a4749ff922d5dcf0805d48a40",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 63054,
                "upload_time": "2018-06-13T04:37:36",
                "url": "https://files.pythonhosted.org/packages/59/58/53c360f8b9bd167e9f4f91d97dfa876f29f3a792a501bd43036e7ac6d2a1/celery-connectors-1.0.27.tar.gz"
            }
        ],
        "1.0.28": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "79b2868ecccf5a335acd2586d8243a63",
                    "sha256": "c3065f2f4e40ba3e61c78ca2fd4196335a988bb7921b313dbd5de53d9beb3b5b"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.28-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "79b2868ecccf5a335acd2586d8243a63",
                "packagetype": "bdist_wheel",
                "python_version": "3.6",
                "requires_python": null,
                "size": 83531,
                "upload_time": "2018-06-26T07:22:24",
                "url": "https://files.pythonhosted.org/packages/36/6f/7f7d0f4427955d7eddfa42cef2d2602d0d00f4c9fc754bb535188d7cb09c/celery_connectors-1.0.28-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "fa7b22d792f69f3dfc112a9a2b5787f7",
                    "sha256": "483c7f7a1117b300dddcb61e8789d5f0b666f5dd01d0528fbcca9d2cdcd4760f"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.28.tar.gz",
                "has_sig": false,
                "md5_digest": "fa7b22d792f69f3dfc112a9a2b5787f7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 63209,
                "upload_time": "2018-06-26T07:22:23",
                "url": "https://files.pythonhosted.org/packages/b0/35/136c22fddbdc4984ff735d2d21a6c343279ea1dc6dbca578c2a90eb3fd0d/celery-connectors-1.0.28.tar.gz"
            }
        ],
        "1.0.29": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9ed666c89d25622ef406676b585a7163",
                    "sha256": "e34c3990da16d334e6527a28b7e74f8b55c259eaa7430aa6c67c5b6f2ec63818"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.29-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "9ed666c89d25622ef406676b585a7163",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 94123,
                "upload_time": "2018-11-02T19:51:13",
                "url": "https://files.pythonhosted.org/packages/15/7f/85332793e73b40399bbc7790e9a1aeede620558e6cbbd70b8cc0c910cae1/celery_connectors-1.0.29-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "db1731dbe1201039a047232558c80ed3",
                    "sha256": "1b07039495498f4420e898a57084b4fcbea44176c3ad8e8c06bffff07e14640b"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.29.tar.gz",
                "has_sig": false,
                "md5_digest": "db1731dbe1201039a047232558c80ed3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 110469,
                "upload_time": "2018-11-02T19:51:16",
                "url": "https://files.pythonhosted.org/packages/01/3d/70d966f1e478d0e6704e3a203fa3973714b1331c088209e812b0fe4ec719/celery-connectors-1.0.29.tar.gz"
            }
        ],
        "1.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "311b1dfe67f842f827947e89a00ea56b",
                    "sha256": "f0fa4fd81aade29b9bdb7d0f54213b155b119016994da9f34c1b3a940ffdf441"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.3-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "311b1dfe67f842f827947e89a00ea56b",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 30642,
                "upload_time": "2017-12-03T21:06:04",
                "url": "https://files.pythonhosted.org/packages/7b/39/9971065ffd0b6f95573823bb32b51b9966f0f802d7f7a596098541f5b633/celery_connectors-1.0.3-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "849182c4297f35ebb87509d422deabcc",
                    "sha256": "55c889271e6ca138a356f55d2949b601b1c7e361e201d6e8cb378d1329c79a0d"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "849182c4297f35ebb87509d422deabcc",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 23729,
                "upload_time": "2017-12-03T21:06:02",
                "url": "https://files.pythonhosted.org/packages/de/d4/cccd47bd3fbabb8e4a9f20e687c7592936a2b436d604b80432e093a94e2c/celery-connectors-1.0.3.tar.gz"
            }
        ],
        "1.0.30": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "50da15210df79f693d6a814eb466636f",
                    "sha256": "6911add947491af900fe1c3d7e9ae64f26e0760b2a01f29f6e252895b7b03c76"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.30-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "50da15210df79f693d6a814eb466636f",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 94152,
                "upload_time": "2018-11-02T19:58:02",
                "url": "https://files.pythonhosted.org/packages/04/36/eb7d9bf3de20ff32b1027e2cbf4e02be1d8a38a3ce4ee415542a760ae563/celery_connectors-1.0.30-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3cd50424b7f05af1709245187f38b7ce",
                    "sha256": "eb854a2e8c299e9ce22e3631923f2c4cc7ccdd5ff610e8651cf77289c77565cc"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.30.tar.gz",
                "has_sig": false,
                "md5_digest": "3cd50424b7f05af1709245187f38b7ce",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 110549,
                "upload_time": "2018-11-02T19:58:04",
                "url": "https://files.pythonhosted.org/packages/f2/40/6141c6c423b3d173f68795b9474b0d392c7e875f5a478d0901658f0a34ad/celery-connectors-1.0.30.tar.gz"
            }
        ],
        "1.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "cc2d895d98e3ada958459c98a2081c1e",
                    "sha256": "305757b2477e1cd7baea1653ff31a9ea026063c7002f075225c43bd83e4aca3f"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.4-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "cc2d895d98e3ada958459c98a2081c1e",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 30640,
                "upload_time": "2017-12-03T21:38:29",
                "url": "https://files.pythonhosted.org/packages/64/e9/a9dcfd823c8a61297607a0cdeec76e4af1f3216f0b1c5b52ffa32669bf78/celery_connectors-1.0.4-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "db57202e29b160ff3da1e5ebc8fd34ca",
                    "sha256": "172aabe4b04e141a0d5ec40371407308c25570dac8b251985f25b049bcee7088"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "db57202e29b160ff3da1e5ebc8fd34ca",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 23743,
                "upload_time": "2017-12-03T21:38:27",
                "url": "https://files.pythonhosted.org/packages/95/e0/8afa1de4aad95486af7123a5feb41cebe3d7bca7b0ed37a68dced879f577/celery-connectors-1.0.4.tar.gz"
            }
        ],
        "1.0.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c8ca2bb0d2e5aef37e5bb507b9c62f4d",
                    "sha256": "4b6ae5da028304213004a6f8bb880b66259564986a36ae695173f6d1a317b64c"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.5-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "c8ca2bb0d2e5aef37e5bb507b9c62f4d",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 33073,
                "upload_time": "2017-12-04T01:38:01",
                "url": "https://files.pythonhosted.org/packages/b3/d8/b705d6ee713d79b7d9281bb9f4c2c029d9b505cfbd5ba82f0bb37b36efa9/celery_connectors-1.0.5-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "08e273c1c6869fcb59921bed85888e47",
                    "sha256": "55b38742f38b94ac7764c3a108c8f4b78279d00d2d9ced04c1a7ac9058d836e4"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.5.tar.gz",
                "has_sig": false,
                "md5_digest": "08e273c1c6869fcb59921bed85888e47",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25754,
                "upload_time": "2017-12-04T01:37:59",
                "url": "https://files.pythonhosted.org/packages/03/d1/173fba919ccc239d6716986ce460adabfd32afc21e6b2718767787b1e49f/celery-connectors-1.0.5.tar.gz"
            }
        ],
        "1.0.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6f3b4ffa15df6be50f68572ecb97cd21",
                    "sha256": "d8f7d67355ee6c16fc15ffec2b508b8b8fb8519338f063ac8be0caf2373d9ea3"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.6-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "6f3b4ffa15df6be50f68572ecb97cd21",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 40124,
                "upload_time": "2017-12-04T07:55:44",
                "url": "https://files.pythonhosted.org/packages/92/5e/396c16459dd828fe3787cd089fb25b8acef2fe5107c4350a4b2820e9c1a6/celery_connectors-1.0.6-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "e70c42bc689271165c69e961ef7e5077",
                    "sha256": "7e371f55b11d5368dcc842c61294de09c19554fcfa3800500a64a6dc2de58287"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.6.tar.gz",
                "has_sig": false,
                "md5_digest": "e70c42bc689271165c69e961ef7e5077",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29453,
                "upload_time": "2017-12-04T07:55:42",
                "url": "https://files.pythonhosted.org/packages/1e/bd/46fec1d49cbea40d04af0be48268d1bd70c63b98e49060d59ba50a41dc07/celery-connectors-1.0.6.tar.gz"
            }
        ],
        "1.0.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "12310b7e4a07479177d086fd5fda182f",
                    "sha256": "2db7e34d944be5b7bc60a26a5882b9042ef5fe53206d714ccb9a739cee63d6d6"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.7-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "12310b7e4a07479177d086fd5fda182f",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 45601,
                "upload_time": "2017-12-07T09:34:04",
                "url": "https://files.pythonhosted.org/packages/11/f3/5ed47e7cb2ffec69336767446d5797a335cbb3facb30c4aeafa54787bee1/celery_connectors-1.0.7-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "a495ebe181f2502eeff3db6f63950725",
                    "sha256": "c2040700f9807c310865479da2058430e79a66da23ed95530c09b75c31f32252"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.7.tar.gz",
                "has_sig": false,
                "md5_digest": "a495ebe181f2502eeff3db6f63950725",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29592,
                "upload_time": "2017-12-07T09:34:02",
                "url": "https://files.pythonhosted.org/packages/3d/39/227910916990e98c04ef3f85b172d329d5de398240ef76c6e43f47b6fe23/celery-connectors-1.0.7.tar.gz"
            }
        ],
        "1.0.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2973366f79e0b4b52ada1fd7284d6334",
                    "sha256": "e792d1594c02b88e76a2403ff768fe2bdbdd9ee1d83450a73c77924f70d9762f"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.8-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "2973366f79e0b4b52ada1fd7284d6334",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 45589,
                "upload_time": "2017-12-08T10:33:48",
                "url": "https://files.pythonhosted.org/packages/57/9e/f7c56f21f6522e07468ed3922138bcaaa2b9b4ec5210ec3ac4efc55a0cf6/celery_connectors-1.0.8-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7d02cefe2a0c79feaf130522ea792d04",
                    "sha256": "9273a0706bf50e5cae3ddef8a5947ae84948ff1df3618f7090f295b1c43372f6"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.8.tar.gz",
                "has_sig": false,
                "md5_digest": "7d02cefe2a0c79feaf130522ea792d04",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 30181,
                "upload_time": "2017-12-08T10:33:46",
                "url": "https://files.pythonhosted.org/packages/f1/f6/0a01de46ce7b2fe7c14b12ed56b557e2efb1980e663c34dcb3ecec6b38d7/celery-connectors-1.0.8.tar.gz"
            }
        ],
        "1.0.9": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "eb893b2ae0321f51e1529682507869f5",
                    "sha256": "26c26ccab97b6eb08a956c641c41daffa81afd63a2bcc1a7e5313044e8bae469"
                },
                "downloads": -1,
                "filename": "celery_connectors-1.0.9-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "eb893b2ae0321f51e1529682507869f5",
                "packagetype": "bdist_wheel",
                "python_version": "3.5",
                "requires_python": null,
                "size": 45636,
                "upload_time": "2017-12-08T10:33:17",
                "url": "https://files.pythonhosted.org/packages/dc/53/cc653db53c57d65d38a8b7667e37983abbcf19a221379850a68d4c79c9af/celery_connectors-1.0.9-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "dd9ccca17d7f39b1d43de274d5887e07",
                    "sha256": "847297217329bd69b6c7b7cac884a22c0b87e0efd289acc5bc95d1a4cc5925ac"
                },
                "downloads": -1,
                "filename": "celery-connectors-1.0.9.tar.gz",
                "has_sig": false,
                "md5_digest": "dd9ccca17d7f39b1d43de274d5887e07",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 30221,
                "upload_time": "2017-12-08T10:33:14",
                "url": "https://files.pythonhosted.org/packages/3d/e9/7aa563329a3a5f1d9e18e2b8b1c69bb350ec440e94d40539b44866fc93f4/celery-connectors-1.0.9.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "50da15210df79f693d6a814eb466636f",
                "sha256": "6911add947491af900fe1c3d7e9ae64f26e0760b2a01f29f6e252895b7b03c76"
            },
            "downloads": -1,
            "filename": "celery_connectors-1.0.30-py2.py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "50da15210df79f693d6a814eb466636f",
            "packagetype": "bdist_wheel",
            "python_version": "py2.py3",
            "requires_python": null,
            "size": 94152,
            "upload_time": "2018-11-02T19:58:02",
            "url": "https://files.pythonhosted.org/packages/04/36/eb7d9bf3de20ff32b1027e2cbf4e02be1d8a38a3ce4ee415542a760ae563/celery_connectors-1.0.30-py2.py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "3cd50424b7f05af1709245187f38b7ce",
                "sha256": "eb854a2e8c299e9ce22e3631923f2c4cc7ccdd5ff610e8651cf77289c77565cc"
            },
            "downloads": -1,
            "filename": "celery-connectors-1.0.30.tar.gz",
            "has_sig": false,
            "md5_digest": "3cd50424b7f05af1709245187f38b7ce",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 110549,
            "upload_time": "2018-11-02T19:58:04",
            "url": "https://files.pythonhosted.org/packages/f2/40/6141c6c423b3d173f68795b9474b0d392c7e875f5a478d0901658f0a34ad/celery-connectors-1.0.30.tar.gz"
        }
    ]
}