{
    "info": {
        "author": "Cumulus Authors",
        "author_email": "info@developmentseed.org",
        "bugtrack_url": null,
        "classifiers": [
            "Intended Audience :: Developers",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.7",
            "Topic :: Software Development :: Build Tools"
        ],
        "description": "# cumulus-message-adapter-python\n\n[![CircleCI](https://circleci.com/gh/nasa/cumulus-message-adapter-python.svg?style=svg)](https://circleci.com/gh/nasa/cumulus-message-adapter-python)\n[![PyPI version](https://badge.fury.io/py/cumulus-message-adapter-python.svg)](https://badge.fury.io/py/cumulus-message-adapter-python)\n\n\n## What is Cumulus?\n\nCumulus is a cloud-based data ingest, archive, distribution and management\nprototype for NASA's future Earth science data streams.\n\nRead the [Cumulus Documentation](https://cumulus-nasa.github.io/)\n\n## What is the Cumulus Message Adapter?\n\nThe Cumulus Message Adapter is a library that adapts incoming messages in the\nCumulus protocol to a format more easily consumable by Cumulus tasks, invokes\nthe tasks, and then adapts their response back to the Cumulus message protocol\nto be sent to the next task.\n\n## Installation\n\n```\n$ pip install git+https://github.com/cumulus-nasa/cumulus-message-adapter-python.git\n```\n\n## Task definition\n\nIn order to use the Cumulus Message Adapter, you will need to create two\nmethods in your task module: a handler function and a business logic function.\n\nThe handler function is a standard Lambda handler function which takes two\nparameters (as specified by AWS): `event` and `context`.\n\nThe business logic function is where the actual work of your task occurs. It\nshould take two parameters: `event` and `context`.\n\nThe `event` object contains two keys:\n\n  * `input` - the task's input, typically the `payload` of the message,\n    produced at runtime\n  * `config` - the task's configuration, with any templated variables\n    resolved\n\nThe `context` parameter is the standard Lambda context as passed by AWS.\n\nThe return value of the business logic function will be placed in the\n`payload` of the resulting Cumulus message.\n\nExpectations for input, config, and return values are all defined by the task,\nand should be well documented. Tasks should thoughtfully consider their inputs\nand return values, as breaking changes may have cascading effects on tasks\nthroughout a workflow. Configuration changes are slightly less impactful, but\nmust be communicated to those using the task.\n\n## Cumulus Message Adapter interface\n\nThe Cumulus Message adapter for python provides one method:\n`run_cumulus_task`. It takes four parameters:\n\n  * `task_function` - the function containing your business logic (as described\n    above)\n  * `cumulus_message` - the event passed by Lambda, and should be a Cumulus\n    Message, or a CMA parameter encapsulated message: \n    \n```json\n{\n  \"cma\": {\n    \"event\": \"<cumulus message object>\"\n    \"SomeCMAConfigKey\": \"CMA configuration object>\"\n  }\n}\n```\n  * `context` - the Lambda context\n  * `schemas` - optional: a dict with `input`, `config`, and `output` properties. Each should be a string set to the filepath of the corresponding JSON schema file. All three properties of this dict are optional. If ommitted, the message adapter will look in `/<task_root>/schemas/<schema_type>.json`, and if not found there, will be ignored.\n  * `taskargs` - Optional. Additional keyword arguments for the `task_function`\n\n## Example\n\nSimple example of using this package's `run_cumulus_task` function as a wrapper around another function:\n\n```py\nfrom run_cumulus_task import run_cumulus_task\n\n# simple task that returns the event\ndef task(event, context):\n    return event\n\n# handler that is provided to aws lambda\ndef handler(event, context):\n    return run_cumulus_task(task, event, context)\n```\n\nFor a full example see the [example folder](./example).\n\n## Creating a deployment package\n\nTasks that use this library are just standard AWS Lambda tasks. Information on\ncreating release packages is available [here](https://docs.aws.amazon.com/lambda/latest/dg/deployment-package-v2.html).\n\n## Usage in a Cumulus Deployment\n\nDuring deployment, Cumulus will automatically obtain and inject the [Cumulus Message Adapter](https://github.com/cumulus-nasa/cumulus-message-adapter) into the compiled code and create a zip file to be deployed to Lambda.\n\nThe example task in the [example folder](./example) of this repository would be configured in lambdas.yml as follows:\n\n```yaml\nPythonExample:\n  handler: task.handler\n  timeout: 300\n  source: './example'\n  useMessageAdapter: true\n  runtime: python2.7\n  memory: 256\n```\n\n## Development\n\n### Dependency Installation\n\n```\n$ pip install -r requirements-dev.txt\n$ pip install -r requirements.txt\n```\n\n### Logging with `CumulusLogger`\n\nIncluded in this package is the `cumulus_logger` which contains a logging class `CumulusLogger` that standardizes the log format for Cumulus. Methods are provided to log error, fatal, warning, debug, info, and trace. \n\n**Import the `CumulusLogger` class:**\n\n```python\nfrom cumulus_logger import CumulusLogger\n```\n\n**Instantiate the logger inside the task definition, name and level are optional:**\n\n```python\nlogger = CumulusLogger(event, context)\n```\n\n**Use the logging methods for different levels:**\n\n```python\nlogger.trace('<your message>')\nlogger.debug('<your message>')\nlogger.info('<your message>')\nlogger.warn('<your message>')\nlogger.error('<your message>')\nlogger.fatal('<your message>')\n```\n\n**It can also take additional non-keyworded and keyworded arguments as in Python Logger.**\n\nThe `msg` is the message format string, the `args` and `kwargs` are the arguments for string formatting.\n\nIf `exc_info` in `kwargs` is not false, the exception information in the `exc_info` or `sys.exc_info()` is added to the message.\n\n```\nlogger.debug(msg, *args, **kwargs)\n```\n\n**Example usage:**\n\n```python\nimport os\nimport sys\n\nfrom run_cumulus_task import run_cumulus_task\nfrom cumulus_logger import CumulusLogger\n\n# instantiate CumulusLogger\nlogger = CumulusLogger()\n\ndef task(event, context):\n    logger.info('task executed')\n\n    # log error when an exception is caught\n    logger.error(\"task formatted message {} exc_info \", \"bar\", exc_info=True)\n\n    # return the output of the task\n    return { \"example\": \"output\" }\n\ndef handler(event, context):\n    # make sure event & context metadata is set in the logger\n    logger.setMetadata(event, context)\n    return run_cumulus_task(task, event, context)\n```\n\n### Running Tests\n\nRunning tests requires [localstack](https://github.com/localstack/localstack).\n\nTests only require localstack running S3, which can be initiated with the following command:\n\n```\n$ SERVICES=s3 localstack start\n```\n\nAnd then you can check tests pass with the following nosetests command:\n\n```\n$ CUMULUS_ENV=testing nosetests -v -s\n```\n\n### Linting\n\n```\n$ pylint run_cumulus_task.py\n```\n\n## Why?\n\nThis approach has a few major advantages:\n\n1. It explicitly prevents tasks from making assumptions about data structures\n   like `meta` and `cumulus_meta` that are owned internally and may therefore\n   be broken in future updates. To gain access to fields in these structures,\n   tasks must be passed the data explicitly in the workflow configuration.\n1. It provides clearer ownership of the various data structures. Operators own\n   `meta`. Cumulus owns `cumulus_meta`. Tasks define their own `config`,\n   `input`, and `output` formats.\n1. The Cumulus Message Adapter greatly simplifies running Lambda functions not\n   explicitly created for Cumulus.\n1. The approach greatly simplifies testing for tasks, as tasks don't need to\n   set up cumbersome structures to emulate the message protocol and can just\n   test their business function.\n\n## License\n\n[Apache 2.0](LICENSE)",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/cumulus-nasa/cumulus-message-adapter-python",
        "keywords": "nasa cumulus",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "cumulus-message-adapter-python",
        "package_url": "https://pypi.org/project/cumulus-message-adapter-python/",
        "platform": "",
        "project_url": "https://pypi.org/project/cumulus-message-adapter-python/",
        "project_urls": {
            "Homepage": "https://github.com/cumulus-nasa/cumulus-message-adapter-python"
        },
        "release_url": "https://pypi.org/project/cumulus-message-adapter-python/1.0.9/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "A handler library for cumulus tasks written in python",
        "version": "1.0.9"
    },
    "last_serial": 5841911,
    "releases": {
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3a07bec482f6b1df09c9dc9901a4db2a",
                    "sha256": "b68f74b39ec1ef5e09626a2d808d5d6e81f45c46c1e6ba74adef4b45958e6292"
                },
                "downloads": -1,
                "filename": "cumulus_message_adapter_python-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "3a07bec482f6b1df09c9dc9901a4db2a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6198,
                "upload_time": "2018-03-12T14:51:56",
                "url": "https://files.pythonhosted.org/packages/70/b0/682bc43c2d47a6e3f3dc3550cef2c15dc3e446324cb53afddd30e46fe050/cumulus_message_adapter_python-1.0.0.tar.gz"
            }
        ],
        "1.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "da6f56e37c1dafead7edd6f04d9061b4",
                    "sha256": "51091b695e56fe0715452d010f54ee9ffc8b1e8d5cc82f5008f58d3631d95fe6"
                },
                "downloads": -1,
                "filename": "cumulus_message_adapter_python-1.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "da6f56e37c1dafead7edd6f04d9061b4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6163,
                "upload_time": "2018-05-16T18:38:39",
                "url": "https://files.pythonhosted.org/packages/a3/99/4e915c7a179cf81bef353735516afdc18dd6a0fa1160f523c1e7c1bbc56c/cumulus_message_adapter_python-1.0.1.tar.gz"
            }
        ],
        "1.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ecdc25ba321e48483fb1a989a59fb481",
                    "sha256": "62af9907389f7a4f5b59b5e343a64f410adc065b8d849916edd0202f0ee618de"
                },
                "downloads": -1,
                "filename": "cumulus_message_adapter_python-1.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "ecdc25ba321e48483fb1a989a59fb481",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6120,
                "upload_time": "2018-05-16T18:48:09",
                "url": "https://files.pythonhosted.org/packages/57/78/c15bd72265ba32852a6079a2f4869999d1eb5242ccb2df57f3f569895d9d/cumulus_message_adapter_python-1.0.2.tar.gz"
            }
        ],
        "1.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "04f0a286e9e678adb200bbf6a72d0a5f",
                    "sha256": "cea2f15085aa60303c0871c910afeb82a175ad4913722b3dfb4fc6076a110353"
                },
                "downloads": -1,
                "filename": "cumulus_message_adapter_python-1.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "04f0a286e9e678adb200bbf6a72d0a5f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6159,
                "upload_time": "2018-05-16T19:09:15",
                "url": "https://files.pythonhosted.org/packages/a6/73/feedf28adc1a1a4d787b1423531dbda81e225b67fbdbe5c48667b245213f/cumulus_message_adapter_python-1.0.3.tar.gz"
            }
        ],
        "1.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "430a31add6f0b662907d3b3263015cb4",
                    "sha256": "3c985164660cfd1ff2e5f42efc9cf68d50fad36a769f1d916db15fbcfe7dde93"
                },
                "downloads": -1,
                "filename": "cumulus_message_adapter_python-1.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "430a31add6f0b662907d3b3263015cb4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6201,
                "upload_time": "2018-08-09T14:19:46",
                "url": "https://files.pythonhosted.org/packages/e9/14/87f7c00b73e3c1a0e810d80cdf28117c27c003f6d32bbd049b03437b3ec2/cumulus_message_adapter_python-1.0.4.tar.gz"
            }
        ],
        "1.0.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a9b0bc0b51b012f159583ff2b4e86a8d",
                    "sha256": "145c5b3dcf7f03082f1c7c4f9db6b45b9ff04c30ea7cd4a67557a804c216d34f"
                },
                "downloads": -1,
                "filename": "cumulus_message_adapter_python-1.0.5.tar.gz",
                "has_sig": false,
                "md5_digest": "a9b0bc0b51b012f159583ff2b4e86a8d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6361,
                "upload_time": "2018-08-16T19:27:37",
                "url": "https://files.pythonhosted.org/packages/39/86/a3c779f29c1f917b69f7de9a486ebcb2e119cf257e497ca0512286b90458/cumulus_message_adapter_python-1.0.5.tar.gz"
            }
        ],
        "1.0.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ad9b59bd8d112b72570050fb9b3f92c1",
                    "sha256": "3375f9f2109d46011ab487ca6b3ca5f3fae147166e290183d133c99c4adf7977"
                },
                "downloads": -1,
                "filename": "cumulus_message_adapter_python-1.0.6.tar.gz",
                "has_sig": false,
                "md5_digest": "ad9b59bd8d112b72570050fb9b3f92c1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6337,
                "upload_time": "2018-12-12T20:05:07",
                "url": "https://files.pythonhosted.org/packages/14/ae/c41e433bcf89eb2b5a9fc096091dbb5576f5c333a3a3832a447d16845272/cumulus_message_adapter_python-1.0.6.tar.gz"
            }
        ],
        "1.0.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ceae50e2b831defbaef5477abb55edaf",
                    "sha256": "67d48cb2db5b472da0cd583ea086d342570af2ef05010c3ddb3cd418a6771d91"
                },
                "downloads": -1,
                "filename": "cumulus_message_adapter_python-1.0.7.tar.gz",
                "has_sig": false,
                "md5_digest": "ceae50e2b831defbaef5477abb55edaf",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6278,
                "upload_time": "2018-12-18T21:58:34",
                "url": "https://files.pythonhosted.org/packages/3d/8f/77b637f8f0c8e1b844764ae67b67587bada9487cebcd8c98750877ef0254/cumulus_message_adapter_python-1.0.7.tar.gz"
            }
        ],
        "1.0.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "751126b4b3cd5e534421fe2ff87b38fc",
                    "sha256": "aaa2bdfe6592bde2f8bc32b8c636fb7db1694c0603d3fd4bd82f6c5b84bd3b2f"
                },
                "downloads": -1,
                "filename": "cumulus_message_adapter_python-1.0.8.tar.gz",
                "has_sig": false,
                "md5_digest": "751126b4b3cd5e534421fe2ff87b38fc",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6906,
                "upload_time": "2019-01-18T16:13:25",
                "url": "https://files.pythonhosted.org/packages/4c/2e/d175cf480e98610daf5c1958f4e9b4d86f0f25f88f0ddc5c139e36b78e86/cumulus_message_adapter_python-1.0.8.tar.gz"
            }
        ],
        "1.0.9": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6889118d82eb7879eba8fc758f6818a4",
                    "sha256": "ace56ca19680f8ce836e74d860ed5a5efa317c1f4dcf4891758e3df8eb1c1990"
                },
                "downloads": -1,
                "filename": "cumulus_message_adapter_python-1.0.9.tar.gz",
                "has_sig": false,
                "md5_digest": "6889118d82eb7879eba8fc758f6818a4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7181,
                "upload_time": "2019-09-17T14:05:38",
                "url": "https://files.pythonhosted.org/packages/e9/5e/179451f71b63e75e6cb10da502a5547d7fc46420add12e11ea2d682c1e0e/cumulus_message_adapter_python-1.0.9.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "6889118d82eb7879eba8fc758f6818a4",
                "sha256": "ace56ca19680f8ce836e74d860ed5a5efa317c1f4dcf4891758e3df8eb1c1990"
            },
            "downloads": -1,
            "filename": "cumulus_message_adapter_python-1.0.9.tar.gz",
            "has_sig": false,
            "md5_digest": "6889118d82eb7879eba8fc758f6818a4",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 7181,
            "upload_time": "2019-09-17T14:05:38",
            "url": "https://files.pythonhosted.org/packages/e9/5e/179451f71b63e75e6cb10da502a5547d7fc46420add12e11ea2d682c1e0e/cumulus_message_adapter_python-1.0.9.tar.gz"
        }
    ]
}