{
    "info": {
        "author": "zylphrex",
        "author_email": "zylphrex@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Environment :: Plugins",
            "License :: OSI Approved :: Apache Software License"
        ],
        "description": "[![Build Status](https://travis-ci.com/getsentry/airflow-metrics.svg?token=TJpWxpbKGxDuV8CPPRzL&branch=master)](https://travis-ci.com/getsentry/airflow-metrics)\n\n# airflow-metrics\n\n`airflow-metrics` is an Airflow plugin for automatically sending metrics from Airflow to Datadog.\n\n**Tested For**: `apache-airflow>=1.10.2, <=1.10.3`\n\n## Installation\n\n```shell\npip install airflow-metrics\n```\n\n### Optional\n\nIf you want to the metrics from `BigQueryOperator` and `GoogleCloudStorageToBigQueryOperator`, then make sure the necessary dependencies are installed.\n\n```shell\npip install apache-airflow[gcp_api]\n```\n\n## Setup\n\n`airflow-metrics` will report all metrics to Datadog, so create an `airflow` connection with your Datadog api key.\n\n```shell\nairflow connections --add --conn_id datadog_default --conn_type HTTP --conn_extr '{\"api_key\": \"<your api key>\"}'\n```\n\n**Note**: If you skip this step, your `airflow` installation should still work but no metrics will be reported.\n\n## Usage\n\nThat's it! `airflow-metrics` will now begin sending metrics from Airflow to Datadog automatically.\n\n### Metrics\n\n`airflow-metrics` will automatically begin reporting the following metrics\n\n* `airflow.task.state` The total number of tasks in a state where the state is stored as a tag.\n* `airflow.task.state.bq` The current number of big query tasks in a state where the state is stored as a tag.\n* `airflow.dag.duration` The duration of a DAG in ms.\n* `airflow.task.duration` The duration of a task in ms.\n* `airflow.request.duration` The duration of a HTTP request in ms.\n* `airflow.request.status.success` The current number of HTTP requests with successful status codes (<400)\n* `airflow.request.status.failure` The current number of HTTP requests with unsuccessful status codes (>=400)\n* `airflow.task.upserted.bq` The number of rows upserted by a BigQueryOperator.\n* `airflow.task.delay.bq` The time taken for the big query job from a BigQueryOperator to start in ms.\n* `airflow.task.duration.bq` The time taken for the big query job from a BigQueryOperator to finish in ms.\n* `airflow.task.upserted.gcs_to_bq` The number of rows upserted by a GoogleCloudStorageToBigQueryOperator.\n* `airflow.task.delay.gcs_to_bq` The time taken for the big query from a GoogleCloudStorageToBigQueryOperator to start in ms.\n* `airflow.task.duration.gcs_to_bq` The time taken for the big query from a GoogleCloudStorageToBigQueryOperator to finish in ms.\n\n## Configuration\n\nBy default, `airflow-metrics` will begin extracting metrics from Airflow as you run your DAGs and send them to Datadog. You can opt out of it entirely or opt out of a subset of the metrics by setting these configurations in your `airflow.cfg`\n\n```\n[airflow_metrics]\n\nairflow_metrics_enabled = True\nairflow_metrics_tasks_enabled = True\nairflow_metrics_bq_enabled = True\nairflow_metrics_gcs_to_bq_enabled = True\nairflow_metrics_requests_enabled = True\nairflow_metrics_thread_enabled = True`\n```\n\n## Limitations\n\n`airflow-metrics` starts a thread to report some metrics, and is not supported when using sqlite as your database.\n\n## Contributing\n\nPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.\n\n### Getting Started\n\nSet up your virtual environment for python3 however you like.\n\n```shell\npip install -e .\nairflow initdb\nairflow connections --add --conn_id datadog_default --conn_type HTTP --conn_extr '{\"api_key\": \"\"}'\n```\n\n**Note**: The last step is necessary, otherwise the plugin will not initialize correctly and will not collect metrics. But you are free to add a dummy key for development purposes.\n\n### Running Tests\n\n```shell\npip install -r requirements-dev.txt\npytest\n```\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/getsentry/airflow-metrics",
        "keywords": "airflow,datadogmetrics,plugin",
        "license": "",
        "maintainer": "zylphrex",
        "maintainer_email": "zylphrex@gmail.com",
        "name": "airflow-metrics",
        "package_url": "https://pypi.org/project/airflow-metrics/",
        "platform": "MacOS",
        "project_url": "https://pypi.org/project/airflow-metrics/",
        "project_urls": {
            "Homepage": "https://github.com/getsentry/airflow-metrics"
        },
        "release_url": "https://pypi.org/project/airflow-metrics/0.1.5/",
        "requires_dist": [
            "datadog (>=0.29.3)"
        ],
        "requires_python": "",
        "summary": "Airflow plugin for automatically sending metrics from Airflow to Datadog",
        "version": "0.1.5"
    },
    "last_serial": 5537341,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f3573c8443e9c3404e4783d143a577db",
                    "sha256": "9e99f9b89a5f30f5ee9956ddd2d60dc1d03d11c3ab81200b3e84880b3c86f380"
                },
                "downloads": -1,
                "filename": "airflow_metrics-0.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f3573c8443e9c3404e4783d143a577db",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 15717,
                "upload_time": "2019-06-19T22:27:29",
                "url": "https://files.pythonhosted.org/packages/26/f4/532c7743add20f6eeba540d6db16c5f6e45fd8e7f344463651194f72e3a3/airflow_metrics-0.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "13a934085b1009cc4b880579ccc3e90f",
                    "sha256": "31a87c91cc52ffa26faf3c0fa50ae79db283f3a90aa76c7fe69d22d8c603dad3"
                },
                "downloads": -1,
                "filename": "airflow-metrics-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "13a934085b1009cc4b880579ccc3e90f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7328,
                "upload_time": "2019-06-19T22:27:31",
                "url": "https://files.pythonhosted.org/packages/eb/4a/5de51b5c70a711e56d83d749be7f63eaabe189e4258cb0dc1e48667c2c58/airflow-metrics-0.1.0.tar.gz"
            }
        ],
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "93bc3b9b309ac9501d73f957dbb8ea32",
                    "sha256": "b8fc5b2c32c664b69233fbbde62b14bde5cce36f8fcdd33c5c32ebe2badc4ee4"
                },
                "downloads": -1,
                "filename": "airflow_metrics-0.1.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "93bc3b9b309ac9501d73f957dbb8ea32",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 15730,
                "upload_time": "2019-06-20T17:54:25",
                "url": "https://files.pythonhosted.org/packages/09/c7/8cfb6e5e697689d270ecca37ec39f753f2fd3b32456f034e0eddecf80455/airflow_metrics-0.1.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "43cd8d18a23eacc545f9093c4cafc483",
                    "sha256": "2f0695c6d5e4bcb5ab674fc1eed36ba5ec1ec15d6279fa08cc771a5b41a712d7"
                },
                "downloads": -1,
                "filename": "airflow-metrics-0.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "43cd8d18a23eacc545f9093c4cafc483",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7364,
                "upload_time": "2019-06-20T17:54:27",
                "url": "https://files.pythonhosted.org/packages/a0/81/bf50d88ab70a9055399c122249382d1b1cd3c5d0cc25a024ff04ff93e5a5/airflow-metrics-0.1.1.tar.gz"
            }
        ],
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f1e2d18f67c27dac6099b68b4cb19c20",
                    "sha256": "f0b3f0448fe9241724f0e67c82f0af1000d141da4657e147857450621f4e6d2a"
                },
                "downloads": -1,
                "filename": "airflow_metrics-0.1.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f1e2d18f67c27dac6099b68b4cb19c20",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 15895,
                "upload_time": "2019-06-20T19:02:32",
                "url": "https://files.pythonhosted.org/packages/fa/53/18b1fe6760dadcee53086144636e6ea7078ce0e9b9de35eb784f9f741475/airflow_metrics-0.1.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "0cd2ea512a58f4749fef3dbba5dfc9fa",
                    "sha256": "4cc26c04130770e682ac1b2237bbd2d0cdaa4f904d11792ce4ca456ffb18d263"
                },
                "downloads": -1,
                "filename": "airflow-metrics-0.1.2.tar.gz",
                "has_sig": false,
                "md5_digest": "0cd2ea512a58f4749fef3dbba5dfc9fa",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7492,
                "upload_time": "2019-06-20T19:02:33",
                "url": "https://files.pythonhosted.org/packages/c1/02/ddefb008d0e5a8e8e9199d6d0aaabe93f1637e0818502c0928236501ac30/airflow-metrics-0.1.2.tar.gz"
            }
        ],
        "0.1.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0905264f356e448846a68e6c1ddb6b25",
                    "sha256": "42799f5ed3ad47e4cc6c1a7063faabba5ce8f9a24bc3a1f58f33d93959367fd9"
                },
                "downloads": -1,
                "filename": "airflow_metrics-0.1.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "0905264f356e448846a68e6c1ddb6b25",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 15902,
                "upload_time": "2019-07-10T18:28:56",
                "url": "https://files.pythonhosted.org/packages/6d/be/4e9221598bb452e69f07233d68706523d56714f29b642617cf17131f1634/airflow_metrics-0.1.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "f5816ff278ca960c9a660bcb18f53ca7",
                    "sha256": "05c2b1e7653f7bef2360f1c38e2049670ed2b35baffc04fae042ff6cb1649beb"
                },
                "downloads": -1,
                "filename": "airflow-metrics-0.1.3.tar.gz",
                "has_sig": false,
                "md5_digest": "f5816ff278ca960c9a660bcb18f53ca7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7506,
                "upload_time": "2019-07-10T18:28:58",
                "url": "https://files.pythonhosted.org/packages/3c/26/086884943ee70aa5183a857ea61fec949760f922931ca0f0b01749f5ab95/airflow-metrics-0.1.3.tar.gz"
            }
        ],
        "0.1.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "69290edca5e52ff02fef53ff69ea083d",
                    "sha256": "3c614dc8ce9a8710c94b371332c117b398240910dbae14599a968acb964588c6"
                },
                "downloads": -1,
                "filename": "airflow_metrics-0.1.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "69290edca5e52ff02fef53ff69ea083d",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 16201,
                "upload_time": "2019-07-15T18:06:28",
                "url": "https://files.pythonhosted.org/packages/ae/63/0d003b374739fb0b5c9b1cab6f3b0c2b1320e9cb45b5e4d4920e4e555c18/airflow_metrics-0.1.4-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3b9b77384916e709b553ccc319034a36",
                    "sha256": "2bb8b93535560371b2588659fdc94d2c18a403346cf0060dc71753963a9a4206"
                },
                "downloads": -1,
                "filename": "airflow-metrics-0.1.4.tar.gz",
                "has_sig": false,
                "md5_digest": "3b9b77384916e709b553ccc319034a36",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7855,
                "upload_time": "2019-07-15T18:06:29",
                "url": "https://files.pythonhosted.org/packages/b2/a5/a5501abd1a29ad3350e6d6dd277b61a25a9aabe713b0d1317e5cf70a1e62/airflow-metrics-0.1.4.tar.gz"
            }
        ],
        "0.1.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "673799a3c85b210784b135e9b895ddcc",
                    "sha256": "13f7f05646ba7a7a3b81abc37bfa3ef1d180202232ee620338de107742ccf270"
                },
                "downloads": -1,
                "filename": "airflow_metrics-0.1.5-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "673799a3c85b210784b135e9b895ddcc",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 16472,
                "upload_time": "2019-07-15T22:15:07",
                "url": "https://files.pythonhosted.org/packages/ea/fc/58bc930282268fc2f3bc0d65856e8e564c78f53b4a7a3058dfe9465ca5d7/airflow_metrics-0.1.5-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3df278a5231ff311756daa0418de763c",
                    "sha256": "864f034bc2c9881d47c1c2753e673b525e895fac644ae21d1204cf12006badb8"
                },
                "downloads": -1,
                "filename": "airflow-metrics-0.1.5.tar.gz",
                "has_sig": false,
                "md5_digest": "3df278a5231ff311756daa0418de763c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7985,
                "upload_time": "2019-07-15T22:15:08",
                "url": "https://files.pythonhosted.org/packages/39/84/191fb3d6f42d26e801ca076d1db4e55a0e4d94d9b89e7fe95f83a9a2c4c2/airflow-metrics-0.1.5.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "673799a3c85b210784b135e9b895ddcc",
                "sha256": "13f7f05646ba7a7a3b81abc37bfa3ef1d180202232ee620338de107742ccf270"
            },
            "downloads": -1,
            "filename": "airflow_metrics-0.1.5-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "673799a3c85b210784b135e9b895ddcc",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 16472,
            "upload_time": "2019-07-15T22:15:07",
            "url": "https://files.pythonhosted.org/packages/ea/fc/58bc930282268fc2f3bc0d65856e8e564c78f53b4a7a3058dfe9465ca5d7/airflow_metrics-0.1.5-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "3df278a5231ff311756daa0418de763c",
                "sha256": "864f034bc2c9881d47c1c2753e673b525e895fac644ae21d1204cf12006badb8"
            },
            "downloads": -1,
            "filename": "airflow-metrics-0.1.5.tar.gz",
            "has_sig": false,
            "md5_digest": "3df278a5231ff311756daa0418de763c",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 7985,
            "upload_time": "2019-07-15T22:15:08",
            "url": "https://files.pythonhosted.org/packages/39/84/191fb3d6f42d26e801ca076d1db4e55a0e4d94d9b89e7fe95f83a9a2c4c2/airflow-metrics-0.1.5.tar.gz"
        }
    ]
}