{
    "info": {
        "author": "Civitaspo",
        "author_email": "civitaspo@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7"
        ],
        "description": "airflow-plugin-glue_presto_apas\n==========================\n\n[![PyPi](https://img.shields.io/pypi/v/airflow-plugin-glue-presto-apas.svg)](https://pypi.org/project/airflow-plugin-glue-presto-apas/)\n\nAn Airflow Plugin to **Add a Partition As Select(APAS)** on Presto that uses Glue Data Catalog as a Hive metastore.\n\n# Usage\n\n```\nfrom datetime import timedelta\n\nimport airflow\nfrom airflow.models import DAG\n\nfrom airflow.operators.glue_add_partition import GlueAddPartitionOperator\nfrom airflow.operators.glue_presto_apas import GluePrestoApasOperator\n\nargs = {\n    'owner': 'airflow',\n    'depends_on_past': False,\n    'start_date': airflow.utils.dates.days_ago(2),\n    'email_on_failure': False,\n    'email_on_retry': False,\n    'retries': 1,\n    'retry_delay': timedelta(minutes=5),\n}\n\n\ndag = DAG(\n    dag_id='example-dag',\n    schedule_interval='0 0 * * *',\n    default_args=args,\n)\n\nGluePrestoApasOperator(task_id='example-task-1',\n                       db='example_db',\n                       table='example_table',\n                       sql='example.sql',\n                       partition_kv={\n                           'table_schema': 'example_db',\n                           'table_name': 'example_table'\n                       },\n                       catalog_region_name='ap-northeast-1',\n                       dag=dag,\n                       )\n\nGlueAddPartitionOperator(task_id='example-task-2',\n                         db='example_db',\n                         table='example_table',\n                         partition_kv={\n                             'table_schema': 'example_db',\n                             'table_name': 'example_table'\n                         },\n                         catalog_region_name='ap-northeast-1',\n                         dag=dag,\n                         )\n\nif __name__ == \"__main__\":\n    dag.cli()\n```\n\n# Configuration\n\n## glue_presto_apas.GluePrestoApasOperator\n\n- **db**: database name for parititioning (string, required)\n- **table**: table name for parititioning (string, required)\n- **sql**: sql file name for selecting data (string, required)\n- **fmt**: data format when storing data (string, default = `parquet`)\n- **additional_properties**: additional properties for creating table. (dict[string, string], optional)\n- **location**: location for the data (string, default = auto generated by hive repairable way)\n- **partition_kv**: key values for partitioning (dict[string, string], required)\n- **save_mode**: mode when storing data (string, default = `overwrite`, available values are `skip_if_exists`, `error_if_exists`, `ignore`, `overwrite`)\n- **catalog_id**: glue data catalog id if you use a catalog different from account/region default catalog. (string, optional)\n- **catalog_region_name**: glue data catalog region if you use a catalog different from account/region default catalog. (string, us-east-1 )\n- **presto_conn_id**: connection id for presto (string, default = 'presto_default')\n- **aws_conn_id**: connection id for aws (string, default = 'aws_default')\n\nTemplates can be used in the options[**db**, **table**, **sql**, **location**, **partition_kv**].\n\n## glue_add_partition.GlueAddPartitionOperator\n\n- **db**: database name for parititioning (string, required)\n- **table**: table name for parititioning (string, required)\n- **location**: location for the data (string, default = auto generated by hive repairable way)\n- **partition_kv**: key values for partitioning (dict[string, string], required)\n- **mode**: mode when storing data (string, default = `overwrite`, available values are `skip_if_exists`, `error_if_exists`, `overwrite`)\n- **follow_location**: Skip to add a partition and drop the partition if the location does not exist. (boolean, default = `True`)\n- **catalog_id**: glue data catalog id if you use a catalog different from account/region default catalog. (string, optional)\n- **catalog_region_name**: glue data catalog region if you use a catalog different from account/region default catalog. (string, us-east-1 )\n- **aws_conn_id**: connection id for aws (string, default = 'aws_default')\n\nTemplates can be used in the options[**db**, **table**, **location**, **partition_kv**].\n\n# Development\n\n## Run Example\n\n```\nPRESTO_HOST=${YOUR PRESTO HOST} PRESTO_PORT=${YOUR PRESTO PORT} ./run-example.sh\n```\n\n## Release\n\n```\npoetry publish --build\n```\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/civitaspo/airflow-plugin-glue_presto_apas",
        "keywords": "",
        "license": "",
        "maintainer": "Civitaspo",
        "maintainer_email": "civitaspo@gmail.com",
        "name": "airflow-plugin-glue-presto-apas",
        "package_url": "https://pypi.org/project/airflow-plugin-glue-presto-apas/",
        "platform": "",
        "project_url": "https://pypi.org/project/airflow-plugin-glue-presto-apas/",
        "project_urls": {
            "Homepage": "https://github.com/civitaspo/airflow-plugin-glue_presto_apas",
            "Repository": "https://github.com/civitaspo/airflow-plugin-glue_presto_apas"
        },
        "release_url": "https://pypi.org/project/airflow-plugin-glue-presto-apas/0.0.11/",
        "requires_dist": [
            "apache-airflow (>=1.10,<2.0)",
            "presto-python-client (>=0.5.1,<0.6.0)"
        ],
        "requires_python": ">=3.6,<4.0",
        "summary": "An Airflow Plugin to Add a Partition As Select(APAS) on Presto that uses Glue Data Catalog as a Hive metastore.",
        "version": "0.0.11"
    },
    "last_serial": 5290263,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "814fbc1ad5a53ec75ed1a81402032f9a",
                    "sha256": "9a8959051ac7c9a50cc75ee7c3ae14fa48162276fa8463024799762b7ae87b9c"
                },
                "downloads": -1,
                "filename": "airflow_plugin_glue_presto_apas-0.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "814fbc1ad5a53ec75ed1a81402032f9a",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.7,<4.0",
                "size": 35067,
                "upload_time": "2019-03-18T06:33:37",
                "url": "https://files.pythonhosted.org/packages/23/68/b63c42da8407bea20acd6808bb8f6e2bf74ebdbeb3d63224943270d3ef03/airflow_plugin_glue_presto_apas-0.0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "5ddc39ef996670032461f6adccb55f7b",
                    "sha256": "184a91478e071da2fbe463be0b41fd3ea86431967dfcd60438ce5799230fcc82"
                },
                "downloads": -1,
                "filename": "airflow-plugin-glue_presto_apas-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "5ddc39ef996670032461f6adccb55f7b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.7,<4.0",
                "size": 10596,
                "upload_time": "2019-03-18T06:33:35",
                "url": "https://files.pythonhosted.org/packages/38/15/9adaedb2018024af7abf482613e1450cfc19cbfbbbd926d2276bfd280e50/airflow-plugin-glue_presto_apas-0.0.1.tar.gz"
            }
        ],
        "0.0.10": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "42ea6d09673194287020a0f1625f0d47",
                    "sha256": "494a4707da93fe4c10ad71889b1980dec18eacb6524457c67e753711a503f905"
                },
                "downloads": -1,
                "filename": "airflow_plugin_glue_presto_apas-0.0.10-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "42ea6d09673194287020a0f1625f0d47",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6,<4.0",
                "size": 44514,
                "upload_time": "2019-04-12T03:35:14",
                "url": "https://files.pythonhosted.org/packages/48/f1/5cf55b5e58a8a98ff79d2683daa84cff94bdba589bd3f5f5411e79e3ab6e/airflow_plugin_glue_presto_apas-0.0.10-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "fdc6e70204b79c061fca52bd058859fb",
                    "sha256": "f868f6cf6252001b24acb128ec9cfa38f6bc815b5837ef17a51298e094b00e08"
                },
                "downloads": -1,
                "filename": "airflow-plugin-glue_presto_apas-0.0.10.tar.gz",
                "has_sig": false,
                "md5_digest": "fdc6e70204b79c061fca52bd058859fb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6,<4.0",
                "size": 12464,
                "upload_time": "2019-04-12T03:35:12",
                "url": "https://files.pythonhosted.org/packages/ff/16/1be2ecb172c7cc898d5473d1473de125e4b96e4afc4bc5835955cc577999/airflow-plugin-glue_presto_apas-0.0.10.tar.gz"
            }
        ],
        "0.0.11": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d9aefa8ff14ae8046016a5d826b3a7fa",
                    "sha256": "a398a67d1d973178a3cdd9ea23e057eaf68e2bf7608b95faf79f0f65b696ac9c"
                },
                "downloads": -1,
                "filename": "airflow_plugin_glue_presto_apas-0.0.11-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "d9aefa8ff14ae8046016a5d826b3a7fa",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6,<4.0",
                "size": 45229,
                "upload_time": "2019-05-20T04:31:22",
                "url": "https://files.pythonhosted.org/packages/07/a5/b544b0c80b229247a433e3cc68786fd5a9972c51f8a2af98112040f96b4d/airflow_plugin_glue_presto_apas-0.0.11-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "569ebebeaa15d8e6e05f2d017c51cd6b",
                    "sha256": "8846a69bf63f76d4368f7de3aae12557bd6e27bb977a914464f07c4951c5e5a7"
                },
                "downloads": -1,
                "filename": "airflow-plugin-glue_presto_apas-0.0.11.tar.gz",
                "has_sig": false,
                "md5_digest": "569ebebeaa15d8e6e05f2d017c51cd6b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6,<4.0",
                "size": 12646,
                "upload_time": "2019-05-20T04:31:20",
                "url": "https://files.pythonhosted.org/packages/73/05/6f689e4c1e9e21b5563b6c6a976491e75a0404afb5080db3e3da8298ce58/airflow-plugin-glue_presto_apas-0.0.11.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "dfde980f498d3f43d86878ad05a6a8cc",
                    "sha256": "828875b95361b63581d6f4ef7b3b65e0d96d8b4facf6bd28f55d6f942e8056a6"
                },
                "downloads": -1,
                "filename": "airflow_plugin_glue_presto_apas-0.0.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "dfde980f498d3f43d86878ad05a6a8cc",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6,<4.0",
                "size": 35062,
                "upload_time": "2019-03-18T13:05:37",
                "url": "https://files.pythonhosted.org/packages/c4/b7/6862b7f09d3e84cfa4142dec6f1b5f333d8a3050a058fa194d6391980d2b/airflow_plugin_glue_presto_apas-0.0.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "07bd54b6d3ffd6b16322b74271a82a52",
                    "sha256": "21234e93aa8bd1c60ad1d8225200d50d7efd60d716f4478de5491a0f090e0206"
                },
                "downloads": -1,
                "filename": "airflow-plugin-glue_presto_apas-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "07bd54b6d3ffd6b16322b74271a82a52",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6,<4.0",
                "size": 10558,
                "upload_time": "2019-03-18T13:05:35",
                "url": "https://files.pythonhosted.org/packages/9c/06/e55fa15516858e67083daaf312e863b1f768121e8af483246300065b19a6/airflow-plugin-glue_presto_apas-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "01580df710062e8716ab651c949bacd6",
                    "sha256": "14a6c36c187812688f6171dd7f505ba33814decdf7077dd991a6f8c2bf06fe33"
                },
                "downloads": -1,
                "filename": "airflow_plugin_glue_presto_apas-0.0.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "01580df710062e8716ab651c949bacd6",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6,<4.0",
                "size": 37827,
                "upload_time": "2019-03-19T07:28:14",
                "url": "https://files.pythonhosted.org/packages/ab/94/3acb522815ed4be050672be1c4732540961cab76298609468fb332515e2c/airflow_plugin_glue_presto_apas-0.0.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "354447a05a0c2b76d16085acabbb71ad",
                    "sha256": "9ba029eb1d49528d5ae84450b82f203b98ee9c9d0a15abb2b2676eef180db80f"
                },
                "downloads": -1,
                "filename": "airflow-plugin-glue_presto_apas-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "354447a05a0c2b76d16085acabbb71ad",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6,<4.0",
                "size": 10864,
                "upload_time": "2019-03-19T07:28:13",
                "url": "https://files.pythonhosted.org/packages/e0/33/9df2ef6d883f33cdafb6075ff0961cacd383d53ee33e8739e3bb69002e5f/airflow-plugin-glue_presto_apas-0.0.3.tar.gz"
            }
        ],
        "0.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "cee98a26f1ac49b2848f6f4482e70ea7",
                    "sha256": "41ff0c35ddbe27841cbbb3e1c0ace706faa7db0d0755c0f9f564dc6b6210f199"
                },
                "downloads": -1,
                "filename": "airflow_plugin_glue_presto_apas-0.0.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "cee98a26f1ac49b2848f6f4482e70ea7",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6,<4.0",
                "size": 39599,
                "upload_time": "2019-03-25T07:59:58",
                "url": "https://files.pythonhosted.org/packages/56/7f/ed5c31350bc44d3a2ac66a276fe8b47b46f76983d2e0a8a6498d6f827edd/airflow_plugin_glue_presto_apas-0.0.4-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3e5e52e0c6dbef55e4227a4850177c68",
                    "sha256": "f6a4f9731cb8089865670a66f11fd34e44b75093ffcb170bd9237f9869c8b28f"
                },
                "downloads": -1,
                "filename": "airflow-plugin-glue_presto_apas-0.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "3e5e52e0c6dbef55e4227a4850177c68",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6,<4.0",
                "size": 11315,
                "upload_time": "2019-03-25T07:59:56",
                "url": "https://files.pythonhosted.org/packages/07/40/64b6b7717900a0aa7b32bc7ef1dc8d2b8e01cdd7b61ef3a14178044d7516/airflow-plugin-glue_presto_apas-0.0.4.tar.gz"
            }
        ],
        "0.0.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a203e534dfba0cde76e9af45b4181dc9",
                    "sha256": "3e0cfee5235d615f31c5c9b94cdf9cf64c2c24d98e6730bc8e38821ea2f86cfd"
                },
                "downloads": -1,
                "filename": "airflow_plugin_glue_presto_apas-0.0.6-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a203e534dfba0cde76e9af45b4181dc9",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6,<4.0",
                "size": 39813,
                "upload_time": "2019-04-02T09:41:07",
                "url": "https://files.pythonhosted.org/packages/5b/c8/ae4ab37c0364e860334c6298f529ffefd8a4d3b3603a0ea324cd8e76238a/airflow_plugin_glue_presto_apas-0.0.6-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "5fe33bf332576e7b4dc40c8020d8ecad",
                    "sha256": "9db1d3d0932c694a8119c6ab2f0d34ba0c89a878fa51f88043c98cb55bb5ccc2"
                },
                "downloads": -1,
                "filename": "airflow-plugin-glue_presto_apas-0.0.6.tar.gz",
                "has_sig": false,
                "md5_digest": "5fe33bf332576e7b4dc40c8020d8ecad",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6,<4.0",
                "size": 11435,
                "upload_time": "2019-04-02T09:41:06",
                "url": "https://files.pythonhosted.org/packages/1c/55/36916e2b68455d12757a0981641584b0d6a5c8df2956935816d2edaf2e37/airflow-plugin-glue_presto_apas-0.0.6.tar.gz"
            }
        ],
        "0.0.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d69ea1642a46b51d6c60597cd26c2dc7",
                    "sha256": "c931851a7369ca2fe486c2f2448dcc97c8aeba60f6dffc5f8bdf4bae774789f8"
                },
                "downloads": -1,
                "filename": "airflow_plugin_glue_presto_apas-0.0.8-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "d69ea1642a46b51d6c60597cd26c2dc7",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6,<4.0",
                "size": 41925,
                "upload_time": "2019-04-02T14:22:13",
                "url": "https://files.pythonhosted.org/packages/d8/e4/589769eb63a938ee465179146956bbf7be218141634924c2fa55f006e7a3/airflow_plugin_glue_presto_apas-0.0.8-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "b4129a3be12c278e9a7dfa5d56d92143",
                    "sha256": "5b3f92c405b4e908fb07a3c4f2f53f8ae5148a51f03a3ea84623125628ae1e25"
                },
                "downloads": -1,
                "filename": "airflow-plugin-glue_presto_apas-0.0.8.tar.gz",
                "has_sig": false,
                "md5_digest": "b4129a3be12c278e9a7dfa5d56d92143",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6,<4.0",
                "size": 11927,
                "upload_time": "2019-04-02T14:22:11",
                "url": "https://files.pythonhosted.org/packages/e5/9f/10089f03395537bddb7263d74cd2d51788b897aacebcd62d0936cd280679/airflow-plugin-glue_presto_apas-0.0.8.tar.gz"
            }
        ],
        "0.0.9": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7d43990fed9198690e414429328a68d3",
                    "sha256": "397fb5ed8ea97cd3740b4e8422c478af80626b00f0e61ee7bad443e11885de2a"
                },
                "downloads": -1,
                "filename": "airflow_plugin_glue_presto_apas-0.0.9-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "7d43990fed9198690e414429328a68d3",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6,<4.0",
                "size": 43712,
                "upload_time": "2019-04-03T00:14:04",
                "url": "https://files.pythonhosted.org/packages/51/0b/166287ea9707f6f0a8e6fe2b0ee4a8b26d40453f61f0b8d222b6a2ccd197/airflow_plugin_glue_presto_apas-0.0.9-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "b1d2e97dc0f8b4be089bd78f8be97db9",
                    "sha256": "633ae9d490d94e351dcfe279c7b00a6bbd62a886d6e02ff784a696b9692d7c7a"
                },
                "downloads": -1,
                "filename": "airflow-plugin-glue_presto_apas-0.0.9.tar.gz",
                "has_sig": false,
                "md5_digest": "b1d2e97dc0f8b4be089bd78f8be97db9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6,<4.0",
                "size": 12322,
                "upload_time": "2019-04-03T00:14:01",
                "url": "https://files.pythonhosted.org/packages/d8/ed/6d5a0204ad8b0b357ec678f520837d849570b2cb354963d19e55e3206847/airflow-plugin-glue_presto_apas-0.0.9.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "d9aefa8ff14ae8046016a5d826b3a7fa",
                "sha256": "a398a67d1d973178a3cdd9ea23e057eaf68e2bf7608b95faf79f0f65b696ac9c"
            },
            "downloads": -1,
            "filename": "airflow_plugin_glue_presto_apas-0.0.11-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "d9aefa8ff14ae8046016a5d826b3a7fa",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.6,<4.0",
            "size": 45229,
            "upload_time": "2019-05-20T04:31:22",
            "url": "https://files.pythonhosted.org/packages/07/a5/b544b0c80b229247a433e3cc68786fd5a9972c51f8a2af98112040f96b4d/airflow_plugin_glue_presto_apas-0.0.11-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "569ebebeaa15d8e6e05f2d017c51cd6b",
                "sha256": "8846a69bf63f76d4368f7de3aae12557bd6e27bb977a914464f07c4951c5e5a7"
            },
            "downloads": -1,
            "filename": "airflow-plugin-glue_presto_apas-0.0.11.tar.gz",
            "has_sig": false,
            "md5_digest": "569ebebeaa15d8e6e05f2d017c51cd6b",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.6,<4.0",
            "size": 12646,
            "upload_time": "2019-05-20T04:31:20",
            "url": "https://files.pythonhosted.org/packages/73/05/6f689e4c1e9e21b5563b6c6a976491e75a0404afb5080db3e3da8298ce58/airflow-plugin-glue_presto_apas-0.0.11.tar.gz"
        }
    ]
}