{
    "info": {
        "author": "Mark Libucha",
        "author_email": "mlibucha@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: Apache Software License",
            "Operating System :: OS Independent",
            "Programming Language :: Python",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: Implementation :: CPython"
        ],
        "description": "HPDR: Hive Partition Date Ranges\n================================\n\nWhen your Hive partitions are YYYY, MM, DD, HH and not YYYYMMDDHH, composing date\nranges with them can get out of control.\n\nhpdr solves this problem.\n\n.. code-block:: python\n\n    >>> from hpdr import api\n    >>> rng = api.build('2016102612', '2017122612').partition_range\n    >>> print(rng.build_display())\n    ((YYYY=2016 AND MM=10 AND DD=26 AND HH>=12) OR (YYYY=2016 AND MM=10 AND DD>26)\n    OR (YYYY=2016 AND MM>10) OR (YYYY=2017 AND MM<12) OR (YYYY=2017 AND MM=12 AND DD<26)\n    OR (YYYY=2017 AND MM=12 AND DD=26 AND HH<12))\n    >>> print(rng.build_display(pretty=True))\n    (\n         (YYYY=2016 AND MM=10 AND DD=26 AND HH>=12)\n      OR (YYYY=2016 AND MM=10 AND DD>26)\n      OR (YYYY=2016 AND MM>10)\n      OR (YYYY=2017 AND MM<12)\n      OR (YYYY=2017 AND MM=12 AND DD<26)\n      OR (YYYY=2017 AND MM=12 AND DD=26 AND HH<12)\n    )\n\nMaybe you think in local time but store your data in UTC?\n\n.. code-block:: python\n\n    >>> from hpdr import api\n    >>> rng = api.build('2016102612', '2017122612',\n    ...                 dzone='America/Los_Angeles',\n    ...                 qzone='UTC').partition_range\n    >>> print(rng.build_display(pretty=True))\n    (\n         (YYYY=2016 AND MM=10 AND DD=26 AND HH>=19)\n      OR (YYYY=2016 AND MM=10 AND DD>26)\n      OR (YYYY=2016 AND MM>10)\n      OR (YYYY=2017 AND MM<12)\n      OR (YYYY=2017 AND MM=12 AND DD<26)\n      OR (YYYY=2017 AND MM=12 AND DD=26 AND HH<20)\n\nOr maybe your date range is too large to run in one query, and it's a pain to break it down?\n\n.. code-block:: python\n\n    import subprocess, os, os.path, tempfile, datetime\n    from hpdr import api\n    \n    QUERY_FILE = 'myquery.hql'\n    OUT_FILE = 'out.txt'\n    begin = datetime.datetime(2016, 11, 1)\n    end = datetime.datetime(2016, 11, 30)\n    step = '5days'\n    \n    with open(QUERY_FILE) as f:\n        template = f.read()\n    \n    specs = api.build_with_steps(begin=begin, end=end, step=step)\n    \n    if os.path.isfile(OUT_FILE):\n        os.remove(OUT_FILE)\n    \n    for spec in specs:\n        query = spec.substitute(template)\n        with tempfile.NamedTemporaryFile() as f:\n            f.write(query)\n            f.flush()\n            cmd = ['/usr/bin/hive', '-f',  f.name]\n            print(spec.partition_range.build_display())\n            with open(OUT_FILE, 'a') as outfile:\n                subprocess.check_call(cmd, stdout=outfile)\n\nThis prints::\n\n    (YYYY=2016 AND MM=11 AND DD>=01 AND DD<06)\n    (YYYY=2016 AND MM=11 AND DD>=06 AND DD<11)\n    (YYYY=2016 AND MM=11 AND DD>=11 AND DD<16)\n    (YYYY=2016 AND MM=11 AND DD>=16 AND DD<21)\n    (YYYY=2016 AND MM=11 AND DD>=21 AND DD<26)\n    (YYYY=2016 AND MM=11 AND DD>=26 AND DD<30)\n\nIt runs 6 Hive queries built from a template containing HPDR\\_ variables. Something like this:\n\n::\n\n    SELECT\n      YEAR(event_timestamp),\n      MONTH(event_timestamp),\n      DAY(event_timestamp),\n      FROM my_table\n      WHERE event_timestamp >= '${HPDR_begin_ts}'\n        AND event_timestamp < '${HPDR_end_ts}'\n        AND ${HPDR_range}\n\nThe first query looks like this.\n\n::\n\n    SELECT\n      YEAR(event_timestamp),\n      MONTH(event_timestamp),\n      DAY(event_timestamp),\n      FROM my_table\n      WHERE event_timestamp >= '2016-11-01 00:00:00'\n        AND event_timestamp < '2016-11-06 00:00:00'\n        AND (YYYY=2016 AND MM=11 AND DD>=01 AND DD<06)\n\nThe full list of HPDR\\_ variables available for that first query is::\n\n    variable                     value\n    ---------------------------  -------------------\n    HPDR_dzone                   UTC\n    HPDR_qzone                   UTC\n    HPDR_begin_ts                2016-11-01 00:00:00\n    HPDR_end_ts                  2016-11-06 00:00:00\n    HPDR_slop_begin_ts           2016-11-01 00:00:00\n    HPDR_slop_end_ts             2016-11-06 00:00:00\n    HPDR_begin_unixtime          1477983600\n    HPDR_begin_unixtime_ms       1477983600000\n    HPDR_begin_yyyymmdd          20161101\n    HPDR_begin_yyyy              2016\n    HPDR_begin_mm                11\n    HPDR_begin_dd                01\n    HPDR_begin_hh                00\n    HPDR_begin_min               00\n    HPDR_begin_sec               00\n    HPDR_end_unixtime            1478415600\n    HPDR_end_unixtime_ms         1478415600000\n    HPDR_end_yyyymmdd            20161106\n    HPDR_end_yyyy                2016\n    HPDR_end_mm                  11\n    HPDR_end_dd                  06\n    HPDR_end_hh                  00\n    HPDR_end_min                 00\n    HPDR_end_sec                 00\n    HPDR_slop_begin_unixtime     1477983600\n    HPDR_slop_begin_unixtime_ms  1477983600000\n    HPDR_slop_begin_yyyymmdd     20161101\n    HPDR_slop_begin_yyyy         2016\n    HPDR_slop_begin_mm           11\n    HPDR_slop_begin_dd           01\n    HPDR_slop_begin_hh           00\n    HPDR_slop_begin_min          00\n    HPDR_slop_begin_sec          00\n    HPDR_slop_end_unixtime       1478415600\n    HPDR_slop_end_unixtime_ms    1478415600000\n    HPDR_slop_end_yyyymmdd       20161106\n    HPDR_slop_end_yyyy           2016\n    HPDR_slop_end_mm             11\n    HPDR_slop_end_dd             06\n    HPDR_slop_end_hh             00\n    HPDR_slop_end_min            00\n    HPDR_slop_end_sec            00\n",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://hpdr.readthedocs.io/en/latest/",
        "keywords": null,
        "license": "Apache 2.0",
        "maintainer": null,
        "maintainer_email": null,
        "name": "hpdr",
        "package_url": "https://pypi.org/project/hpdr/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/hpdr/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "http://hpdr.readthedocs.io/en/latest/"
        },
        "release_url": "https://pypi.org/project/hpdr/1.0.0/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Hive date ranges simplified.",
        "version": "1.0.0"
    },
    "last_serial": 2809035,
    "releases": {
        "0.13": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "38dd5c61ade07d1c5b93321395087705",
                    "sha256": "1aabfef4ded5941de27d5ae0060ae2bb7f15cd2677455a31cc1685274caf26ea"
                },
                "downloads": -1,
                "filename": "hpdr-0.13.tar.gz",
                "has_sig": false,
                "md5_digest": "38dd5c61ade07d1c5b93321395087705",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12989,
                "upload_time": "2017-01-03T00:56:08",
                "url": "https://files.pythonhosted.org/packages/9d/39/f840efab9a3f00ad3c48afabd1ffe7327bc71df0b4f1b1dc554c568f533d/hpdr-0.13.tar.gz"
            }
        ],
        "0.14": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "861775aeb6b352701b1aefcb4bd6cb2f",
                    "sha256": "8da1616df9c6d454a22ab20c6f6877b5d6dbd3ca890c0ae7741a8c63168bee02"
                },
                "downloads": -1,
                "filename": "hpdr-0.14.tar.gz",
                "has_sig": false,
                "md5_digest": "861775aeb6b352701b1aefcb4bd6cb2f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 13008,
                "upload_time": "2017-01-03T03:42:02",
                "url": "https://files.pythonhosted.org/packages/6d/d9/c07f743c68bbda40645dc9ba97706062f11e3201c7de91b2968cec4eb815/hpdr-0.14.tar.gz"
            }
        ],
        "0.15": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fc29856c869567a6e9d5744f6633c99c",
                    "sha256": "a97507297ae6ab2ffbc883d23208c729df65ec1618219dff7c5e150e4e073e7b"
                },
                "downloads": -1,
                "filename": "hpdr-0.15.tar.gz",
                "has_sig": false,
                "md5_digest": "fc29856c869567a6e9d5744f6633c99c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 13036,
                "upload_time": "2017-01-04T04:25:36",
                "url": "https://files.pythonhosted.org/packages/62/b9/ad223e9be18428cdc299a41c932c449dcca37bf5191d3668da9a5d69e86c/hpdr-0.15.tar.gz"
            }
        ],
        "0.16": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2eed4f6666709ccd08ce3c3ba0624d0a",
                    "sha256": "754181ad63f0bd28ea4128237d41f24ca2638fb55cdf68c5c3716fa895618102"
                },
                "downloads": -1,
                "filename": "hpdr-0.16.tar.gz",
                "has_sig": false,
                "md5_digest": "2eed4f6666709ccd08ce3c3ba0624d0a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 13324,
                "upload_time": "2017-02-26T04:00:04",
                "url": "https://files.pythonhosted.org/packages/d5/00/e2f602e16ec36cd863981c7de6298acf95065f1c59c6a697d45511b2a979/hpdr-0.16.tar.gz"
            }
        ],
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "53e3f2db8b736fa496ed47d03ab8bb55",
                    "sha256": "c0680e463d45291dd949d28eb1901bc6644bf680ff817c2daa422e6a6438f791"
                },
                "downloads": -1,
                "filename": "hpdr-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "53e3f2db8b736fa496ed47d03ab8bb55",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12967,
                "upload_time": "2017-04-15T18:12:37",
                "url": "https://files.pythonhosted.org/packages/a2/ab/d3e7c1a6ff25f9697e89d85889473dd6ab178babb02d62fd9574d51074b4/hpdr-1.0.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "53e3f2db8b736fa496ed47d03ab8bb55",
                "sha256": "c0680e463d45291dd949d28eb1901bc6644bf680ff817c2daa422e6a6438f791"
            },
            "downloads": -1,
            "filename": "hpdr-1.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "53e3f2db8b736fa496ed47d03ab8bb55",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 12967,
            "upload_time": "2017-04-15T18:12:37",
            "url": "https://files.pythonhosted.org/packages/a2/ab/d3e7c1a6ff25f9697e89d85889473dd6ab178babb02d62fd9574d51074b4/hpdr-1.0.0.tar.gz"
        }
    ]
}