{
    "info": {
        "author": "Rob Mansfield",
        "author_email": "rob.mansfield@proagrica.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Intended Audience :: Developers",
            "Intended Audience :: Information Technology",
            "Intended Audience :: Science/Research",
            "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
            "Natural Language :: English",
            "Programming Language :: Python :: 3",
            "Topic :: System :: Distributed Computing",
            "Topic :: Utilities"
        ],
        "description": "hpycc Readme\n============\n\nThe hpycc package is intended to simplify the use of data stored on HPCC and make it easily available to both users and other servers through basic Python calls. Its long-term goal is to make access to and manipulation of HPCC data as quick and easy as any other type system. \n\nDocumentation\n-------------\nThe below readme and package documentation is available at https://hpycc.readthedocs.io/en/latest/\n\nThe package's github is available at: https://github.com/OdinProAgrica/hpycc\n\nThis package is released under GNU GPLv3 Licence: https://www.gnu.org/licenses/gpl-3.0.en.html\n\nWant to use this in R? Then the reticulate package is your friend! Just save as a CSV and read back in. That\nor you can use an R notebook with a Python chunk.\n\n\nInstallation\n------------\nInstall with:\n\npip install hpycc\n\nOr, if you are still a bit old school:\n\npython -m pip install hpycc\n\nCurrent Status\n--------------\nTested and working on HPCC v6.4.2 and python 3.5.2 under windows 10. Has been used on Linux systems but not extensively tested.\n\nDependencies\n------------\nThe package itself mainly uses core Python, Pandas is needed for outputting dataframes.\n\nThere is a dependency for client tools to run ECL scripts (you need ecl.exe and eclcc.exe).\nMake sure you install the right client tools for your HPCC version and add the dir to your system path,\ne.g. C:\\\\Program Files (x86)\\\\HPCCSystems\\\\X.X.X\\\\clienttools\\\\bin.\n\nTests and docker container functions require docker to spin up HPCC environments.\n\nMain Functions\n--------------\nBelow summarises the key functions and non-optional parameters. For specific arguments see the relevant\nfunction's documentation. Note that while retrieving a file is a multi-thread process, running a script\nand getting the results is not. Therefore if your file is quite big you may be better off saving the\nresults of a script using run.run_script() with a thor file output then downloading the file with\nget.get_thor_file().\n\nconnection(username, server=\"localhost\", port=8010, repo=None, password=\"password\", legacy=False, test_conn=True)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nCreate a connection to a new HPCC instance. This is then passed to any interface functions.\n\nget_output(connection, script, ...) & save_output(connection, script, path, ...)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nRun a given ECL script and either return the first result as a pandas dataframe or save it to file.\n\nget_outputs(connection, script, ...)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nRun a given ECL script and return all results as a dict of pandas dataframes or save them to files.\n\nget_thor_file(connection, logical_file, path, ...) & save_thor_file(connection, logical_file, path, ...)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nGet a logical file and either return as a pandas dataframe or save it to file.\n\nrun_script(connection, script, ...)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nRun a given ECL script. 10 rows will be returned but they will be dumped, no output is given.\n\nspray_file(connection, source_file, logical_file, ...)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nSpray a csv or pandas DataFrame into HPCC.\n\ndocker_tools.HPCCContainer(tag=\"6.4.26-1\", ...)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nDesigned for our testing but made available generally, a collection of functions for running and managing\nHPCC docker containers is also available. The above function starts a container, see help file for shutting\ndown and other management tasks.\n\n\nExamples \n--------\nThe below code gives an example of functionality::\n\n    import hpycc\n    import pandas as pd\n    from hpycc.utils import docker_tools\n    from os import remove\n\n    # Start an HPCC docker image for testing\n    docker_tools.HPCCContainer(tag=\"6.4.26-1\")\n\n    # Setup stuff\n    username = 'HPCC_dev'\n    test_file = 'test.csv'\n    f_hpcc_1 = '~temp::testfile1'\n    f_hpcc_2 = '~temp::testfile2'\n    ecl_script = 'ecl_script.ecl'\n\n    # Let's create a connection object so we can interface with HPCC.\n    # up with Docker\n    conn = hpycc.Connection(username, server=\"localhost\")\n    try:\n        # So, let's spray up some data:\n        pd.DataFrame({'col1': [1, 2, 3, 4], 'col2': ['a', 'b', 'c', 'd']}).to_csv(test_file, index=False)\n        hpycc.spray_file(conn, test_file, f_hpcc_1, expire=7)\n\n        # Lovely, we can now extract that as a Thor file:\n        df = hpycc.get_thor_file(conn, f_hpcc_1)\n        print(df)\n        # Note __fileposition__ column. This will be drop-able in future versions.\n\n        #################################\n        #   col1 col2  \\__fileposition__#\n        # 0    1    a                 0 #\n        # 1    3    c                20 #\n        # 2    2    b                10 #\n        # 3    4    d                30 #\n        #################################\n\n        # If preferred data can also be extracted using an ECL script.\n        with open(ecl_script, 'w') as f:\n            f.writelines(\"DATASET('%s', {STRING col1; STRING col2;}, THOR);\" % f_hpcc_1)\n            # Note, all columns are currently string-ified by default\n        df = hpycc.get_output(conn, ecl_script)\n        print(df)\n\n        ################\n        #   col1 col2  #\n        # 0    1    a  #\n        # 1    3    c  #\n        # 2    2    b  #\n        # 3    4    d  #\n        ############## #\n\n\n        # get_thor_file() is optimised for large files, get_output is not (yet). To run a script and\n        # download a large result you should therefore save a thor file and grab that.\n\n        with open(ecl_script, 'w') as f:\n            f.writelines(\"a := DATASET('%s', {STRING col1; STRING col2;}, THOR);\"\n                         \"OUTPUT(a, , '%s');\" % (f_hpcc_1, f_hpcc_2))\n        hpycc.run_script(conn, ecl_script)\n        df = hpycc.get_thor_file(conn, f_hpcc_2)\n        print(df)\n\n        #################################\n        #   col1 col2  \\__fileposition__#\n        # 0    1    a                 0 #\n        # 1    3    c                20 #\n        # 2    2    b                10 #\n        # 3    4    d                30 #\n        #################################\n\n    finally:\n        # Shutdown our docker container\n        docker_tools.HPCCContainer(pull=False, start=False).stop_container()\n        remove(ecl_script)\n        remove(test_file)\n\nIssues, Bugs, Comments? \n-----------------------\nPlease use the package's github: https://github.com/OdinProAgrica/hpycc\n\nAny contributions are also welcome.\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/OdinProAgrica/hpycc",
        "keywords": "HPCC ECL data access distributed ROXIE THOR",
        "license": "GNU GPLv3",
        "maintainer": "",
        "maintainer_email": "",
        "name": "hpycc",
        "package_url": "https://pypi.org/project/hpycc/",
        "platform": "",
        "project_url": "https://pypi.org/project/hpycc/",
        "project_urls": {
            "Bug Reports": "https://github.com/OdinProAgrica/hpycc/issues",
            "Homepage": "https://github.com/OdinProAgrica/hpycc",
            "ReadTheDocs": "http://hpycc.readthedocs.io",
            "Source": "https://github.com/OdinProAgrica/hpycc"
        },
        "release_url": "https://pypi.org/project/hpycc/0.2.2/",
        "requires_dist": [
            "pandas",
            "requests (>=2.18.4)",
            "docker",
            "numpy",
            "passlib",
            "simplejson (==3.16.0)"
        ],
        "requires_python": "",
        "summary": "Download THOR files, run ECL scripts and download their results.",
        "version": "0.2.2"
    },
    "last_serial": 5930086,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6b3e640c77b7441f0268adbe996dc121",
                    "sha256": "0a2d6aeae9586eeef2930cd381dc476833c27afd9065150211ca73e558874892"
                },
                "downloads": -1,
                "filename": "hpycc-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "6b3e640c77b7441f0268adbe996dc121",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14465,
                "upload_time": "2018-05-13T12:36:15",
                "url": "https://files.pythonhosted.org/packages/ed/a8/369ddbccb6d16c555c8a5163161b23254961891b8c0d21647d7aee9d79f5/hpycc-0.0.1.tar.gz"
            }
        ],
        "0.0.1a0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e7f1517517a750bd9184992f98ad650a",
                    "sha256": "aa7a6d58e0490f64f7b696924c6038838347824ff4f3e5193a80b121e71d1148"
                },
                "downloads": -1,
                "filename": "hpycc-0.0.1a0.tar.gz",
                "has_sig": false,
                "md5_digest": "e7f1517517a750bd9184992f98ad650a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14471,
                "upload_time": "2018-05-13T12:37:43",
                "url": "https://files.pythonhosted.org/packages/1d/d8/f4c00646f7b6ec220b0ad61898e3531d8882643610958a8804ab042aaaeb/hpycc-0.0.1a0.tar.gz"
            }
        ],
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0a16513a84f17d0bcfbdf9e1c394b800",
                    "sha256": "de1d43a3f12d06d04f0c9855471c0236acfd5e866df2e6ce8b175970dfc1a6a3"
                },
                "downloads": -1,
                "filename": "hpycc-0.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "0a16513a84f17d0bcfbdf9e1c394b800",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 24529,
                "upload_time": "2018-06-13T09:21:24",
                "url": "https://files.pythonhosted.org/packages/98/d6/973a1684949c0af07be7c453e67a9c8d2f472c0c87a764c57e6aeeb9dbca/hpycc-0.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "fcff8ffc18c77a2bdd3f5002ec1c819d",
                    "sha256": "e6ed0355b7d0743a75daaad66a6923215cdc5a32cdfb708f223e705abe3c1b1f"
                },
                "downloads": -1,
                "filename": "hpycc-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "fcff8ffc18c77a2bdd3f5002ec1c819d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9684,
                "upload_time": "2018-06-13T09:21:52",
                "url": "https://files.pythonhosted.org/packages/8f/5e/f1dfb6e5943949d28ab1d6f5c139498dc03ea5373916135033f6f4027d6f/hpycc-0.1.0.tar.gz"
            }
        ],
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f408ad27d4aa4a0002f9450220ca6af4",
                    "sha256": "eb5d740fb75a8d2c903dfb50f95a137ed261f09203728e651a67782383c30d8e"
                },
                "downloads": -1,
                "filename": "hpycc-0.1.2.tar.gz",
                "has_sig": false,
                "md5_digest": "f408ad27d4aa4a0002f9450220ca6af4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 23685,
                "upload_time": "2018-06-29T08:58:28",
                "url": "https://files.pythonhosted.org/packages/7b/32/5d2fa84a9c72dae07b756f653a93fd72be70e1288b3cb7aa4b27dfe9e460/hpycc-0.1.2.tar.gz"
            }
        ],
        "0.1.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "209c8cb2208100024db3415472e9cfa8",
                    "sha256": "4137198126bde46486160996ed57dc756f060f2c580c8fb29feb1b84fdffc5a5"
                },
                "downloads": -1,
                "filename": "hpycc-0.1.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "209c8cb2208100024db3415472e9cfa8",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 63113,
                "upload_time": "2018-08-16T21:03:12",
                "url": "https://files.pythonhosted.org/packages/45/b1/7d787b795e69c3ef6a54a9fc640de158a273212a6c2e912d0a898b858f5c/hpycc-0.1.4-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "9a70318df5cf2598bacc8b19c99b5f17",
                    "sha256": "03eac4ba442e521c2c14cf225eb15bbad011c3062735628e7c6f104ab7a9fc03"
                },
                "downloads": -1,
                "filename": "hpycc-0.1.4.tar.gz",
                "has_sig": false,
                "md5_digest": "9a70318df5cf2598bacc8b19c99b5f17",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 34609,
                "upload_time": "2018-08-16T21:03:13",
                "url": "https://files.pythonhosted.org/packages/5d/97/1c72aa773ac5e0e69d1e6338a8951b005bcae5a11d4f78c4efe53c8b7710/hpycc-0.1.4.tar.gz"
            }
        ],
        "0.2.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fa6b215d39f030166e516695d16c6602",
                    "sha256": "c650217a28c65cebbc7a5fb29cc138c99917651396367d5fc280a95f3ffea8df"
                },
                "downloads": -1,
                "filename": "hpycc-0.2.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "fa6b215d39f030166e516695d16c6602",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 61771,
                "upload_time": "2018-12-15T20:59:23",
                "url": "https://files.pythonhosted.org/packages/e2/17/66dc2f9492c3661745d64ff03185455d0b4508ceaa9d44a92ef6b3572e4c/hpycc-0.2.0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3b59d05214f27725118ef0373405c537",
                    "sha256": "1464397ad04bfb4df5fa4ec08e903f16094aa0bdda5cd79f904f575ac68e1aba"
                },
                "downloads": -1,
                "filename": "hpycc-0.2.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "3b59d05214f27725118ef0373405c537",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 35504,
                "upload_time": "2018-12-15T20:59:24",
                "url": "https://files.pythonhosted.org/packages/b4/db/03ed462262ec964b200e5da5576621022b6a5026ffe7ad9082b2a5209495/hpycc-0.2.0.1.tar.gz"
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1aac127b3f9db68358b580bf637463f2",
                    "sha256": "8c7a247c59d9a2221af159ed37e768a61e1b021714690bd7dda5591718bc693c"
                },
                "downloads": -1,
                "filename": "hpycc-0.2.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1aac127b3f9db68358b580bf637463f2",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 58738,
                "upload_time": "2019-10-04T20:38:37",
                "url": "https://files.pythonhosted.org/packages/11/c5/75fc94834b4098b89ab37c152300a3679ef48bd6d164074d2fdadcc1f3ab/hpycc-0.2.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7af17d89086851276fe11730e5df0cdd",
                    "sha256": "ee7c9277c6bd31dceb954a2268a2b0af6d48b444762407789297b5c38dde252e"
                },
                "downloads": -1,
                "filename": "hpycc-0.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "7af17d89086851276fe11730e5df0cdd",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 50568,
                "upload_time": "2019-10-04T20:38:39",
                "url": "https://files.pythonhosted.org/packages/d8/70/4b814999b2940ace1c54182e452c1048a568cac02d6d967bee4f1a32abbc/hpycc-0.2.2.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "1aac127b3f9db68358b580bf637463f2",
                "sha256": "8c7a247c59d9a2221af159ed37e768a61e1b021714690bd7dda5591718bc693c"
            },
            "downloads": -1,
            "filename": "hpycc-0.2.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "1aac127b3f9db68358b580bf637463f2",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 58738,
            "upload_time": "2019-10-04T20:38:37",
            "url": "https://files.pythonhosted.org/packages/11/c5/75fc94834b4098b89ab37c152300a3679ef48bd6d164074d2fdadcc1f3ab/hpycc-0.2.2-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "7af17d89086851276fe11730e5df0cdd",
                "sha256": "ee7c9277c6bd31dceb954a2268a2b0af6d48b444762407789297b5c38dde252e"
            },
            "downloads": -1,
            "filename": "hpycc-0.2.2.tar.gz",
            "has_sig": false,
            "md5_digest": "7af17d89086851276fe11730e5df0cdd",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 50568,
            "upload_time": "2019-10-04T20:38:39",
            "url": "https://files.pythonhosted.org/packages/d8/70/4b814999b2940ace1c54182e452c1048a568cac02d6d967bee4f1a32abbc/hpycc-0.2.2.tar.gz"
        }
    ]
}