{
    "info": {
        "author": "Andre Luckow",
        "author_email": "aluckow@clemson.edu",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Environment :: Console",
            "Programming Language :: Python",
            "Topic :: Utilities"
        ],
        "description": "# SAGA Hadoop\n\nLast Updated: 10/01/2016\n\n# Overview:\n\nUse [SAGA](http://saga-project.github.io/saga-python/) to spawn an Hadoop Cluster within an HPC batch job.\n\nCurrently supported SAGA adaptors:\n\n- Fork\n- Torque\n\nRequirements:\n\n\t* PBS/Torque cluster\n\t* Working directory should be on a shared filesystem\n\nBy default SAGA-Hadoop deploys an Hadoop 2.2.0 YARN cluster. The cluster can be customized by adjusting the templates for the Hadoop configuration files in `core-site.xml`, `hdfs-site.xml`, `mapred-site.xml` and `yarn-site.xml` in the `hadoop2/bootstrap_hadoop2.py`.\n\n# Usage\n\n\nTry to run a local Hadoop (e.g. for development and testing)\n\n    easy_install saga-hadoop\n    saga-hadoop --resource fork://localhost\n    \n    \nTry to run a Hadoop cluster inside a PBS/Torque job:\n\n    saga-hadoop --resource pbs+ssh://india.futuregrid.org --number_cores 8\n\nSome Blog Posts about SAGA-Hadoop:\n\n    * <http://randomlydistributed.blogspot.com/2011/01/running-hadoop-10-on-distributed.html>\n\n\n# Packages:\n\nsee `hadoop1` for setting up a Hadoop 1.x.x cluster\n\nsee `hadoop2` for setting up a Hadoop 2.7.x cluster\n \nsee `spark` for setting up a Spark 2.0.x cluster\n\nsee `kafka` for setting up a Kafka 0.10.x cluster\n\n\n# Examples:\n\n\n***Stampede:***\n\n    saga-hadoop --resource=slurm://localhost --queue=normal --walltime=239 --number_cores=256 --project=xxx\n\n\n***Gordon:***\n\n    saga-hadoop --resource=pbs://localhost --walltime=59 --number_cores=16 --project=TG-CCR140028 --framework=spark\n    \n\n***Wrangler***\n\n    export JAVA_HOME=/usr/java/jdk1.8.0_45/\n    saga-hadoop --resource=slurm://localhost --queue=normal --walltime=59 --number_cores=24 --project=xxx\n\n\n",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/drelu/saga-hadoop",
        "keywords": null,
        "license": "License :: OSI Approved :: Apache Software License",
        "maintainer": null,
        "maintainer_email": null,
        "name": "SAGA-Hadoop",
        "package_url": "https://pypi.org/project/SAGA-Hadoop/",
        "platform": null,
        "project_url": "https://pypi.org/project/SAGA-Hadoop/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "https://github.com/drelu/saga-hadoop"
        },
        "release_url": "https://pypi.org/project/SAGA-Hadoop/0.31.2/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "SAGA to launch an Hadoop cluster as a normal batch job on Torque/PBS/SLURM clusters",
        "version": "0.31.2"
    },
    "last_serial": 2375210,
    "releases": {
        "0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "af0d94d47854c079fddbb9eee8b47128",
                    "sha256": "48b6e6b8f6fafc384b66f635403330a2915dcded4d9d033a2b23d2f4fbb9ef69"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "af0d94d47854c079fddbb9eee8b47128",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5652,
                "upload_time": "2013-12-25T19:00:32",
                "url": "https://files.pythonhosted.org/packages/09/83/76f20bfd598bc8eab946d40a259c39f17b66eba9fa204fb27c96013dfdd7/SAGA-Hadoop-0.1.tar.gz"
            }
        ],
        "0.11": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ebfab1c5bdc42fe4fe5e6fc3ffbd3786",
                    "sha256": "8c73be80c5612abed8f422ef60237d764fa8bd399580b8b8f6f068fa727546b0"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.11.tar.gz",
                "has_sig": false,
                "md5_digest": "ebfab1c5bdc42fe4fe5e6fc3ffbd3786",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9964,
                "upload_time": "2013-12-26T02:22:55",
                "url": "https://files.pythonhosted.org/packages/27/b9/80cae4f9feca7685e4ace3728cbee636920e8393a177cbcb196f2f05f1a9/SAGA-Hadoop-0.11.tar.gz"
            }
        ],
        "0.12": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "71217d2f8d71f13234c814b89c830a7a",
                    "sha256": "8bd4ab5ccff98943ab2e7b62bf4d235a33b7a942bf9821225802cc4b23aeb6b2"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.12.tar.gz",
                "has_sig": false,
                "md5_digest": "71217d2f8d71f13234c814b89c830a7a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10049,
                "upload_time": "2013-12-26T02:56:25",
                "url": "https://files.pythonhosted.org/packages/0f/f4/3a82bcc7b497ff26b1ee164ea2271ab73a8f987a40ee3650341b03778995/SAGA-Hadoop-0.12.tar.gz"
            }
        ],
        "0.13": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "960ae19aa07a608d553f568f3666a8a2",
                    "sha256": "77a896664736c3ad6958dbb78e9434d2da75d62a1d3d24ce32c0b7c49d9507d5"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.13.tar.gz",
                "has_sig": false,
                "md5_digest": "960ae19aa07a608d553f568f3666a8a2",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10108,
                "upload_time": "2013-12-26T03:03:18",
                "url": "https://files.pythonhosted.org/packages/64/5a/d38d667d30ff86c0762283ab0ca01a38eaa28b0249dca1c40f8f8a241a94/SAGA-Hadoop-0.13.tar.gz"
            }
        ],
        "0.14": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f70791185f660cd24d33ac9c00b64c97",
                    "sha256": "17467bd1481dd6372382ca4fd692e5743594b5ff3e0e02c175c8985f651f8789"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.14.tar.gz",
                "has_sig": false,
                "md5_digest": "f70791185f660cd24d33ac9c00b64c97",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10112,
                "upload_time": "2013-12-26T03:14:17",
                "url": "https://files.pythonhosted.org/packages/4d/7b/8259db4f2aed924dfa6687233cc394fe6ec3f31066bf8e3bcb059957daf5/SAGA-Hadoop-0.14.tar.gz"
            }
        ],
        "0.15-1-gbbb8b32": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "96e76c7815fbd22145e5d33b8c15ff58",
                    "sha256": "1f2b47b3c73bc906c77b4c92525ef7d542529a74ef92dbf171cf27bcb48dd226"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.15-1-gbbb8b32.tar.gz",
                "has_sig": false,
                "md5_digest": "96e76c7815fbd22145e5d33b8c15ff58",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10160,
                "upload_time": "2013-12-26T03:30:22",
                "url": "https://files.pythonhosted.org/packages/2e/a4/d94ee02ca5a36f93ba67c3ad5c2b8afcae7422f8ac14ecd1546f7f71acc9/SAGA-Hadoop-0.15-1-gbbb8b32.tar.gz"
            }
        ],
        "0.16": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "274398a28f6756eaf618a9c961746b29",
                    "sha256": "26b522608ee635a90f80dc795f9c4c5853988c40429c060f7d9ca9a41c133e22"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.16.tar.gz",
                "has_sig": false,
                "md5_digest": "274398a28f6756eaf618a9c961746b29",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10168,
                "upload_time": "2013-12-26T03:38:23",
                "url": "https://files.pythonhosted.org/packages/01/e0/a463ce5ccc549c853e7f237a8123a8890cc28f9d9c42a21eebdd9796119f/SAGA-Hadoop-0.16.tar.gz"
            }
        ],
        "0.17": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fd9a63be3b300d873261363d28811104",
                    "sha256": "0b415c1bc81ad9ed14c1399d07c8daf6517445c100874a5b76e259b131de745f"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.17.tar.gz",
                "has_sig": false,
                "md5_digest": "fd9a63be3b300d873261363d28811104",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10163,
                "upload_time": "2013-12-26T03:43:33",
                "url": "https://files.pythonhosted.org/packages/f4/e3/7c709970e64ee86c623809c5f3d38c67ea3dcfc814f15988e04a10986f37/SAGA-Hadoop-0.17.tar.gz"
            }
        ],
        "0.18": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e01bfa25f0785fbc882ac54ec3211fd8",
                    "sha256": "88f6f73fceb12f9199dbf506d671d70d6363d22c735276ec57a13ab26c5cf281"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.18.tar.gz",
                "has_sig": false,
                "md5_digest": "e01bfa25f0785fbc882ac54ec3211fd8",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10175,
                "upload_time": "2014-01-02T02:31:25",
                "url": "https://files.pythonhosted.org/packages/64/c4/dedd80b9db91313386f465660e6b7d369cf463cfb140a27f79673ccf82b6/SAGA-Hadoop-0.18.tar.gz"
            }
        ],
        "0.19": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "68c309e25a60edb1661639705264defa",
                    "sha256": "ce882c4f2480e496398dc01a36df45767ce4fdf4363fc7127fb547bc0509c934"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.19.tar.gz",
                "has_sig": false,
                "md5_digest": "68c309e25a60edb1661639705264defa",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10247,
                "upload_time": "2014-01-02T03:48:17",
                "url": "https://files.pythonhosted.org/packages/a7/ee/0050b7851c1ab90884919e65c6d992e5edb2d88243964963734ada441794/SAGA-Hadoop-0.19.tar.gz"
            }
        ],
        "0.20": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d843d96624dafc4eeceff2572e89f028",
                    "sha256": "fe439f89cbeb93dc4af6550617575ca911fa88839b4e3e201c1221116763f51a"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.20.tar.gz",
                "has_sig": false,
                "md5_digest": "d843d96624dafc4eeceff2572e89f028",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10184,
                "upload_time": "2014-01-11T15:16:39",
                "url": "https://files.pythonhosted.org/packages/f7/41/16d35762446a51376ff42426734e9c110d13b76882d2708b29eeb622ce60/SAGA-Hadoop-0.20.tar.gz"
            }
        ],
        "0.21": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9a1acb997b0428ce38e2006fe6c55bf1",
                    "sha256": "ac75487bc778836604be60d1d742084ffb15ae6f775d12a2bc059fd5bb79a2de"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.21.tar.gz",
                "has_sig": false,
                "md5_digest": "9a1acb997b0428ce38e2006fe6c55bf1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10205,
                "upload_time": "2014-06-18T01:10:36",
                "url": "https://files.pythonhosted.org/packages/7e/bc/9d78aaf269df40d46dfeb0b4230e3f58b873aea1bcb73772f2d942fa73e9/SAGA-Hadoop-0.21.tar.gz"
            }
        ],
        "0.22": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "aff7d5e4bec39cd80561ffe42ee51b7f",
                    "sha256": "f47adcb8664c8eb14b1d568536ec3dae5ed68cb2e64034c75bc2b4c8b72420ad"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.22.tar.gz",
                "has_sig": false,
                "md5_digest": "aff7d5e4bec39cd80561ffe42ee51b7f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10565,
                "upload_time": "2014-06-19T02:07:39",
                "url": "https://files.pythonhosted.org/packages/7e/65/ca797f3d4ded3d8d4878c92363d2474743b499c3909265ad1ce6b612f132/SAGA-Hadoop-0.22.tar.gz"
            }
        ],
        "0.25": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "faa0a779c7307faf9e5199640bb16b8c",
                    "sha256": "fcab6a684a8a94d14acabd5988ca62cc9d9804fcadfea62e4c00fec15b67ce3c"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.25.tar.gz",
                "has_sig": false,
                "md5_digest": "faa0a779c7307faf9e5199640bb16b8c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25251,
                "upload_time": "2015-10-18T15:04:55",
                "url": "https://files.pythonhosted.org/packages/98/91/5f8e86d38a72e34fd29350a47d52967488df81c7d04bddf7e97fa7befc47/SAGA-Hadoop-0.25.tar.gz"
            }
        ],
        "0.26": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3b2dda42aa218daf3a12170e1263cb34",
                    "sha256": "48b8cf1abd7001bca4d3399c096a133a7bb9f25a558df2d78c05eda3e826c5e6"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.26.tar.gz",
                "has_sig": false,
                "md5_digest": "3b2dda42aa218daf3a12170e1263cb34",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25371,
                "upload_time": "2015-11-10T01:14:15",
                "url": "https://files.pythonhosted.org/packages/6f/a6/ca19664ff5372624490d6c42103a29523be411a915dd984803687ed0b6e7/SAGA-Hadoop-0.26.tar.gz"
            }
        ],
        "0.27": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7c6ce5e54cafb97c23227c5a6508af93",
                    "sha256": "b8ef8ec827d5afd51f832ab00c8a9179ccc086036d06ed805fe05c5bfbf4faff"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.27.tar.gz",
                "has_sig": false,
                "md5_digest": "7c6ce5e54cafb97c23227c5a6508af93",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25365,
                "upload_time": "2015-11-13T13:36:29",
                "url": "https://files.pythonhosted.org/packages/52/56/cfa9db82383dec131f3a8386ac9a9e32b7a71d2a04cab357fa43433a4266/SAGA-Hadoop-0.27.tar.gz"
            }
        ],
        "0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d86bb145fda084738220305559f8d8d9",
                    "sha256": "fe501fcb6f68ae53d1368ad41c37e6bee03ec5c7ab0e195c0409c31325f7c863"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "d86bb145fda084738220305559f8d8d9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 32297,
                "upload_time": "2016-10-01T18:16:10",
                "url": "https://files.pythonhosted.org/packages/49/9b/65c6d10b0d1dda2c4ce97d2d07a00e1f74fbb853b4b1c87665465f28cfa8/SAGA-Hadoop-0.3.tar.gz"
            }
        ],
        "0.31.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3d7d2476587b80e726755af28bc131dd",
                    "sha256": "f28a0a288048ddec6321fb82804482715273ce58a3c7ae6ad1a67c9092276b71"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.31.1.tar.gz",
                "has_sig": false,
                "md5_digest": "3d7d2476587b80e726755af28bc131dd",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 36104,
                "upload_time": "2016-10-01T18:20:25",
                "url": "https://files.pythonhosted.org/packages/98/75/9962388c77954f8002465e65e590bfbcdcdd8e0a2041ffa38ef372c86f47/SAGA-Hadoop-0.31.1.tar.gz"
            }
        ],
        "0.31.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "354cdfd5833e5410148dd3085b0bb500",
                    "sha256": "9a672846094152c3fd45b54959d1fd52afcf6a139da196cf32754aabbf4ee277"
                },
                "downloads": -1,
                "filename": "SAGA-Hadoop-0.31.2.tar.gz",
                "has_sig": false,
                "md5_digest": "354cdfd5833e5410148dd3085b0bb500",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 36152,
                "upload_time": "2016-10-01T18:22:47",
                "url": "https://files.pythonhosted.org/packages/04/e4/13f73dd51ad57e7670f87a56fdabf4993295dd09cbb551f4b9e541b11690/SAGA-Hadoop-0.31.2.tar.gz"
            }
        ],
        "e91fe6b": []
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "354cdfd5833e5410148dd3085b0bb500",
                "sha256": "9a672846094152c3fd45b54959d1fd52afcf6a139da196cf32754aabbf4ee277"
            },
            "downloads": -1,
            "filename": "SAGA-Hadoop-0.31.2.tar.gz",
            "has_sig": false,
            "md5_digest": "354cdfd5833e5410148dd3085b0bb500",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 36152,
            "upload_time": "2016-10-01T18:22:47",
            "url": "https://files.pythonhosted.org/packages/04/e4/13f73dd51ad57e7670f87a56fdabf4993295dd09cbb551f4b9e541b11690/SAGA-Hadoop-0.31.2.tar.gz"
        }
    ]
}