{
    "info": {
        "author": "Adam Bronte",
        "author_email": "adam@bronte.me",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "=====================================\nPyspark Gateway |Build Status| |PyPi|\n=====================================\n\nPypsark Gateway is a library to seamlessly connect to remote spark clusters.\n\nQuick Start\n-----------\nInstall the pysparkgateway package on both the remote Spark cluster you are connecting to and the local machine.\n\n::\n\n   pip install pysparkgateway\n   \nStart the Pyspark Gateway server on the cluster.\n\n::\n\n   pyspark-gateway start\n   \nPyspark Gateway communicates over 3 ports, ``25000``, ``25001``, ``25002``. Currently the client only supports connecting to these ports on localhost so you'll need to tunnel them.\n\n\n::\n\n   ssh myuser@foo.bar.cluster.com -L 25000:localhost:25000 -L 25001:localhost:25001 -L 25002:localhost:25002\n\nNow you're ready to connect. The main thing to keep in mind is the Pyspark Gateway import **needs to come before any other import.** Pypsark Gateway needs to patch your local pyspark in order to function properly.\n\nThe way that your local Python connects to the remote cluster is via a custom py4j gateway. Pyspark Gateway will create and configure automatically, you just need to pass it into the ``SparkContext`` options.\n\nAlso to enable all pyspark functions to work, ``spark.io.encryption.enabled`` needs to be set to ``true``.\n\n::\n\n   # This import comes first!\n   from pyspark_gateway import PysparkGateway\n   pg = PysparkGateway()\n   \n   from pyspark import SparkContext, SparkConf\n   \n   conf = conf.set('spark.io.encryption.enabled', 'true')\n   sc = SparkContext(gateway=pg.gateway, conf=conf)\n\nNow you have a working spark context connected to a remote cluster.\n\n.. |Build Status| image:: https://travis-ci.org/abronte/PysparkGateway.svg?branch=master\n   :target: https://travis-ci.org/abronte/PysparkGateway\n\n.. |PyPi| image:: https://img.shields.io/pypi/v/pysparkgateway.svg\n   :target: https://pypi.org/project/PysparkGateway/",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/abronte/PysparkGateway",
        "keywords": "",
        "license": "Apache 2.0",
        "maintainer": "",
        "maintainer_email": "",
        "name": "PysparkGateway",
        "package_url": "https://pypi.org/project/PysparkGateway/",
        "platform": "",
        "project_url": "https://pypi.org/project/PysparkGateway/",
        "project_urls": {
            "Homepage": "https://github.com/abronte/PysparkGateway"
        },
        "release_url": "https://pypi.org/project/PysparkGateway/0.0.21/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Connect Pyspark to remote clusters",
        "version": "0.0.21"
    },
    "last_serial": 5950928,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "34ff2e9d5ae4c7ddafa2ad2e25ae420d",
                    "sha256": "d93bd5b379a1e8ae7aa5f2ed41cb2909ec883967e3a27faec90a3d2138faf4b4"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "34ff2e9d5ae4c7ddafa2ad2e25ae420d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5747,
                "upload_time": "2018-12-21T05:45:28",
                "url": "https://files.pythonhosted.org/packages/36/3f/0b04cffd701773f07782548d0557ab129d8d584172f9693287d5ef1969f9/PysparkGateway-0.0.1.tar.gz"
            }
        ],
        "0.0.13": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6e5cff9dc51e472d2106fb73788bbfd9",
                    "sha256": "3868725308bff0e99eb77ca7e2d7c5e70969797643a50acbf9949d56baf957a6"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.13.tar.gz",
                "has_sig": false,
                "md5_digest": "6e5cff9dc51e472d2106fb73788bbfd9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9235,
                "upload_time": "2019-06-06T04:36:01",
                "url": "https://files.pythonhosted.org/packages/ae/7a/8ddb45d75b55f8d73ea9ba317f5c5f974f5118e55b25f11d922ae98cd074/PysparkGateway-0.0.13.tar.gz"
            }
        ],
        "0.0.14": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "72e6f5b423680de2afd41f4c21530e8b",
                    "sha256": "6338d5ba174f95b4a77c47a8721c38c4f3e10abcc2364538fbfc0d9114491958"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.14.tar.gz",
                "has_sig": false,
                "md5_digest": "72e6f5b423680de2afd41f4c21530e8b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9252,
                "upload_time": "2019-06-06T05:56:04",
                "url": "https://files.pythonhosted.org/packages/f1/57/b7e4b51c22b1cb2d2d3e8185f8410f07452fcd7c9a4e5b9291aaf8e6f972/PysparkGateway-0.0.14.tar.gz"
            }
        ],
        "0.0.15": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "009d03172894a81f0593405c29bc422a",
                    "sha256": "be7efe8a85ccd546335f5a21b0d07c1717a5d17b19a4dab03c63588f642d9daa"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.15.tar.gz",
                "has_sig": false,
                "md5_digest": "009d03172894a81f0593405c29bc422a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9260,
                "upload_time": "2019-06-11T22:40:15",
                "url": "https://files.pythonhosted.org/packages/b3/14/6f1f65eef8be8394c499a1ff407f66b47b43df03c5c61cbdf6c346e04b55/PysparkGateway-0.0.15.tar.gz"
            }
        ],
        "0.0.16": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f823392aa8b4a82bbec600008d778f8f",
                    "sha256": "e84cd5fb8770043cb2a892bafb7aeff3eed3a95c194958c1405d476640994257"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.16.tar.gz",
                "has_sig": false,
                "md5_digest": "f823392aa8b4a82bbec600008d778f8f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9262,
                "upload_time": "2019-06-27T17:25:56",
                "url": "https://files.pythonhosted.org/packages/18/23/f07e65c2ab0b1e77ee710a136ba8aac53780fc894e60812a207ed43e0cb9/PysparkGateway-0.0.16.tar.gz"
            }
        ],
        "0.0.17": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d1762adc6963c78403d5b9adf7948f99",
                    "sha256": "f74db19db69590eb8b08c5cf3eec54048c5c47dc87a421d51d65f80b41436f48"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.17.tar.gz",
                "has_sig": false,
                "md5_digest": "d1762adc6963c78403d5b9adf7948f99",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9260,
                "upload_time": "2019-06-27T22:08:45",
                "url": "https://files.pythonhosted.org/packages/ca/88/95a5d8136818befca4f45afb16ae9d8e066e38469484004d8f0cac0d8035/PysparkGateway-0.0.17.tar.gz"
            }
        ],
        "0.0.18": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "477b3820013419f41ce2277824cd755a",
                    "sha256": "7d70e43c75454aa5cd8a8167421f4efdd79a3662e51314a8cf8257f0ff8b19e5"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.18.tar.gz",
                "has_sig": false,
                "md5_digest": "477b3820013419f41ce2277824cd755a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9296,
                "upload_time": "2019-07-04T21:34:29",
                "url": "https://files.pythonhosted.org/packages/4f/17/dfd59842cf974c781412c5a35d0bdfd031a815896f7da7d9cb989099de97/PysparkGateway-0.0.18.tar.gz"
            }
        ],
        "0.0.19": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0ae079e87e6d65d07db877b55913a66c",
                    "sha256": "0c8a53686745c27d0c6a1658e8ce610c15d3e72a1dc656a4801a37f8b10801bf"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.19.tar.gz",
                "has_sig": false,
                "md5_digest": "0ae079e87e6d65d07db877b55913a66c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6890,
                "upload_time": "2019-07-15T11:38:23",
                "url": "https://files.pythonhosted.org/packages/56/1d/3988365e4ab6b712fe7944559c63d071759d51b243bd865c85f0bd52e319/PysparkGateway-0.0.19.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5354d83e18da8efc10d3d52979860102",
                    "sha256": "07bd280c876518961ee47c64824a49960d9339a09192179c1d8f7a2fc4b5d796"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "5354d83e18da8efc10d3d52979860102",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6681,
                "upload_time": "2018-12-22T00:20:40",
                "url": "https://files.pythonhosted.org/packages/d1/2b/c96a36fdec93783e831d62ce3682e2bf408ebb8707d60a53740a773428f9/PysparkGateway-0.0.2.tar.gz"
            }
        ],
        "0.0.20": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d03ed7fa9603947702b181adb59da076",
                    "sha256": "56441c1d9a4bae1bbfc6bb3896cec6e6e73089c13a5767b01a00e4ab6ab08d51"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.20.tar.gz",
                "has_sig": false,
                "md5_digest": "d03ed7fa9603947702b181adb59da076",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9286,
                "upload_time": "2019-07-16T05:29:01",
                "url": "https://files.pythonhosted.org/packages/1f/ce/1685b6ec0ac97729f3267962d6bb12b394e19e886a75893bd1758ee23bd7/PysparkGateway-0.0.20.tar.gz"
            }
        ],
        "0.0.21": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "943010146fd7863fabf5edee236f285b",
                    "sha256": "75a338f46eed7130b1a0f409c5de8e239397594f5a859b4441b4d59be9f2790f"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.21.tar.gz",
                "has_sig": false,
                "md5_digest": "943010146fd7863fabf5edee236f285b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9339,
                "upload_time": "2019-10-09T16:54:51",
                "url": "https://files.pythonhosted.org/packages/75/b4/7c68e12341672b14cf82379ed9273df72aaa1c0ac7d146be0a682d77b4db/PysparkGateway-0.0.21.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "58a328237359cbd76e8837eabab66833",
                    "sha256": "5f50166f1f3bd8921dcec7511dca8565e9b7e7d27cc65ed664397ed5bb179546"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "58a328237359cbd76e8837eabab66833",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7194,
                "upload_time": "2019-04-19T04:44:43",
                "url": "https://files.pythonhosted.org/packages/be/80/77c8ffe4de97f90887c267d49fc808fb591ea0140c08f19afbf1a0245f8d/PysparkGateway-0.0.3.tar.gz"
            }
        ],
        "0.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7eab9565047ea5348df040aeb25225a7",
                    "sha256": "41a656254f33e1ef9846e797a162e241de484926495d4038a7bc6a6eab5f38a2"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "7eab9565047ea5348df040aeb25225a7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7276,
                "upload_time": "2019-04-19T04:50:56",
                "url": "https://files.pythonhosted.org/packages/6c/0e/76650e30a49860ba31ae6a85fc2eb6b212b947b35242dcdb98ee6fdecf36/PysparkGateway-0.0.4.tar.gz"
            }
        ],
        "0.0.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "525bedecc50988af7bedfd358195bf4c",
                    "sha256": "7ab115c838fea56e637774801b40b2c8e6b2bfdec16a5a03e099987efffae87b"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.5.tar.gz",
                "has_sig": false,
                "md5_digest": "525bedecc50988af7bedfd358195bf4c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7908,
                "upload_time": "2019-04-19T05:01:34",
                "url": "https://files.pythonhosted.org/packages/a9/da/c95c6d5afd0437a174a2c40f21620a58e30c496a86faafc3c1107409ebab/PysparkGateway-0.0.5.tar.gz"
            }
        ],
        "0.0.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6e90a6b3dc236a8ca261d118f9698950",
                    "sha256": "3d9f151ae13c6905208901640597b66caf18a033ecdfb9f5e21cb41537eb0279"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.6.tar.gz",
                "has_sig": false,
                "md5_digest": "6e90a6b3dc236a8ca261d118f9698950",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7919,
                "upload_time": "2019-04-19T19:15:55",
                "url": "https://files.pythonhosted.org/packages/a7/f0/5e4fcb67bb994e45915c234c5698a6a95377551f2d5f49d07c2b978fab83/PysparkGateway-0.0.6.tar.gz"
            }
        ],
        "0.0.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ac57e86da814c68b6efe665ada095c44",
                    "sha256": "f6f560cfcefc5cfd5b5f38fa22fc175f05568f2aa4d8eae9c6768700bfb579db"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.7.tar.gz",
                "has_sig": false,
                "md5_digest": "ac57e86da814c68b6efe665ada095c44",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8352,
                "upload_time": "2019-06-04T23:32:57",
                "url": "https://files.pythonhosted.org/packages/ff/0c/d9dbb81111c7a4e7b1d67d7537df5a8e77bb039f75bfc0e644303107d80a/PysparkGateway-0.0.7.tar.gz"
            }
        ],
        "0.0.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5e49c686957a5cf3f96348f849d09d27",
                    "sha256": "8b3c0d41b70f22e7a9f3706466b7b5ad719d00a2e4b9cb718b9c0683c3d049a6"
                },
                "downloads": -1,
                "filename": "PysparkGateway-0.0.8.tar.gz",
                "has_sig": false,
                "md5_digest": "5e49c686957a5cf3f96348f849d09d27",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9162,
                "upload_time": "2019-06-05T17:57:19",
                "url": "https://files.pythonhosted.org/packages/5e/ca/6f73f1a9171d85e7cddd4d187edcfeb934ea5c7d5d45c379fa05b5c8a073/PysparkGateway-0.0.8.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "943010146fd7863fabf5edee236f285b",
                "sha256": "75a338f46eed7130b1a0f409c5de8e239397594f5a859b4441b4d59be9f2790f"
            },
            "downloads": -1,
            "filename": "PysparkGateway-0.0.21.tar.gz",
            "has_sig": false,
            "md5_digest": "943010146fd7863fabf5edee236f285b",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 9339,
            "upload_time": "2019-10-09T16:54:51",
            "url": "https://files.pythonhosted.org/packages/75/b4/7c68e12341672b14cf82379ed9273df72aaa1c0ac7d146be0a682d77b4db/PysparkGateway-0.0.21.tar.gz"
        }
    ]
}