{
    "info": {
        "author": "Tubular Engineering",
        "author_email": "dev@tubularlabs.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: Apache Software License",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.5",
            "Topic :: Software Development :: Build Tools"
        ],
        "description": "Sparkly\n=======\n\n|Sparkly PyPi Version| |Sparkly Build Status| |Documentation Status|\n\nHelpers & syntax sugar for PySpark. There are several features to make your life easier:\n\n- Definition of spark packages, external jars, UDFs and spark options within your code;\n- Simplified reader/writer api for Cassandra, Elastic, MySQL, Kafka;\n- Testing framework for spark applications.\n\nMore details could be found in `the official\ndocumentation <https://sparkly.readthedocs.org>`__.\n\nInstallation\n------------\n\nSparkly itself is easy to install::\n\n    pip install sparkly\n\nThe tricky part is ``pyspark``. There is no official distribution on\nPyPI. As a workaround we can suggest:\n\n1) Use env variable ``PYTHONPATH`` to point to your Spark installation,\n   something like::\n\n       export PYTHONPATH=\"/usr/local/spark/python/lib/pyspark.zip:/usr/local/spark/python/lib/py4j-0.10.4-src.zip\"\n\n2) Use our ``setup.py`` file for ``pyspark``. Just add this to your\n   ``requirements.txt``::\n\n       -e git+https://github.com/Tubular/spark@branch-2.1.0#egg=pyspark&subdirectory=python\n\nHere in Tubular, we published ``pyspark`` to our internal PyPi\nrepository.\n\nGetting Started\n---------------\n\nHere is a small code snippet to show how to easily read Cassandra table\nand write its content to ElasticSearch index::\n\n    from sparkly import SparklySession\n\n\n    class MySession(SparklySession):\n        packages = [\n            'datastax:spark-cassandra-connector:2.0.0-M2-s_2.11',\n            'org.elasticsearch:elasticsearch-spark-20_2.11:6.5.4',\n        ]\n        \n\n    if __name__ == '__main__':\n        spark = MySession()\n        df = spark.read_ext.cassandra('localhost', 'my_keyspace', 'my_table')\n        df.write_ext.elastic('localhost', 'my_index', 'my_type')\n\nSee `the online documentation <https://sparkly.readthedocs.org>`__ for\nmore details.\n\nTesting\n-------\n\nTo run tests you have to have `docker <https://www.docker.com/>`__ and\n`docker-compose <https://docs.docker.com/compose/>`__ installed on your\nsystem. If you are working on MacOS we highly recommend you to use\n`docker-machine <https://docs.docker.com/machine/>`__. As soon as the\ntools mentioned above have been installed, all you need is to run::\n\n    make test\n\nSupported Spark Versions\n------------------------\n\nAt the moment we support:\n\n+---------------------------------------------------------------------------+\n| sparkly >= 2.7 | Spark 2.4.x                                              |\n+---------------------------------------------------------------------------+\n| sparkly 2.x    | Spark 2.0.x and Spark 2.1.x and Spark 2.2.x              |\n+---------------------------------------------------------------------------+\n| sparkly 1.x    | Spark 1.6.x                                              |\n+---------------------------------------------------------------------------+\n\n.. |Sparkly PyPi Version| image:: http://img.shields.io/pypi/v/sparkly.svg\n   :target: https://pypi.python.org/pypi/sparkly\n.. |Sparkly Build Status| image:: https://travis-ci.org/tubular/sparkly.svg?branch=master\n   :target: https://travis-ci.org/tubular/sparkly\n.. |Documentation Status| image:: https://readthedocs.org/projects/sparkly/badge/?version=latest\n   :target: http://sparkly.readthedocs.io/en/latest/?badge=latest\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/Tubular/sparkly",
        "keywords": "sparkly spark pyspark",
        "license": "Apache License 2.0",
        "maintainer": "",
        "maintainer_email": "",
        "name": "sparkly",
        "package_url": "https://pypi.org/project/sparkly/",
        "platform": "",
        "project_url": "https://pypi.org/project/sparkly/",
        "project_urls": {
            "Homepage": "https://github.com/Tubular/sparkly"
        },
        "release_url": "https://pypi.org/project/sparkly/2.8.1/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Helpers & syntax sugar for PySpark.",
        "version": "2.8.1"
    },
    "last_serial": 5813625,
    "releases": {
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2968f941c1c916d574c62ae919008a56",
                    "sha256": "e8f20e388c36ea282004c73227426ef1f92a90d3dbe03ce995ead16a0cad32a3"
                },
                "downloads": -1,
                "filename": "sparkly-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "2968f941c1c916d574c62ae919008a56",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 15728,
                "upload_time": "2016-12-02T18:46:51",
                "url": "https://files.pythonhosted.org/packages/7e/52/88952e1b5064d04f5cb22ee3ff5cbbb080dc140fd626d334003e1657f679/sparkly-1.0.0.tar.gz"
            }
        ],
        "1.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d96569217869a28dda28eeda9db74f58",
                    "sha256": "08b698e302d4b0ef0578df475fe4fb78058a0ceabbaad5a07c0ac7ee61b5efe4"
                },
                "downloads": -1,
                "filename": "sparkly-1.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "d96569217869a28dda28eeda9db74f58",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 19233,
                "upload_time": "2017-01-19T12:27:53",
                "url": "https://files.pythonhosted.org/packages/d1/39/f23deca363d40923693e7c066ac607afe5830ef36da061667c7acebfb2ed/sparkly-1.1.0.tar.gz"
            }
        ],
        "1.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "18c2bc2e1607efa96939414c0cb0475c",
                    "sha256": "c4f14aee7478f0e4e4ff3ee232ea464ef33a03e2d4798d78d6f333c9f95576c5"
                },
                "downloads": -1,
                "filename": "sparkly-1.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "18c2bc2e1607efa96939414c0cb0475c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 19514,
                "upload_time": "2017-01-23T11:15:48",
                "url": "https://files.pythonhosted.org/packages/ab/39/b93968b3d4c7d7b6bcf8bb6de35d32773c278bc90b435b37391411fe62c9/sparkly-1.1.1.tar.gz"
            }
        ],
        "2.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "730f8be43d12c4ee6a63787ebc24b113",
                    "sha256": "2b5112734f4be1a9e49e8d141a0cf8bf11ebc55635649c32cc99720d80e942c1"
                },
                "downloads": -1,
                "filename": "sparkly-2.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "730f8be43d12c4ee6a63787ebc24b113",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17531,
                "upload_time": "2017-02-13T15:57:39",
                "url": "https://files.pythonhosted.org/packages/d1/16/12d0aabcd55bb57e829dbf8ea25a6c24cf9bd20b20a7220f89bfb152dc2b/sparkly-2.0.1.tar.gz"
            }
        ],
        "2.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2489796793928a42f35d2dd230205b11",
                    "sha256": "e1d0bd4f7a866af30de759fe65d9f2863f1999f6d477302bd5ff1c13aea0e4bd"
                },
                "downloads": -1,
                "filename": "sparkly-2.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "2489796793928a42f35d2dd230205b11",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17647,
                "upload_time": "2017-03-23T14:59:05",
                "url": "https://files.pythonhosted.org/packages/52/2a/cb1adf225859a6a27884655368e8e8243c7cfd7130528164e565c2efb5d9/sparkly-2.0.2.tar.gz"
            }
        ],
        "2.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7beeb519d8b771c1620999a73725c55e",
                    "sha256": "4ab91c5b50063f5f26d8e6e579a3c7556c3a14d08178b3154967c26d9105298d"
                },
                "downloads": -1,
                "filename": "sparkly-2.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "7beeb519d8b771c1620999a73725c55e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 18411,
                "upload_time": "2017-06-02T17:32:32",
                "url": "https://files.pythonhosted.org/packages/a3/3f/3256370d310b39d70f1670b3b8cea51fcdf53d88d7e43bde1754dce4cdfa/sparkly-2.0.4.tar.gz"
            }
        ],
        "2.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ed4ebb4ada8fe52613b7ad93d77bd022",
                    "sha256": "b5bcd5ac780c56c2320ff14f65b8f6c94a123d7001e167f4d43b84495d8912be"
                },
                "downloads": -1,
                "filename": "sparkly-2.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ed4ebb4ada8fe52613b7ad93d77bd022",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 18536,
                "upload_time": "2017-07-10T10:54:16",
                "url": "https://files.pythonhosted.org/packages/4e/1e/85ae2586a3deacd5c0b634375fb606eb1135b5a50613464905892ea638b3/sparkly-2.1.0.tar.gz"
            }
        ],
        "2.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "723ec7c52ba21000f33fdba4008f54bc",
                    "sha256": "5c8900174d697c03f70fc95e90afd7262670e4019737b80ac8596d3dfef3afe0"
                },
                "downloads": -1,
                "filename": "sparkly-2.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "723ec7c52ba21000f33fdba4008f54bc",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 18709,
                "upload_time": "2017-07-27T11:51:00",
                "url": "https://files.pythonhosted.org/packages/2d/8d/cb2483b2ed5b93c2d3342b4eaefd4fe3e26835e5352250d433f38ace2686/sparkly-2.1.1.tar.gz"
            }
        ],
        "2.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3faa9270ac10e7007bcb7c2e51c1495a",
                    "sha256": "a6f8074fe253d07a881426126e713658352e826f01af20e73d3cd1f772144013"
                },
                "downloads": -1,
                "filename": "sparkly-2.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "3faa9270ac10e7007bcb7c2e51c1495a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20838,
                "upload_time": "2017-08-03T09:40:55",
                "url": "https://files.pythonhosted.org/packages/c6/34/9510cc2ccc93465d96b60f795fbddf8ab03fc12d094f4c8593e688d1a82e/sparkly-2.2.0.tar.gz"
            }
        ],
        "2.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "61db8c7eff6782e4c88b2dc8c529547e",
                    "sha256": "d72012ddafcb7ab4b74beb676b747aac013f66142e93bd61406f217478070afe"
                },
                "downloads": -1,
                "filename": "sparkly-2.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "61db8c7eff6782e4c88b2dc8c529547e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20854,
                "upload_time": "2017-08-03T10:24:07",
                "url": "https://files.pythonhosted.org/packages/ce/af/7d6521b4dc9c77da9906536b5582fb3348ba85ed6b4dcbb50f690645fccb/sparkly-2.2.1.tar.gz"
            }
        ],
        "2.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a1c6d9a68bb1756bd4697936183fcf12",
                    "sha256": "5c4c932f692ae6c5988dfc0e2d0be0c395ae58ae47ddaa120ab792da93f07b04"
                },
                "downloads": -1,
                "filename": "sparkly-2.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "a1c6d9a68bb1756bd4697936183fcf12",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 30853,
                "upload_time": "2017-08-25T09:35:45",
                "url": "https://files.pythonhosted.org/packages/29/22/3c63e15053acd923dea7d0f61b1413a3d8b6e83df0a7c1ce0b459b8422bc/sparkly-2.3.0.tar.gz"
            }
        ],
        "2.4.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "08f00468fedf22cc45589f9845d669bd",
                    "sha256": "47a1c6dc15e1b32056e9304949768b88bfdc93a4f3f94332fe3f9b5623d82c00"
                },
                "downloads": -1,
                "filename": "sparkly-2.4.0.tar.gz",
                "has_sig": false,
                "md5_digest": "08f00468fedf22cc45589f9845d669bd",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 28507,
                "upload_time": "2018-07-04T09:53:46",
                "url": "https://files.pythonhosted.org/packages/f1/7c/f1e53ecf2f6f3ba29d3c0a17fd7daeabe85bb1a1ba7d7f929c4ce72aa163/sparkly-2.4.0.tar.gz"
            }
        ],
        "2.4.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ef6540692aef81d90ae2574994dff8b9",
                    "sha256": "8663f563a9ecbe25d0ef305625579aa082140da01e399d029a4851a08a3e6a1c"
                },
                "downloads": -1,
                "filename": "sparkly-2.4.1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "ef6540692aef81d90ae2574994dff8b9",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 38646,
                "upload_time": "2018-11-09T13:58:00",
                "url": "https://files.pythonhosted.org/packages/ed/32/0b3aeea86ae9d11c3e47bedda144f5dfdbf2435e3fd7db670180d767280d/sparkly-2.4.1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "e6f7b072bd3ca5eb5e1216e1b3f608d7",
                    "sha256": "db55368753ed97501528adde2722bcb0aaa228c6885ceac13aa890271f88e533"
                },
                "downloads": -1,
                "filename": "sparkly-2.4.1.tar.gz",
                "has_sig": false,
                "md5_digest": "e6f7b072bd3ca5eb5e1216e1b3f608d7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 28541,
                "upload_time": "2018-11-09T13:58:02",
                "url": "https://files.pythonhosted.org/packages/10/e9/ca445d80e930edf1658b71cb0f375cab17dd8d72dd8ee19e8e6c9175d91b/sparkly-2.4.1.tar.gz"
            }
        ],
        "2.5.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fa2b1a93dd4c95dff45f1a8071f21540",
                    "sha256": "6f19ece2b2847240e8d2a2c8399e0f43f462c5ddc3ccf2bd71381d7986f6dd44"
                },
                "downloads": -1,
                "filename": "sparkly-2.5.0.tar.gz",
                "has_sig": false,
                "md5_digest": "fa2b1a93dd4c95dff45f1a8071f21540",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 28594,
                "upload_time": "2019-06-26T09:17:16",
                "url": "https://files.pythonhosted.org/packages/80/da/5a80eafab5be27cb1ca477e496d995828376d352f9e9782d30b4cbf5a81d/sparkly-2.5.0.tar.gz"
            }
        ],
        "2.5.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f7ff66d5afc30ddbc62b84eaf51e02b4",
                    "sha256": "4068478921e72b8a2439ef409a219ba681be4c509dd2e0255c410efe8830446e"
                },
                "downloads": -1,
                "filename": "sparkly-2.5.1.tar.gz",
                "has_sig": false,
                "md5_digest": "f7ff66d5afc30ddbc62b84eaf51e02b4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 28605,
                "upload_time": "2019-06-26T09:22:47",
                "url": "https://files.pythonhosted.org/packages/14/15/e135f5e9d00a42c2d29dcdc84959d6dffaac41d860cfc2189ecbb48782fc/sparkly-2.5.1.tar.gz"
            }
        ],
        "2.6.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a38754356fee2261a110e46bc3f34044",
                    "sha256": "f62dafd397f2fcb57d7e798613e30f0637daef712a9568403baa4e715cb93514"
                },
                "downloads": -1,
                "filename": "sparkly-2.6.0.tar.gz",
                "has_sig": false,
                "md5_digest": "a38754356fee2261a110e46bc3f34044",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29013,
                "upload_time": "2019-06-26T09:25:35",
                "url": "https://files.pythonhosted.org/packages/19/48/63a8bf1f8b61f0635e51f6347f0531eb607f7c5e39e1fc87bc84e39cffc9/sparkly-2.6.0.tar.gz"
            }
        ],
        "2.7.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e20a0e8a8d6e08630b8cd40c9b788875",
                    "sha256": "17b8b745f86de58173fb58fec8a646a81a84e48713b4417fa2066c57bb2381f7"
                },
                "downloads": -1,
                "filename": "sparkly-2.7.0.tar.gz",
                "has_sig": false,
                "md5_digest": "e20a0e8a8d6e08630b8cd40c9b788875",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29136,
                "upload_time": "2019-06-26T09:42:20",
                "url": "https://files.pythonhosted.org/packages/6f/6e/0a4c2a9d20c3715cf58191ab0c07d8b3c48a41034040464bab84da1bc7e7/sparkly-2.7.0.tar.gz"
            }
        ],
        "2.7.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1cfed56e785ce4a51eff95a517fc6bb3",
                    "sha256": "45dcd363685cf0a8bdc9590d9462ebb2437a3a1b8d89e26dc48b31caef522629"
                },
                "downloads": -1,
                "filename": "sparkly-2.7.1.tar.gz",
                "has_sig": false,
                "md5_digest": "1cfed56e785ce4a51eff95a517fc6bb3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29149,
                "upload_time": "2019-06-26T09:57:38",
                "url": "https://files.pythonhosted.org/packages/22/88/50b86d1609cfe6e920ea431979c75723f36f2ebf24bcd04b2b95506649c0/sparkly-2.7.1.tar.gz"
            }
        ],
        "2.8.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9998ea45f3fb329ab404043db8a016eb",
                    "sha256": "448be76b511acbe997962fd0a97b257ce93292f5b4c28409c6c6994ad6cf2c59"
                },
                "downloads": -1,
                "filename": "sparkly-2.8.0.tar.gz",
                "has_sig": false,
                "md5_digest": "9998ea45f3fb329ab404043db8a016eb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29966,
                "upload_time": "2019-09-03T08:08:42",
                "url": "https://files.pythonhosted.org/packages/b1/04/2e3e504aa1e03016e164ea51df5373ed76b420e011a64ca55b250e38b591/sparkly-2.8.0.tar.gz"
            }
        ],
        "2.8.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f3d9ed34f02bb67cbef2545a72c94a2d",
                    "sha256": "7eb340e31f0773bd0257a3516618ce40767e63563d3b7b183619c107b36b0b7e"
                },
                "downloads": -1,
                "filename": "sparkly-2.8.1.tar.gz",
                "has_sig": false,
                "md5_digest": "f3d9ed34f02bb67cbef2545a72c94a2d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 30671,
                "upload_time": "2019-09-11T08:24:32",
                "url": "https://files.pythonhosted.org/packages/1e/59/10311b3ae3e4d97fd836ecb444bf77e068f43b6abd2efdaeef39a4029a49/sparkly-2.8.1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "f3d9ed34f02bb67cbef2545a72c94a2d",
                "sha256": "7eb340e31f0773bd0257a3516618ce40767e63563d3b7b183619c107b36b0b7e"
            },
            "downloads": -1,
            "filename": "sparkly-2.8.1.tar.gz",
            "has_sig": false,
            "md5_digest": "f3d9ed34f02bb67cbef2545a72c94a2d",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 30671,
            "upload_time": "2019-09-11T08:24:32",
            "url": "https://files.pythonhosted.org/packages/1e/59/10311b3ae3e4d97fd836ecb444bf77e068f43b6abd2efdaeef39a4029a49/sparkly-2.8.1.tar.gz"
        }
    ]
}