{
    "info": {
        "author": "Data Bootcamp",
        "author_email": "contato@databootcamp.com.br",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "NBThread Spark\n##############\n\nSpark multithread in IPython Notebooks.\n\nIt's now simple to execute Spark Structured Streaming in Jupyter Notebooks\n\nInstall\n=======\n\n.. code:: shell\n\n    pip install nbthread_spark --process-dependency-links\n\nUsage\n=====\n\nShow Spark Buttons for stop and UI:\n\n.. code:: python\n\n    from nbthread_spark.spark import SparkRunner\n\n    spark = SparkRunner.builder.getOrCreate() # same as original SparkSession\n\n    ## you will see buttons ;)\n\n\n\nGiven a Socket Stream:\n\n.. code:: python\n\n    TCP_IP = \"localhost\"\n    TCP_PORT = 9005\n\n    from pyspark.sql.functions import from_json\n    from pyspark.sql import SparkSession\n    from pyspark.sql.types import StructField, StructType, IntegerType\n\n    schema = StructType([\n        StructField(\"bip\", IntegerType(), True),\n        StructField(\"is_on\", IntegerType(), True)\n    ])\n\n    spark = SparkSession \\\n        .builder \\\n        .appName(\"IOTStreamApp\") \\\n        .getOrCreate()\n\n    iot_stream = spark \\\n        .readStream \\\n        .format(\"socket\") \\\n        .option(\"host\", TCP_IP) \\\n        .option(\"port\", TCP_PORT) \\\n        .load()\n\n    iot_expanded = iot_stream.withColumn('value_json', \n                                        from_json('value', schema)\n                                        ).drop('value').select('value_json.*')\n\n    query = iot_expanded \\\n        .writeStream \\\n        .outputMode(\"update\") \\\n        .format(\"memory\") \\\n        .queryName(\"iot_table\") \\\n        .start()\n\nYou can run queries using this:\n\n.. code:: python\n\n    from nbthread_spark.stream import StreamRunner\n\n    runner = StreamRunner(query)\n    \n    runner.controls()\n    ## you will see buttons ;)\n\n    runner.start() # start without controls\n\n    runner.status() # show stream status\n\n    runner.stop() # stop streaming and thread\n\nFor Stream Manager you can control lot of streams in a easy way:\n\n.. code:: python\n\n    from nbthread_spark.manager import StreamManager\n\n    sm = StreamManager()\n\n    sm.append(runner)\n    sm.append(runner1)\n    sm.append(runner2)\n\n    sm.all_controls()\n    ## you will see all buttons from streams ;)\n\n    sm.start_all() # start all streams\n\n    sm.stop_all() # stop all streams\n\nSpecial Thanks\n==============\n\nHere_ the list of students that contribute with this module.\n\n.. _Here: https://github.com/databootcampbr/nbthread-spark/blob/master/CONTRIBUTORS.md    ",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "",
        "keywords": "",
        "license": "Apache",
        "maintainer": "",
        "maintainer_email": "",
        "name": "nbthread_spark",
        "package_url": "https://pypi.org/project/nbthread_spark/",
        "platform": "",
        "project_url": "https://pypi.org/project/nbthread_spark/",
        "project_urls": null,
        "release_url": "https://pypi.org/project/nbthread_spark/0.0.6/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Spark Structured Streaming multithread in IPython Notebooks",
        "version": "0.0.6"
    },
    "last_serial": 3443879,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e3acf765cf6771f53a39763bf58c4413",
                    "sha256": "9c9940ed9391d675c39ba7cfe884c7a3adc3b5688a3efe3b783c2f842fd15fd3"
                },
                "downloads": -1,
                "filename": "nbthread_spark-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "e3acf765cf6771f53a39763bf58c4413",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 1846,
                "upload_time": "2017-12-21T22:41:53",
                "url": "https://files.pythonhosted.org/packages/3b/0d/c1e9ce46622a2b48df637646e9e7b9471a7ecdd6e9c74baf084dabd47edb/nbthread_spark-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "60851d185d6e0c9b70e4eb18ec1f88f6",
                    "sha256": "851bb3d545dc1b18580531fc451ec9a4bda2f86b1f58b72f173b65b08db25cb5"
                },
                "downloads": -1,
                "filename": "nbthread_spark-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "60851d185d6e0c9b70e4eb18ec1f88f6",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 1892,
                "upload_time": "2017-12-22T17:15:32",
                "url": "https://files.pythonhosted.org/packages/1a/15/961457961c4260700c9af36aaa2e90b44384b8cc96d6fbc9f82497f21d1e/nbthread_spark-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6c1b4abb7a617c2685544ab1dde1d1bb",
                    "sha256": "6e864c1d42655c230689a9736809d7e14c15effd6b987ee3c077b2c57d9fd2ca"
                },
                "downloads": -1,
                "filename": "nbthread_spark-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "6c1b4abb7a617c2685544ab1dde1d1bb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2394,
                "upload_time": "2017-12-24T22:14:31",
                "url": "https://files.pythonhosted.org/packages/d2/3b/38017a9e969d05e7354994c5d89bd1167c0f498bf77910a2d0bbdc38f5d8/nbthread_spark-0.0.3.tar.gz"
            }
        ],
        "0.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e79327b2efb72ea8d9ba422a72bf8928",
                    "sha256": "6f3f44a5fcf265289f1ad397e6bea77b9f4afb5c96e544d31fe3282bf2e073f9"
                },
                "downloads": -1,
                "filename": "nbthread_spark-0.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "e79327b2efb72ea8d9ba422a72bf8928",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2393,
                "upload_time": "2017-12-24T22:48:49",
                "url": "https://files.pythonhosted.org/packages/a9/c3/161ac18a2de14e53d5daa4c4284c9d725f97c966bc43239a06d28f698882/nbthread_spark-0.0.4.tar.gz"
            }
        ],
        "0.0.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0bf2e2a2b9ace4364d94effe108c73c9",
                    "sha256": "8da4c4d5f7877f9513f4df4f4be386fb1c301cbefe7ac8a87bf4485277263407"
                },
                "downloads": -1,
                "filename": "nbthread_spark-0.0.5.tar.gz",
                "has_sig": false,
                "md5_digest": "0bf2e2a2b9ace4364d94effe108c73c9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2682,
                "upload_time": "2017-12-26T18:39:43",
                "url": "https://files.pythonhosted.org/packages/93/14/bd9ac9392c682eb45834ef8dc4952f4f1a2afe9ee375451e10dcf1c6191c/nbthread_spark-0.0.5.tar.gz"
            }
        ],
        "0.0.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d459619eadea51877c05ab2efd1b9fda",
                    "sha256": "ce72ffb1492317162ebbaaeecc0a2140f111b108a33d9c000109e57a252d4e16"
                },
                "downloads": -1,
                "filename": "nbthread_spark-0.0.6.tar.gz",
                "has_sig": false,
                "md5_digest": "d459619eadea51877c05ab2efd1b9fda",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2990,
                "upload_time": "2017-12-26T19:18:27",
                "url": "https://files.pythonhosted.org/packages/f9/7f/14eb10642c82974337987ae0fbb93d1e56d6b160b83605102a1208c6a23a/nbthread_spark-0.0.6.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "d459619eadea51877c05ab2efd1b9fda",
                "sha256": "ce72ffb1492317162ebbaaeecc0a2140f111b108a33d9c000109e57a252d4e16"
            },
            "downloads": -1,
            "filename": "nbthread_spark-0.0.6.tar.gz",
            "has_sig": false,
            "md5_digest": "d459619eadea51877c05ab2efd1b9fda",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 2990,
            "upload_time": "2017-12-26T19:18:27",
            "url": "https://files.pythonhosted.org/packages/f9/7f/14eb10642c82974337987ae0fbb93d1e56d6b160b83605102a1208c6a23a/nbthread_spark-0.0.6.tar.gz"
        }
    ]
}