{
    "info": {
        "author": "James Pickering",
        "author_email": "james_pic@hotmail.com",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "pyspark-flame\n=============\n\nA low-overhead profiler for Spark on Python\n\nPyspark-flame hooks into Pyspark's existing profiling capabilities to\nprovide a low-overhead stack-sampling profiler, that outputs performance\ndata in a format compatible with `Brendan Gregg's FlameGraph\nVisualizer <https://github.com/brendangregg/FlameGraph>`__.\n\nBecause pyspark-flame hooks into Pyspark's profiling capabilities, it\ncan profile the entire execution of an RDD, across the whole of the\ncluster, and provides RDD-level visibility of performance.\n\nUnlike the cProfile-based profiler included with Pyspark, pyspark-flame\nuses stack sampling. It takes stack traces at regular (configurable)\nintervals, which allows its overhead to be low and tunable, and doesn't\nskew results, making it suitable for use in performance test\nenvironments at high volumes.\n\nInstallation\n------------\n\n.. code:: bash\n\n    pip install pyspark-flame\n\nUsage\n-----\n\n.. code:: python\n\n    from pyspark_flame import FlameProfiler\n    from pyspark import SparkConf, SparkContext\n\n    conf = SparkConf().set(\"spark.python.profile\", \"true\")\n    conf = conf.set(\"spark.python.profile.dump\", \".\")  # Optional - if not, dumps to stdout at exit\n    sc = SparkContext(\n        'local', 'test', conf=conf, profiler_cls=FlameProfiler,\n        environment={'pyspark_flame.interval': 0.25}  # Optional - default is 0.2 seconds\n    )\n    # Do stuff with Spark context...\n    sc.show_profiles()\n    # Or maybe\n    sc.dump_profiles('.')\n\nFor convenience, flamegraph.pl is vendored in, so you can produce a\nflame graph with:\n\n.. code:: bash\n\n    flamegraph.pl rdd-1.flame > rdd-1.svg\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "https://github.com/jamespic/pyspark-flame/archive/0.2.4.tar.gz",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/jamespic/pyspark-flame",
        "keywords": "",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "pyspark-flame",
        "package_url": "https://pypi.org/project/pyspark-flame/",
        "platform": "",
        "project_url": "https://pypi.org/project/pyspark-flame/",
        "project_urls": {
            "Download": "https://github.com/jamespic/pyspark-flame/archive/0.2.4.tar.gz",
            "Homepage": "https://github.com/jamespic/pyspark-flame"
        },
        "release_url": "https://pypi.org/project/pyspark-flame/0.2.6/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "A low-overhead sampling profiler for PySpark, that outputs Flame Graphs",
        "version": "0.2.6"
    },
    "last_serial": 4305076,
    "releases": {
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "82721ff0da337514095330db0c02b6cd",
                    "sha256": "84c0a90f8763cd1cd3f043ce7b08812fc7e7a68812bd656d9c413c8071affdbc"
                },
                "downloads": -1,
                "filename": "pyspark-flame-0.1.2.tar.gz",
                "has_sig": false,
                "md5_digest": "82721ff0da337514095330db0c02b6cd",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3361,
                "upload_time": "2017-04-24T15:50:14",
                "url": "https://files.pythonhosted.org/packages/f7/bf/2678a2b115a0e6eab7d9e67e823e3ad592c6515c698273ee31c500e7a992/pyspark-flame-0.1.2.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "582cf497128654375cc65dffc6b4dd7d",
                    "sha256": "a3ce2e384b254338cef2aca40113f8398f7b09b7da638a3db866d41c6f8a7b72"
                },
                "downloads": -1,
                "filename": "pyspark-flame-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "582cf497128654375cc65dffc6b4dd7d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 15189,
                "upload_time": "2017-07-10T13:36:53",
                "url": "https://files.pythonhosted.org/packages/07/39/dc8258b66c6e08330131727c53e1e9257834e18f88a86083a1e470ebcd7d/pyspark-flame-0.2.0.tar.gz"
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3578cbc04d50df0f68683e3368260d1a",
                    "sha256": "a9aae260e27f18eac7f56ab05f1ee2d8bf1a9eae4ba34b4f88c68f18099da1d6"
                },
                "downloads": -1,
                "filename": "pyspark-flame-0.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "3578cbc04d50df0f68683e3368260d1a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 15569,
                "upload_time": "2017-07-10T14:02:50",
                "url": "https://files.pythonhosted.org/packages/f7/a3/ba4f91005feecdb81c9569152e07d8e057d0b7d7892ec304a853b709159d/pyspark-flame-0.2.2.tar.gz"
            }
        ],
        "0.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3baa0d90093ed6ccf8e83bd69d04a27b",
                    "sha256": "c36d713f09d2f558e57b083cffbda7a74f99949315af1c727bd81585fe05213f"
                },
                "downloads": -1,
                "filename": "pyspark-flame-0.2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "3baa0d90093ed6ccf8e83bd69d04a27b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 15653,
                "upload_time": "2017-08-16T10:47:13",
                "url": "https://files.pythonhosted.org/packages/21/9c/fa5308b80ec8e2f39b6a5c980c4655f4441b4ad1292f2bff3f63d5f14f46/pyspark-flame-0.2.3.tar.gz"
            }
        ],
        "0.2.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a029c1d33d8f1fb7d63e7a5c46d7bbb1",
                    "sha256": "385d3af388dc66b44e4265058057e9ac42ebf2828486155d15eccbbf70808548"
                },
                "downloads": -1,
                "filename": "pyspark-flame-0.2.4.tar.gz",
                "has_sig": false,
                "md5_digest": "a029c1d33d8f1fb7d63e7a5c46d7bbb1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 15651,
                "upload_time": "2018-01-24T11:29:33",
                "url": "https://files.pythonhosted.org/packages/d8/0b/be253555441e99bf326bef1b9e9dcd25c13613f4524de477536ae2a88468/pyspark-flame-0.2.4.tar.gz"
            }
        ],
        "0.2.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "567e60a3e40b2a27c78e29bbc363e736",
                    "sha256": "a41c4ab29c9e3c5fdf7c1339d222705dbf7379cbb41c9cdfe9f6b723b78507b6"
                },
                "downloads": -1,
                "filename": "pyspark-flame-0.2.5.tar.gz",
                "has_sig": false,
                "md5_digest": "567e60a3e40b2a27c78e29bbc363e736",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 15632,
                "upload_time": "2018-09-24T15:11:46",
                "url": "https://files.pythonhosted.org/packages/3a/1a/64706e90776c1de972a470cb8df2bb4df6cd549d6ef813d639a3a1c4181d/pyspark-flame-0.2.5.tar.gz"
            }
        ],
        "0.2.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ce4e6fa4eaa35aa9176763c8f9f91dc7",
                    "sha256": "807099c77fbaefaeb71217995bd22d203c697c6ce32c8e75ff53e1b1b56d9322"
                },
                "downloads": -1,
                "filename": "pyspark-flame-0.2.6.tar.gz",
                "has_sig": false,
                "md5_digest": "ce4e6fa4eaa35aa9176763c8f9f91dc7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 15700,
                "upload_time": "2018-09-24T15:14:55",
                "url": "https://files.pythonhosted.org/packages/bc/5a/e1ba51fadac3de4c738e63bf942e035f4665439643b6f419be4a4ccbe838/pyspark-flame-0.2.6.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "ce4e6fa4eaa35aa9176763c8f9f91dc7",
                "sha256": "807099c77fbaefaeb71217995bd22d203c697c6ce32c8e75ff53e1b1b56d9322"
            },
            "downloads": -1,
            "filename": "pyspark-flame-0.2.6.tar.gz",
            "has_sig": false,
            "md5_digest": "ce4e6fa4eaa35aa9176763c8f9f91dc7",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 15700,
            "upload_time": "2018-09-24T15:14:55",
            "url": "https://files.pythonhosted.org/packages/bc/5a/e1ba51fadac3de4c738e63bf942e035f4665439643b6f419be4a4ccbe838/pyspark-flame-0.2.6.tar.gz"
        }
    ]
}