{
    "info": {
        "author": "Jonathan Porta",
        "author_email": "jonathan@jonathanPorta.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5",
            "Topic :: Software Development :: Build Tools"
        ],
        "description": "# pyetl Framework\nA flask based framework for building and running ETL pipelines\n\n### Usage\nPyetl is meant as a framework to help with the extraction, transformation and loading of data between sources.\n\nTo get started, create a new Python project and then `pip install pyetl-framework`.\n\nTo run the app flask app frontend: `pyetl_flask`\nTo run the worker process: `pyetl_worker`\n\nThe following two environment vars are required:\n```\nexport APP_SETTINGS='DevelopmentConfig' # name of the corresponding config class for this env.\nexport APP_BASEDIR=$(pwd) # must point to directory containing your config file.\n```\n\nA config file is also required. See `config.py.example`.\n\n### Concepts\nThe framework relies heavily on naming conventions and magically imports.\n#### Pipe\nThis represents the flow of data from a source to a target. `pipe.start()` should do whatever is necessary determine and enqueue any and all `ETLJob`s that must execute in order for a run to be considered succesfull.\n\n#### ETLJob\nA base class defined in the framework. It has three methods: extract, transform, load.\n\n#### Transformer/Extractor/Loader\nExtend this class with a class that has the same name as your pipe's class. The ETLJob will run `{Transformer|Extractor|Loader}.execute()` when executing.\n\n### Setup Local Dev\nInstall python, virtualenv and deps. To get started go [here](https://realpython.com/blog/python/flask-by-example-part-1-project-setup).\n\nOnce\n\nThen: `pip install -r requirements.txt`\n\n### Pushing to Production/Staging on Heroku (Don't do this, ci should do this)\nThis is an example of how a sample app would deploy. This shouldn't be here.\ngit remote add heroku-staging git@heroku.com:pyscrape-staging.git\ngit remote add heroku-production git@heroku.com:pyscrape-production.git\nOr\n`make deploy`\n\n### Release\nFirst, create a new pip package. This will bump the patch version and write it to `VERSION`.\n`make package`\n\nThen, to push to the package to the repository:\n`make release`",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/jonathanporta/pyetl_framework",
        "keywords": "etl scraping",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "pyetl-framework",
        "package_url": "https://pypi.org/project/pyetl-framework/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/pyetl-framework/",
        "project_urls": {
            "Homepage": "https://github.com/jonathanporta/pyetl_framework"
        },
        "release_url": "https://pypi.org/project/pyetl-framework/0.0.53/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "A python based scraper for any data source.",
        "version": "0.0.53"
    },
    "last_serial": 2207432,
    "releases": {
        "0.0.42": [],
        "0.0.44": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fe8d696fedb970363016bb2ab4b7eee0",
                    "sha256": "b523fb8e0a57e562a3d7cfb23821cfbf2fe85043e0244eae460dfed575a24d62"
                },
                "downloads": -1,
                "filename": "pyetl_framework-0.0.44.tar.gz",
                "has_sig": false,
                "md5_digest": "fe8d696fedb970363016bb2ab4b7eee0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8492,
                "upload_time": "2016-06-11T05:54:51",
                "url": "https://files.pythonhosted.org/packages/8e/d7/bb996d42478a4c13e0c60a748a7a2fee5e5acef0d7285cbeaf00ae95b8bf/pyetl_framework-0.0.44.tar.gz"
            }
        ],
        "0.0.46": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "08b3229b619bb0721fde7a69a96aa0a9",
                    "sha256": "e991b42dcf56616f027389d7f88d55fc025a0e7a982736476c2490a4d904ce36"
                },
                "downloads": -1,
                "filename": "pyetl_framework-0.0.46.tar.gz",
                "has_sig": false,
                "md5_digest": "08b3229b619bb0721fde7a69a96aa0a9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8417,
                "upload_time": "2016-06-11T07:51:25",
                "url": "https://files.pythonhosted.org/packages/e1/db/aecc0c81a1c70b6d9a0cbbc1f844c58cb2f399c4f42f58f94da8b5fb3e84/pyetl_framework-0.0.46.tar.gz"
            }
        ],
        "0.0.47": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "96965eb3d65515265089a32432160c15",
                    "sha256": "11886e97218e37aa7f010a503dbba6a6fbe59cb56583f8d22c3cac8ab7ccad2a"
                },
                "downloads": -1,
                "filename": "pyetl_framework-0.0.47.tar.gz",
                "has_sig": false,
                "md5_digest": "96965eb3d65515265089a32432160c15",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8427,
                "upload_time": "2016-06-26T05:21:07",
                "url": "https://files.pythonhosted.org/packages/37/cf/9754b7f70d17d137035f92f3ff8c06cd0e713decd0247e578ade9459c04d/pyetl_framework-0.0.47.tar.gz"
            }
        ],
        "0.0.48": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "61f8e76d1c1b8ec8054ea0aae8c7d11b",
                    "sha256": "5177caf8825fc67492333eb71c41180a70e67103916288e52b14353c984b1056"
                },
                "downloads": -1,
                "filename": "pyetl_framework-0.0.48.tar.gz",
                "has_sig": false,
                "md5_digest": "61f8e76d1c1b8ec8054ea0aae8c7d11b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8547,
                "upload_time": "2016-06-26T05:48:33",
                "url": "https://files.pythonhosted.org/packages/01/6c/5b5143e420bf8545c218db1f420576da3689bdc86f2f06b0dc3f1a7ac74f/pyetl_framework-0.0.48.tar.gz"
            }
        ],
        "0.0.49": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1f3db69a76e9421e30696846f2359b37",
                    "sha256": "be5c311670e87255832b42c6251bafff3907fc9e058ce206b1a99b7152f73995"
                },
                "downloads": -1,
                "filename": "pyetl_framework-0.0.49.tar.gz",
                "has_sig": false,
                "md5_digest": "1f3db69a76e9421e30696846f2359b37",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8193,
                "upload_time": "2016-07-07T06:57:38",
                "url": "https://files.pythonhosted.org/packages/06/4f/a50ca58110bee9008a1bd8a36ba0cb798a62e4628447dfd1cdb5acd3f584/pyetl_framework-0.0.49.tar.gz"
            }
        ],
        "0.0.50": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4455e1468632d280c2b55976ad952be7",
                    "sha256": "5647204370f42688c5c9b57133aca79228270a935de1778bd0ac495f942a70dc"
                },
                "downloads": -1,
                "filename": "pyetl_framework-0.0.50.tar.gz",
                "has_sig": false,
                "md5_digest": "4455e1468632d280c2b55976ad952be7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8237,
                "upload_time": "2016-07-07T07:36:43",
                "url": "https://files.pythonhosted.org/packages/a5/37/10df7d6244742b5cd4d9db5e4b5f5b20b8adfa08d51237bbe1eb413294cb/pyetl_framework-0.0.50.tar.gz"
            }
        ],
        "0.0.51": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3ac1800b3451e965269969fb7557972b",
                    "sha256": "f8f23fdddf312e5b70cb22502385dec283553c49d55eda62844ac3d1fb8e16db"
                },
                "downloads": -1,
                "filename": "pyetl_framework-0.0.51.tar.gz",
                "has_sig": false,
                "md5_digest": "3ac1800b3451e965269969fb7557972b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8241,
                "upload_time": "2016-07-07T07:42:25",
                "url": "https://files.pythonhosted.org/packages/fa/b2/501d5c32e9fc41dd050b4ea9252c787accfcf16c2196fbde970ce422afad/pyetl_framework-0.0.51.tar.gz"
            }
        ],
        "0.0.52": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1ddbf02727354218c5bc23639cef1aa9",
                    "sha256": "2ead1488f7110a5c4e69f307534975bd206fa692eb2d337153b2d742ca9cce41"
                },
                "downloads": -1,
                "filename": "pyetl_framework-0.0.52.tar.gz",
                "has_sig": false,
                "md5_digest": "1ddbf02727354218c5bc23639cef1aa9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8238,
                "upload_time": "2016-07-07T07:44:45",
                "url": "https://files.pythonhosted.org/packages/b6/21/d0215b542a31eaca68f279052ea3d73f5e57cf58f0ab99c0bab631e88665/pyetl_framework-0.0.52.tar.gz"
            }
        ],
        "0.0.53": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4ddde6011a6b069e59d5c7912b4a62ef",
                    "sha256": "8e6ca0589a4d727c1d891d5eee2bae15ec6f36f580a0592552cc72a2f7cbb8a3"
                },
                "downloads": -1,
                "filename": "pyetl_framework-0.0.53.tar.gz",
                "has_sig": false,
                "md5_digest": "4ddde6011a6b069e59d5c7912b4a62ef",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8233,
                "upload_time": "2016-07-07T07:48:53",
                "url": "https://files.pythonhosted.org/packages/f2/5c/b51ef9e42dad0682d0c81b10c322037bc26e079a12283ae7989ecb3d2900/pyetl_framework-0.0.53.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "4ddde6011a6b069e59d5c7912b4a62ef",
                "sha256": "8e6ca0589a4d727c1d891d5eee2bae15ec6f36f580a0592552cc72a2f7cbb8a3"
            },
            "downloads": -1,
            "filename": "pyetl_framework-0.0.53.tar.gz",
            "has_sig": false,
            "md5_digest": "4ddde6011a6b069e59d5c7912b4a62ef",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 8233,
            "upload_time": "2016-07-07T07:48:53",
            "url": "https://files.pythonhosted.org/packages/f2/5c/b51ef9e42dad0682d0c81b10c322037bc26e079a12283ae7989ecb3d2900/pyetl_framework-0.0.53.tar.gz"
        }
    ]
}