{
    "info": {
        "author": "thingsplode",
        "author_email": "tamas.csaba@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Environment :: Console",
            "Environment :: Web Environment",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: Apache Software License",
            "Operating System :: POSIX",
            "Programming Language :: Python :: 3"
        ],
        "description": "## Why SQLizer\nIn many cases you can use SQL only for ETL (extract/transform/load) pipelines relying on CTAS (create table as) queries\nand the builting import/export futures of your RDBMS or data warehouse software (eg. Redshift).\n\n## What is SQLizer\nA simple orchestration service for SQL-only ETL workflows.\nThis service was born out of a need to orchestrate a complete data processing pipeline atop of AWS Redshift.\n\n### Roadmap\n[x] PostgreSQL/Resdhift support\n[x] Execiting multiple queries from a folder\n[ ] Executing a named query\n[ ] Executing an inline query\n[ ] MySQL support/Aurora support\n[ ] MongoDB support\n[ ] parallel execution of queries in one stage\n[ ] validation of the wrokflow \n[ ] DAG for stages\n[ ] multi-connection support\n\n## Developing SQLizer\n\n### Setting up the development environment\n\n```bash\npython3 -m venv ./.venv\necho \".venv/\" >> .gitignore\nsource .venv/bin/activate\npip install -e .\n```\n\nOptionally install development/test dependencies:\n```bash\npip install pytest pytest-runner codecov pytest-cov recommonmark\n```\n\nPrepare the docker image (and test it):\n```bash\ndocker build -t sqlizer .\ndocker run --rm  --name sqlizer-runner -e \"job_id=sqlizer\" -e \"bucket=sss\" sqlizer\n```\n\nPrepare test data:\n```bash\naws s3 mb s3://sqlizer-workflows --profile your-profile\naws s3 sync ~/Code/sqlizer/test-data/ s3://sqlizer-workflows --profile your-profile\n```\n\nAdd parameters to the Systems Manager's Parameter Store:\n```bash\naws ssm put-parameter --overwrite --name sqlizer.default.auth --value user:password --type SecureString --description \"authentication details for data-source\" --profile your-profile\naws ssm put-parameter --overwrite --name sqlizer.default.host --value \"some-cluster.redshift.amazonaws.com:5439/database\" --type SecureString --description \"url access for default data source\" --profile your-profile\n\n```\n\nRun it locally:\n```bash\nexport AWS_PROFILE=your-profile\n#sqlizer --connection-url=\"root:some_secret_pass@some-cluster.redshift.amazonaws.com:5439/database\" --bucket=\"s3://sqlizer-workflows\"\nsqlizer\n```\n\nPrepare the distribution: \n```bash\npip install -U setuptools wheel\npython setup.py build -vf && python setup.py bdist_wheel\npip install -U twine\n```\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/thingsplode/sqlizer",
        "keywords": "microservice,ETL,SQL,ETL Workflow,ETL Pipeline,DWH,data warehouse,airflow,luigi,orchestration",
        "license": "Apache 2.0",
        "maintainer": "",
        "maintainer_email": "",
        "name": "sqlizer",
        "package_url": "https://pypi.org/project/sqlizer/",
        "platform": "any",
        "project_url": "https://pypi.org/project/sqlizer/",
        "project_urls": {
            "Homepage": "https://github.com/thingsplode/sqlizer"
        },
        "release_url": "https://pypi.org/project/sqlizer/0.0.1/",
        "requires_dist": [
            "psycopg2",
            "pyyaml",
            "enum34",
            "simplejson",
            "boto3",
            "tabulate",
            "jinja2"
        ],
        "requires_python": "",
        "summary": "Orchestration service for SQL only ETL workflows.",
        "version": "0.0.1"
    },
    "last_serial": 5698720,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9d1cc6930af32c37eb861927106c8951",
                    "sha256": "d81939ca164c75cc1b56c2635d4dbc8ee45fc6c46f673202399665e6d08a2173"
                },
                "downloads": -1,
                "filename": "sqlizer-0.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "9d1cc6930af32c37eb861927106c8951",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 11541,
                "upload_time": "2019-08-19T14:30:23",
                "url": "https://files.pythonhosted.org/packages/a0/e8/6c49e35a22d74ff7010497982282e931f374747fc0204607d04148fd52a6/sqlizer-0.0.1-py3-none-any.whl"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "9d1cc6930af32c37eb861927106c8951",
                "sha256": "d81939ca164c75cc1b56c2635d4dbc8ee45fc6c46f673202399665e6d08a2173"
            },
            "downloads": -1,
            "filename": "sqlizer-0.0.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "9d1cc6930af32c37eb861927106c8951",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 11541,
            "upload_time": "2019-08-19T14:30:23",
            "url": "https://files.pythonhosted.org/packages/a0/e8/6c49e35a22d74ff7010497982282e931f374747fc0204607d04148fd52a6/sqlizer-0.0.1-py3-none-any.whl"
        }
    ]
}