{ "info": { "author": "Andrew Chavez / The Dallas Morning News", "author_email": "newsapps@dallasnews.com", "bugtrack_url": null, "classifiers": [], "description": "# \ud83c\udfdb\ufe0f socrata2sql ![CircleCI (all branches)](https://img.shields.io/circleci/project/github/DallasMorningNews/socrata2sql.svg) ![](https://img.shields.io/pypi/v/socrata2sql.svg?style=flat) ![LICENSE](https://img.shields.io/github/license/DallasMorningNews/socrata2sql.svg?style=flat)\n\nPlenty of state and local governments use Socrata to run their open data portals. This tool allows you to grab a dataset from one of these portals and copy it into a SQL database of your choice. It uses the Socrata API to understand the columns in the dataset and attempts to create correctly-typed columns in the SQL database to match, including PostGIS geometries if the database and source dataset support them.\n\n## Requirements\n\n- Python 3.x\n\n## Installation\n\n```sh\n$ pipenv install socrata2sql\n```\n\n*or*\n\n```sh\n$ pip install socrata2sql\n```\n\n## Usage\n\n```\nSocrata to SQL database loader\n\nLoad a dataset from a Socrata-powered open data portal into a SQL database.\nUses the Socrata API to inspect the dataset, then sets up a table with matching\nSQL types and loads all rows. The loader supports any database supported by\nSQLalchemy.\n\nUsage:\n socrata2sql insert [-d=] [-a=] [-t=]\n socrata2sql ls [-a=]\n socrata2sql (-h | --help)\n socrata2sql (-v | --version)\n\nOptions:\n The domain for the open data site. Ex: www.dallasopendata.com\n The ID of the dataset on the open data site. This is usually\n a few characters, separated by a hyphen, at the end of the\n URL. Ex: 64pp-jeba\n -d= Database connection string for destination database as\n dialect+driver://username:password@host:port/database.\n Default: sqlite:///.sqlite\n -t= Destiation table in the database. Defaults to a sanitized\n version of the dataset's name on Socrata.\n -a= App token for the site. Only necessary for high-volume\n requests. Default: None\n -h --help Show this screen.\n -v --version Show version.\n\nExamples:\n List all datasets on the Dallas open data portal:\n $ socrata2sql ls www.dallasopendata.com\n\n Load the Dallas check register into a local SQLite file (file name chosen\n from the dataset name):\n $ socrata2sql insert www.dallasopendata.com 64pp-jeba\n\n Load it into a PostgreSQL database called mydb:\n $ socrata2sql insert www.dallasopendata.com 64pp-jeba -d=postgresql:///mydb\n```\n\n## Local development\n\nThere are (for now) very limited tests. They can be run with:\n\n```\n$ python setup.py tests\n```\n\n## Copyright\n\n© 2019 The Dallas Morning News\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/DallasMorningNews/socrata2sql", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "socrata2sql", "package_url": "https://pypi.org/project/socrata2sql/", "platform": "", "project_url": "https://pypi.org/project/socrata2sql/", "project_urls": { "Homepage": "http://github.com/DallasMorningNews/socrata2sql" }, "release_url": "https://pypi.org/project/socrata2sql/0.1.4/", "requires_dist": [ "docopt (~=0.6)", "progress (~=1.4)", "sodapy (~=1.5)", "SQLalchemy (~=1.2)", "tabulate (~=0.8)", "geoalchemy2 (~=0.5)" ], "requires_python": ">=3", "summary": "SQL loader for Socrata data sets", "version": "0.1.4" }, "last_serial": 4902698, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "41ba4814b4bb9a2578dc8ee506fc0892", "sha256": "448bac70bcdc6d2573ba106961b544d7b96465909036f0b68962e50b2a939055" }, "downloads": -1, "filename": "socrata2sql-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "41ba4814b4bb9a2578dc8ee506fc0892", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 6345, "upload_time": "2019-02-07T20:55:07", "url": "https://files.pythonhosted.org/packages/46/29/8ca27293c896f6a4dcc43d4e26ebeb4d976c2baee919b130eda9e4357ca2/socrata2sql-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "91e992c059175d90889f33f27bb5416c", "sha256": "0dadb5eb5dcfb6f41ccb7697b4ee91e85b0c3d90cf37652e969f4e0b7fcae27e" }, "downloads": -1, "filename": "socrata2sql-0.1.0.tar.gz", "has_sig": false, "md5_digest": "91e992c059175d90889f33f27bb5416c", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 5391, "upload_time": "2019-02-07T20:55:10", "url": "https://files.pythonhosted.org/packages/fa/f6/e74af938b25954c5c648a30649924518fe89268aefe0147550db763424a4/socrata2sql-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "2599d9680c5acfffc59f213069865e3f", "sha256": "8ef4ec68a9d5ab3d41de836d179101217a64fd3383961e697b513fc5642a9b89" }, "downloads": -1, "filename": "socrata2sql-0.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "2599d9680c5acfffc59f213069865e3f", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 8204, "upload_time": "2019-02-07T21:12:32", "url": "https://files.pythonhosted.org/packages/6f/61/6267d6cf7115d0d573b1da5e067b87580b86a05b05a6e0feaf708c634d78/socrata2sql-0.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2f44dcd8c0d99ccf27920c8c96073d28", "sha256": "b6cc1aca128380cbd8e5be3bcbfdf0d83a1e1ebbb07b005f068fca488723badd" }, "downloads": -1, "filename": "socrata2sql-0.1.1.tar.gz", "has_sig": false, "md5_digest": "2f44dcd8c0d99ccf27920c8c96073d28", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 6089, "upload_time": "2019-02-07T21:12:34", "url": "https://files.pythonhosted.org/packages/ed/6e/90923ec87ee06d6316412bd80f9d2b4c9ecd716e6dad79d4f25f230e8cdd/socrata2sql-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "62e5b6225fac22201db5e3358f013a3c", "sha256": "e1123c6eec3eac25a7191329db3b2a9bf108ee345aa7ce75734bd715cd5f830e" }, "downloads": -1, "filename": "socrata2sql-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "62e5b6225fac22201db5e3358f013a3c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 8210, "upload_time": "2019-02-08T16:35:15", "url": "https://files.pythonhosted.org/packages/b8/9a/124ce501e7642fb6a4d12445f0cfbe336aae4f5c9f83d014fc8c3b672883/socrata2sql-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "130cec9aa8ede1b642a6bcf8e0f399ea", "sha256": "8b040e8e564c6bd5df8014787c35a95c43811b2b663678b94650f8256ff35c81" }, "downloads": -1, "filename": "socrata2sql-0.1.2.tar.gz", "has_sig": false, "md5_digest": "130cec9aa8ede1b642a6bcf8e0f399ea", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 5792, "upload_time": "2019-02-08T16:35:17", "url": "https://files.pythonhosted.org/packages/4c/77/8b4fc27b5f964a2305c0d0c3ff0bc5e586931456e51ffe1acec047ec3443/socrata2sql-0.1.2.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "0510d61818b996ced977c54df3d7ad79", "sha256": "710b1bce42140b903cd5669560f71e8378b1cc4acd54798dfc6311bb3c9a1c17" }, "downloads": -1, "filename": "socrata2sql-0.1.4-py3-none-any.whl", "has_sig": false, "md5_digest": "0510d61818b996ced977c54df3d7ad79", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 8421, "upload_time": "2019-03-06T00:17:45", "url": "https://files.pythonhosted.org/packages/69/ca/8f9f7816bf71607002288412ffc14851b21fa74c4aa0783eb40384aad20e/socrata2sql-0.1.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "51662bb2d729b9dc2aca424c933b919a", "sha256": "e58536fc91004694d41203bf4412d4205fc822c18cf06cd5d51af0096e97ad77" }, "downloads": -1, "filename": "socrata2sql-0.1.4.tar.gz", "has_sig": false, "md5_digest": "51662bb2d729b9dc2aca424c933b919a", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 5990, "upload_time": "2019-03-06T00:17:46", "url": "https://files.pythonhosted.org/packages/22/3d/dac86bf74761a2d3674952355d81aa8ed90f89b442054881881f2d288941/socrata2sql-0.1.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "0510d61818b996ced977c54df3d7ad79", "sha256": "710b1bce42140b903cd5669560f71e8378b1cc4acd54798dfc6311bb3c9a1c17" }, "downloads": -1, "filename": "socrata2sql-0.1.4-py3-none-any.whl", "has_sig": false, "md5_digest": "0510d61818b996ced977c54df3d7ad79", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 8421, "upload_time": "2019-03-06T00:17:45", "url": "https://files.pythonhosted.org/packages/69/ca/8f9f7816bf71607002288412ffc14851b21fa74c4aa0783eb40384aad20e/socrata2sql-0.1.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "51662bb2d729b9dc2aca424c933b919a", "sha256": "e58536fc91004694d41203bf4412d4205fc822c18cf06cd5d51af0096e97ad77" }, "downloads": -1, "filename": "socrata2sql-0.1.4.tar.gz", "has_sig": false, "md5_digest": "51662bb2d729b9dc2aca424c933b919a", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 5990, "upload_time": "2019-03-06T00:17:46", "url": "https://files.pythonhosted.org/packages/22/3d/dac86bf74761a2d3674952355d81aa8ed90f89b442054881881f2d288941/socrata2sql-0.1.4.tar.gz" } ] }