{ "info": { "author": "['Jarred Glaser']", "author_email": "jarred.glaser@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3.6" ], "description": "# fast_to_sql\n\n## Introduction\n\nfast_to_SQL is an improved way to upload pandas dataframes to MS SQL server. The method borrows an idea from [here](https://iabdb.me/2016/07/13/a-better-way-load-data-into-microsoft-sql-server-from-pandas/), and turns it into a usable function. This function takes advantage of MS SQL server's multi-row insert ability. This can lead to MUCH faster speeds for uploading dataframes to SQL server (uploading a 10,000 row 5 column dataframe with pd.to_sql() took 517.97s, while uploading the same dataframe with fast_to_SQL took only 5.45s!). \n\nThe funciton also automatically preserves datatypes for: integer, float, string, boolean, and datetime64[ns] and converts them to SQL datatypes: int, float, varchar(255), bit, and datetime. Custom data types can also be set for a subset or all of the columns (see [usage](#usage)).\n\n## Installation\n\n```python\npip install fast_to_sql\n```\n\n## Requirements\n\n* Written for Python 3.6+\n* Requires pandas, sqlalchemy, datetime\n\n## Example\n\n```py\nfrom fast_to_SQL import fast_to_sql as fts\n\n# Create a SQL Alchemy Engine to desired server/database\nsqluser = \"DOMAIN\\USER\"\nsqlpass = \"Password\"\nserver = \"some_server\"\ndb = \"some_DB\"\n\nengine = create_engine(\"mssql+pyodbc://{}:{}@{}/{}?driver=SQL+Server&trusted_connection=true\"\n .format(sqluser,sqlpass,server,db))\n\ndf = SomePandasDF\n\n# Run main function\nfts.to_sql_fast(df, 'DFName', engine, if_exists = 'append', series = False, custom = {'column1':varchar(500)}, temp = False)\n```\n\n## Usage\n\n### Main function\n\n```python\nfts.to_sql_fast(df, name, engine, if_exists = 'append', series = False, custom = None, temp = False)\n```\n\n* ```df```: pandas DataFrame to upload\n* ```name```: String of desired name for the table in SQL server\n* ```engine```: A SQL alchemy engine\n* ```if_exists```: Option for what to do if the specified name already exists in the dataframe. If the dataframe does not exist a new one will be created. By default this option is set to 'append'\n * __'append'__: Appends the dataframe to the table if it already exists in SQL server.\n * __'fail'__: Purposely raises a FailError if the table already exists in SQL server.\n * __'replace'__: Drops the old table with the specified name, and creates a new one. Be careful with this option, it will completely delete a table with the specified name in SQL server.\n* ```series```: By default this is set to False. Set to True if the DataFrame is a series (only has one column).\n* ```custom```: A dictionary object with one or more of the column names being uploaded as the key, and a valid SQL data type as the value, this will override the default data type assigned to the column by the function.\n * Example: `{'ColumnName':'varchar(1000)'}`\n* ```temp```: Either `True` if creating a local temporary table, or `False` (default) if not. If set to `True` the temporary table will be dropped after the connection is closed\n\n## Caveats\n\n* This has only been tested with Microsoft SQL Server 2016 and `pyodbc` This may not work for other SQL databases.\n* The larger the database, the smaller speed imrpovements you will most likely see. This means that a 100 column, 500,000 row table, may still take a while to upload. This is because multi-row insert can only do a max of 1000 rows at a time.\n\n## Credits\n\n* This package is based on an excellent article from [here](https://iabdb.me/2016/07/13/a-better-way-load-data-into-microsoft-sql-server-from-pandas/)", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/jdglaser/fast_to_SQL", "keywords": "pandas to_sql fast sql", "license": "", "maintainer": "", "maintainer_email": "", "name": "fast-to-sql", "package_url": "https://pypi.org/project/fast-to-sql/", "platform": "", "project_url": "https://pypi.org/project/fast-to-sql/", "project_urls": { "Homepage": "https://github.com/jdglaser/fast_to_SQL" }, "release_url": "https://pypi.org/project/fast-to-sql/1.1.3/", "requires_dist": null, "requires_python": "", "summary": "An improved way of uploading pandas DataFrames to MS SQL server", "version": "1.1.3" }, "last_serial": 5773488, "releases": { "1.1.2": [ { "comment_text": "", "digests": { "md5": "1627c20d21869425dd98f2f357adc76c", "sha256": "ddd65f5a26fd2ae50c861119b8295be131f1c6b3f63826a91ed167630f8b1c7a" }, "downloads": -1, "filename": "fast_to_sql-1.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "1627c20d21869425dd98f2f357adc76c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 4455, "upload_time": "2018-07-17T17:07:37", "url": "https://files.pythonhosted.org/packages/09/cc/ec23159deeb04a04970f750087f10604563b786f0b2e69d194ee50e8fa58/fast_to_sql-1.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "283e08cc78f07466aa543059c9382504", "sha256": "e6dfae8d5635135fb7602222316ba4d0cdf68fde31008ec78e791c57cef27830" }, "downloads": -1, "filename": "fast-to-sql-1.1.2.tar.gz", "has_sig": false, "md5_digest": "283e08cc78f07466aa543059c9382504", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5793, "upload_time": "2018-07-17T17:07:39", "url": "https://files.pythonhosted.org/packages/92/11/7157d4865d0db565503d7d0474a746aab5685ec2384276768899b03f36c2/fast-to-sql-1.1.2.tar.gz" } ], "1.1.3": [ { "comment_text": "", "digests": { "md5": "fd5779eb786a4ce2e78059b360ec862e", "sha256": "ecda7c4d9e96addd2ce24caeee7ab0be7c74b36ec3b01aa33e807c5fe1739961" }, "downloads": -1, "filename": "fast_to_sql-1.1.3-py3-none-any.whl", "has_sig": false, "md5_digest": "fd5779eb786a4ce2e78059b360ec862e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 6934, "upload_time": "2019-09-03T02:14:56", "url": "https://files.pythonhosted.org/packages/bf/0a/aa4556efc862be586e4fc22c728c691aa76a82bf2c65c3b70c73089801c7/fast_to_sql-1.1.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2018279288f17a288048bc78bad4b495", "sha256": "caee71659a928667e8314cbf8d9d11460cf6b297d912b45cff6ee2b0f5dce902" }, "downloads": -1, "filename": "fast-to-sql-1.1.3.tar.gz", "has_sig": false, "md5_digest": "2018279288f17a288048bc78bad4b495", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5785, "upload_time": "2019-09-03T02:10:41", "url": "https://files.pythonhosted.org/packages/01/ce/d274c350c5212fcf70a6382f365cb3188044900cf5788190f8fc53979c3b/fast-to-sql-1.1.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "fd5779eb786a4ce2e78059b360ec862e", "sha256": "ecda7c4d9e96addd2ce24caeee7ab0be7c74b36ec3b01aa33e807c5fe1739961" }, "downloads": -1, "filename": "fast_to_sql-1.1.3-py3-none-any.whl", "has_sig": false, "md5_digest": "fd5779eb786a4ce2e78059b360ec862e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 6934, "upload_time": "2019-09-03T02:14:56", "url": "https://files.pythonhosted.org/packages/bf/0a/aa4556efc862be586e4fc22c728c691aa76a82bf2c65c3b70c73089801c7/fast_to_sql-1.1.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2018279288f17a288048bc78bad4b495", "sha256": "caee71659a928667e8314cbf8d9d11460cf6b297d912b45cff6ee2b0f5dce902" }, "downloads": -1, "filename": "fast-to-sql-1.1.3.tar.gz", "has_sig": false, "md5_digest": "2018279288f17a288048bc78bad4b495", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5785, "upload_time": "2019-09-03T02:10:41", "url": "https://files.pythonhosted.org/packages/01/ce/d274c350c5212fcf70a6382f365cb3188044900cf5788190f8fc53979c3b/fast-to-sql-1.1.3.tar.gz" } ] }