{ "info": { "author": "Sanhe Hu", "author_email": "husanhe@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: MacOS", "Operating System :: Microsoft :: Windows", "Operating System :: Unix", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4" ], "description": "A high performance and easy to use sqlite API for Data Scientist.\r\n================================================================================\r\n\r\n.. image:: http://sqlite4dummy-project.readthedocs.org/_static/sqlite4dummy-logo.jpg\r\n\t:width: 500px\r\n\t:height: 500px\r\n\r\nQuick links:\r\n\r\n- `Home page `_\r\n- `Online Documentation `_\r\n- `PyPI download `_\r\n- `Install `_\r\n\r\n\r\nEnglish Introduction\r\n--------------------------------------------------------------------------------\r\n\r\n``sqlite4dummy`` is a high level Python sqlite3 database API. \r\n\r\n\r\n- What ``sqlite4dummy`` can do?\r\n\r\n\t1. Provide an OOD database syntax, allow developer doing regular CRUD and DB level operation **without writing any SQL**.\r\n\t2. Extend ability of SQL with Python, so seamlessly combine benefits of these two.\r\n\t3. Provide a feasible way to **save/read any Python pickable type to/from database.** Once the column is defined, no extra code is needed.\r\n\t4. **Ultra fast speed** that beat MySql, Postgres, Oracle, MSSQL...\r\n\r\n- Intended audience:\r\n\r\n\t1. developer\r\n\t2. **data analyst deal with large volumn data**\r\n\t3. **data scientist**\r\n\t4. test engineer\r\n\t5. non-dba developer\r\n\r\n- Highlights:\r\n\r\n\t1. Faster than the Top Database Project in Python Community - ``sqlalchemy``.\r\n\t2. Human language like syntax, minimal code is needed.\r\n\t3. A lots of vanilla method are provided for frequent-used work and data manipulation.\r\n\r\n- Why sqlite?\r\n\r\n\tOne of a common complain I heard for R/Matlab is that: It is too slow with large amount of data! sqlite is a database that:\r\n\r\n\t1. single file (no software install needed) portable\r\n\t2. ultra fast, (C implementation), high speed I/O\r\n\r\n\tAlthough sqlite doens't provide features like ``user group``, ``transaction``, ``failure over and backup``, but in non-production environment, we don't need it at all! But **we can always take the benefit of high I/O performance and \"out of box, no setup needed\" in development and data analysis!**\r\n\r\n- why ``sqlite4dummy`` beat ``sqlalchemy`` in sqlite?\r\n\r\n\t1. In bulk insert operation, sometime we meet primary key conflict. In this scenario, we have to insert records one by one, catch the exception and handle it. Because Sqlalchemy is created to be compatible with most of database system, the way Sqlalchemy handle the exception is rollback. But, sqlite is so special. In sqlite, there's only one writer is allowed at one time, and there's no transaction. That's why sqlite don't need the rollback mechanism. In the sqlite Python generic API, we can simple pass that exception. As result, **the generic API is 50-100 times faster than Sqlalchemy** when there's conflict in bulk insert.\r\n\r\n\t2. Sqlalchemy use Rowproxy to preprocess the data that cursor returns. After that, we can visit value by the column name. But sometime, we actually don't need this feature. A better way is to activate this feature when we need it. That makes **Sqlalchemy is 1.5 to 3 times slower generic API**.\r\n\r\nOne final comment, you can use ``Sqlite3Engine.execute(\"any sql...\")`` to execute any SQL with generic sqlite3 Python API for those features ``sqlite4dummy`` doesn't have (such as foreign key, left joint).\r\n\r\nThank you for using ``sqlite4dummy``, for installation, click `here `_.\r\n\r\n\r\nChinese Introduction (\u4e2d\u6587\u4ecb\u7ecd)\r\n--------------------------------------------------------------------------------\r\n\r\n``sqlite4dummy`` \u662f\u4e00\u4e2a ``sqlite`` \u6570\u636e\u5e93\u7684\u7b80\u5355API\u5e93\u3002 \u76ee\u7684\u662f **\u4e3a\u6570\u636e\u5206\u6790\u4eba\u5458, \u548c\u975e\u6570\u636e\u5e93\u4e13\u4e1a\u7684\u5f00\u53d1\u8005** \u4f7f\u7528\u6570\u636e\u63d0\u4f9b\u4e00\u5957: \u7c7b\u4f3c\u4eba\u7c7b\u7684\u8bed\u8a00, \u6ca1\u6709\u5197\u4f59\u4ee3\u7801, \u4e0d\u7528\u5199\u7eafSQL\u8bed\u53e5\u7684\u6570\u636e\u5e93\u89e3\u51b3\u65b9\u6848\u3002 \u8ba9\u975e\u8ba1\u7b97\u673a\u80cc\u666f\u7684\u6570\u636e\u5206\u6790\u4eba\u5458\u4e5f\u80fd\u8f7b\u677e\u6109\u5feb\u7684\u4f7f\u7528sqlite\u6570\u636e\u5e93\u5e26\u6765\u7684\u6781\u5927\u4fbf\u5229\u3002\r\n\r\n- ``sqlite4dummy`` \u80fd\u505a\u4ec0\u4e48?\r\n\r\n\t``sqlite4dummy`` \u662f\u4e00\u4e2a\u5efa\u9020\u4e8ePython\u539f\u751f ``sqlite3`` \u6a21\u5757\u4e4b\u4e0a\u7684 ``sqlite`` \u6570\u636e\u5e93\u7684\u4e00\u5957\u9762\u5411\u5bf9\u8c61API\u3002 \u5176\u7279\u70b9\u662f\u80fd\u8ba9\u7528\u6237\u5728\u4e0d\u5199\u4e00\u884cSQL\u8bed\u53e5\u7684\u60c5\u51b5\u4e0b, \u5bf9\u6570\u636e\u5e93\u8fdb\u884c\u5e38\u89c4\u7684 \"\u589e\u5220\u67e5\u6539\" \u4ee5\u53ca\u6570\u636e\u5e93\u7ef4\u62a4\u7684\u5de5\u4f5c\u3002 \u5e76\u4e14\u7531\u4e8e\u4f7f\u7528\u7684\u662f\u6570\u636e\u5e93\u4e2d\u62bd\u8c61\u6982\u5ff5\u7684\u5bf9\u8c61\u4f5c\u4e3a\u63a5\u53e3, \u6240\u4ee5\u8ba9\u6570\u636e\u5e93\u8fdb\u884c\u6279\u91cf\u590d\u6742\u64cd\u4f5c\u53d8\u5f97\u53ef\u7f16\u7a0b\u5316\u3002 \u4f7f\u5f97\u80fd\u505a\u8bb8\u591aSQL\u6240\u4e0d\u80fd\u505a\u7684\u4e8b\u60c5\u3002\r\n\r\n- ``sqlite4dummy`` \u9002\u5408\u4ec0\u4e48\u4eba\u7528?\r\n\r\n\t\u5f00\u53d1\u8005, \u5229\u7528\u6570\u636e\u5e93\u7684\u9ad8\u541e\u5410\u6027\u80fd\u8fdb\u884c\u6570\u636e\u5206\u6790\u7684\u4eba, \u6570\u636e\u79d1\u5b66\u5bb6, \u6d4b\u8bd5\u4eba\u5458\u548c\u975e\u6570\u636e\u5e93\u7ba1\u7406\u5458\u5f00\u53d1\u8005\u3002\r\n\r\n- ``sqlite4dummy`` \u6709\u4ec0\u4e48\u4eae\u70b9?\r\n\r\n\t1. \u901f\u5ea6\u8d85\u5feb, \u5feb\u4e8ePython\u793e\u533a\u7b2c\u4e00\u7684\u5173\u7cfb\u6570\u636e\u5e93\u9879\u76ee ``sqlalchemy``\r\n\t2. \u8bed\u6cd5\u7c7b\u4f3c\u4eba\u7c7b\u8bed\u8a00, \u6613\u8bfb, \u4f7f\u5f00\u53d1\u65f6\u9700\u8981\u5199\u7684\u4ee3\u7801\u91cf\u5927\u91cf\u51cf\u5c11\r\n\t3. \u63d0\u4f9b\u4e86\u5f88\u591a\u989d\u5916\u7684\u5feb\u6377\u65b9\u6cd5(vanilla method)\r\n\r\n- \u4e3a\u4ec0\u4e48\u8981\u7528 ``sqlite`` \u6570\u636e\u5e93?\r\n\r\n\tsqlite\u7531\u4e8e\u5e76\u6ca1\u6709\u5f88\u591a\u5176\u4ed6\u5173\u7cfb\u6570\u636e\u5e93\u5728\u8fd0\u7ef4\u65b9\u9762\u7684\u590d\u6742\u529f\u80fd, \u6240\u4ee5\u4f7f\u5f97 **\u6027\u80fd\u975e\u5e38\u4f18\u5f02**\u3002 \u8fd9\u4e00\u7279\u6027 **\u975e\u5e38\u9002\u5408\u79d1\u5b66\u5b66\u79d1\u7684\u5de5\u4f5c\u8005\u8fdb\u884c\u5927\u5bb9\u91cf\u7684\u6570\u636e\u5206\u6790**\u3002 \u4f8b\u5982\u5728\u5904\u7406\u5927\u4e8e1G\u4ee5\u4e0a\u7684csv\u6587\u4ef6\u65f6, \u5c06\u6240\u6709\u6570\u636e\u653e\u5165\u6570\u636e\u5e93, \u5efa\u7acb\u7d22\u5f15, \u8fdb\u884c\u67e5\u627e\u7684\u901f\u5ea6\u8981\u8fdc\u8fdc\u5feb\u4e8e\u76f4\u63a5\u5bf9\u6570\u636e\u6587\u4ef6\u8fdb\u884c\u5904\u7406\u3002 \r\n\r\n\t\u800cR, Matlab\u7b49\u8bed\u8a00\u90fd\u9700\u8981\u5c06\u6570\u636e\u5168\u90e8\u653e\u5165\u5185\u5b58\u4e2d\u624d\u80fd\u8fdb\u884c\u5904\u7406\u3002 \u800c\u4f7f\u7528\u6570\u636e\u5e93\u4f5c\u4e3a\u4e2d\u95f4\u4ef6\u50a8\u5b58\u6570\u636e, \u7136\u540e\u5229\u7528\u5176\u9ad8\u541e\u5410\u6027\u80fd\u8fdb\u884c\u5904\u7406\u6570\u636e, \u4f7f\u5f97\u4eba\u4eec\u80fd\u638c\u63a7\u7684\u6570\u636e\u91cf\u4e0a\u9650\u5927\u5927\u589e\u52a0\u3002\r\n\r\n- \u76f8\u6bd4 ``sqlalchemy``, ``sqlite4dummy`` \u6709\u4ec0\u4e48\u4f18\u52bf?\r\n\r\n\tSA\u4e3a\u4e86\u80fd\u591f\u517c\u5bb9\u6240\u6709\u4e3b\u6d41\u5173\u7cfb\u6570\u636e\u5e93, \u6240\u4ee5\u727a\u7272\u4e86\u4e00\u4e9b\u6027\u80fd\u3002 SA\u5728\u6570\u636e\u5e93\u8fd0\u7ef4, \u4ee5\u53caORM\u7684\u60c5\u51b5\u4e0b\u6709\u5f88\u591a\u529f\u80fd\u975e\u5e38\u6709\u7528, \u4f46\u662f\u5728\u8bb8\u591a\u60c5\u51b5\u4e0b, \u4f8b\u5982\u6570\u636e\u5206\u6790\u4e2d, \u5e76\u4e0d\u80fd\u7ed9\u6211\u4eec\u5e26\u6765\u591a\u5c11\u4fbf\u5229\u3002\r\n\r\n\t\u5bf9\u4e8e\u6570\u636e\u79d1\u5b66\u5bb6\u800c\u8a00, sqlite\u662f\u4e00\u4e2a\u975e\u5e38\u9002\u5408\u52a0\u901fIO\u7684\u6570\u636e\u5e93\u3002\u5355\u6587\u4ef6, C\u5b9e\u73b0, \u7b80\u5355\u9ad8\u901f, \r\n\t\u8fd9\u4e9b\u7279\u6027\u90fd\u975e\u5e38\u9002\u5408\u5206\u6790\u4e2d\u7b49\u5927\u5c0f(1GB - 1TB)\u7684\u6570\u636e\u96c6\u3002\u800cTransaction, Session, User Group\u8fd9\u4e9b\u529f\u80fd, \u6211\u4eec\u5e76\u4e0d\u9700\u8981\u3002\r\n\r\n\t\u6b64\u5916, SA\u5728\u6027\u80fd\u4e0a\u6709\u4e24\u4e2a\u81f4\u547d\u7684\u5f31\u70b9:\r\n\r\n\t1. SA\u5728\u6267\u884c ``Select`` \u7684\u65f6\u5019, \u8c03\u7528\u4e86\u4e00\u79cd\u53eb\u505a ``Rowproxy`` \u7684\u673a\u5236, \u5c06\u6240\u6709\u7684\u884c\u6253\u5305\u6210\u5b57\u5178, \u65b9\u4fbf\u6211\u4eec\u8fdb\u884c\u8bfb\u53d6\u3002\u8fd9\u4e00\u7279\u6027\u6211\u4eec\u5e76\u4e0d\u662f100%\u7684\u9700\u8981, \u800c\u6211\u4eec\u5b8c\u5168\u53ef\u4ee5\u5728\u9700\u8981\u7684\u65f6\u5019, \u518d\u6253\u5305\u6210\u5b57\u5178\u3002 \u8fd9\u4f7f\u5f97SA **\u5728Select\u8fd4\u56de\u5927\u91cf\u6570\u636e\u7684\u60c5\u51b5\u4e0b, \u8981\u6bd4 ``sqlite4dummy`` \u616250%\u5de6\u53f3\u3002**\r\n\r\n\t2. SA\u5728\u6267\u884c ``Insert`` \u7684\u65f6\u5019, \u5982\u679c\u53d1\u751f\u4e86 ``primary key conflict``, \u7531\u4e8eSA\u9700\u8981\u517c\u5bb9\u6240\u6709\u7684\u6570\u636e\u5e93,\u6240\u4ee5SA\u4f7f\u7528\u4e86 ``rollback`` \u673a\u5236\u3002 \u800c\u7531\u4e8esqlite3\u53ea\u652f\u6301\u5355\u7ebf\u7a0b\u7684write, \u6240\u4ee5\u5728\u5904\u7406\u51b2\u7a81\u7684\u65f6\u5019\u8981\u6bd4\u591a\u7ebf\u7a0b\u7b80\u5355\u5f88\u591a, \u5bfc\u81f4SA\u7684\u901f\u5ea6\u5728 **\u5f53\u5199\u5165\u7684\u6570\u636e\u4e0e\u6570\u636e\u8868\u4e2d\u7684\u6570\u636e\u6709\u51b2\u7a81\u7684\u65f6\u5019, \u901f\u5ea6\u8981\u6bd4\u539f\u751fsqliteAPI\u6162\u51e0\u5341\u500d\u751a\u81f3\u767e\u500d\u3002**\r\n\r\n\u76ee\u524d ``sqlite4dummy`` \u6ca1\u6709\u63d0\u4f9b\u5b9a\u4e49 ``foreign key`` \u7684\u8bed\u6cd5, \u4f46\u662f\u4f60\u53ef\u4ee5\u4f7f\u7528: ``Sqlite3Engine.execute(\"any sql...\")`` \u8fdb\u884c\u5b9a\u4e49\u3002 \u6362\u8a00\u4e4b, \u6240\u6709 ``sqlite4dummy`` \u6ca1\u6709\u7684\u529f\u80fd, \u90fd\u53ef\u4ee5\u901a\u8fc7\u8fd9\u4e00\u65b9\u6cd5\u8c03\u7528\u539f\u751f\u7684SQL\u8bed\u53e5\u5b8c\u6210\u3002\r\n\r\n\r\n.. _install:\r\n\r\nInstall\r\n---------------------------------------------------------------------------------------------------\r\n\r\n``sqlite4dummy`` is released on PyPI, so all you need is:\r\n\r\n.. code-block:: console\r\n\r\n\t$ pip install sqlite4dummy\r\n\r\nTo upgrade to latest version:\r\n\r\n.. code-block:: console\r\n\t\r\n\t$ pip install --upgrade sqlite4dummy", "description_content_type": null, "docs_url": null, "download_url": "https://github.com/MacHu-GWU/sqlite4dummy-project/tarball/2015-11-03", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/MacHu-GWU/sqlite4dummy-project", "keywords": "Python, sqlite, sqlite3, API, database, data science, fast", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "sqlite4dummy", "package_url": "https://pypi.org/project/sqlite4dummy/", "platform": "Windows,MacOS,Unix", "project_url": "https://pypi.org/project/sqlite4dummy/", "project_urls": { "Download": "https://github.com/MacHu-GWU/sqlite4dummy-project/tarball/2015-11-03", "Homepage": "https://github.com/MacHu-GWU/sqlite4dummy-project" }, "release_url": "https://pypi.org/project/sqlite4dummy/0.0.6/", "requires_dist": null, "requires_python": null, "summary": "A high performance and easy to use sqlite API for Data Scientist.", "version": "0.0.6" }, "last_serial": 1826655, "releases": { "0.0.1": [], "0.0.2": [ { "comment_text": "", "digests": { "md5": "f18ed489d101f6b132f65c902f7d2641", "sha256": "1e7f70dab6f0f8a56a10d0098fed869e4c72c91a593d2e8d978a9c89a6155fe3" }, "downloads": -1, "filename": "sqlite4dummy-0.0.2.zip", "has_sig": false, "md5_digest": "f18ed489d101f6b132f65c902f7d2641", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 53273, "upload_time": "2015-09-17T05:05:24", "url": "https://files.pythonhosted.org/packages/93/8d/0a0b17e87463d65c5ed69250b2c24d3785b01d30ad96f8a698483eaf534d/sqlite4dummy-0.0.2.zip" } ], "0.0.3": [], "0.0.4": [ { "comment_text": "", "digests": { "md5": "4426c2daa507735315e7b6bda74362a0", "sha256": "9a82e022ce555a92732ac63bfbe5fbeb9b2c57bfb1c31994dad4918bb850760b" }, "downloads": -1, "filename": "sqlite4dummy-0.0.4.zip", "has_sig": false, "md5_digest": "4426c2daa507735315e7b6bda74362a0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 61238, "upload_time": "2015-09-19T01:55:39", "url": "https://files.pythonhosted.org/packages/25/45/485ded10ca146122749339c8f39571691f612a56a8bccb0c4f51063630ef/sqlite4dummy-0.0.4.zip" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "cfa9045bfa06db8f21829a53618a40aa", "sha256": "fb8a877bd77d1a77fd08f527df45d774a44ae3827fb34996e0f8eaadade2f0db" }, "downloads": -1, "filename": "sqlite4dummy-0.0.5.zip", "has_sig": false, "md5_digest": "cfa9045bfa06db8f21829a53618a40aa", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 62864, "upload_time": "2015-09-25T00:56:09", "url": "https://files.pythonhosted.org/packages/9f/1b/0f3030daa1a24e3e677187841b9091a971f919bf7aad3b53512efe2d25ac/sqlite4dummy-0.0.5.zip" } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "7515422064d37ad379be3ad297764d7a", "sha256": "b4253472bbd638a8cdfd6bc4c028d283e2ede1aea230f369d9f8badd6f1d498b" }, "downloads": -1, "filename": "sqlite4dummy-0.0.6.zip", "has_sig": false, "md5_digest": "7515422064d37ad379be3ad297764d7a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 75495, "upload_time": "2015-11-03T22:47:01", "url": "https://files.pythonhosted.org/packages/37/f2/6ddc1142943ff9b15efd983b3819497787abc5b788f2ae7f561b68c468a1/sqlite4dummy-0.0.6.zip" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "7515422064d37ad379be3ad297764d7a", "sha256": "b4253472bbd638a8cdfd6bc4c028d283e2ede1aea230f369d9f8badd6f1d498b" }, "downloads": -1, "filename": "sqlite4dummy-0.0.6.zip", "has_sig": false, "md5_digest": "7515422064d37ad379be3ad297764d7a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 75495, "upload_time": "2015-11-03T22:47:01", "url": "https://files.pythonhosted.org/packages/37/f2/6ddc1142943ff9b15efd983b3819497787abc5b788f2ae7f561b68c468a1/sqlite4dummy-0.0.6.zip" } ] }