{ "info": { "author": "Hideaki Takahashi", "author_email": "mymelo@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy", "Topic :: Database", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "|travisci build status|_\n|appveyor build status|_\n\nsqlitefts-python\n================\n\nsqlitefts-python provides binding for tokenizer of `SQLite Full-Text search(FTS3/4)`_ and `FTS5`_. it allows you to write tokenizers in Python.\n\n\nSQLite has Full-Text search feature FTS3/FTS4 and FTS5 along with some `predefined tokenizers for FTS3/4`_, and also `predefined tokenizers for FTS5`_.\nIt is easy to use and has enough functionality. Python has a built-in SQLite module,\nso that it is easy to use and deploy. You don't need anything else to full-text search.\n\nBut... the predefined tokenizers are not enough for some languages including Japanese. Also it is not easy to write own tokenizers.\nThis module provides ability to write tokenizers using Python with CFFI_, so that you don't need C compiler to write your tokenizer.\n\nIt also has ranking functions based on `peewee`_, utility function to add FTS5 auxiliary functions, and an FTS5 aux function implementation.\n\nNOTE: all connections using this modules should be explicitly closed. due to GC behavior, it can be crashed if a connection is left open when a program terminated.\n\nSample tokenizer\n----------------\nThere are differences between FTS3/4 and FTS5, so 2 different base classes are defined.\n\n- a tokenizer for FTS3/4 can be used with FTS5 by using FTS3TokenizerAdaptor.\n- a tokenizer for FTS5 can be used with FTS3/4 if 'flags' is not used.\n\nFTS3/4::\n\n import sqlitefts as fts\n\n class SimpleTokenizer(fts.Tokenizer):\n _p = re.compile(r'\\w+', re.UNICODE)\n\n def tokenize(self, text):\n for m in self._p.finditer(text):\n s, e = m.span()\n t = text[s:e]\n l = len(t.encode('utf-8'))\n p = len(text[:s].encode('utf-8'))\n yield t, p, p + l\n\n tk = sqlitefts.make_tokenizer_module(SimpleTokenizer())\n fts.register_tokenizer(conn, 'simple_tokenizer', tk)\n\nFTS5::\n\n from sqlitefts import fts5\n\n class SimpleTokenizer(fts5.FTS5Tokenizer):\n _p = re.compile(r'\\w+', re.UNICODE)\n\n def tokenize(self, text, flags=None):\n for m in self._p.finditer(text):\n s, e = m.span()\n t = text[s:e]\n l = len(t.encode('utf-8'))\n p = len(text[:s].encode('utf-8'))\n yield t, p, p + l\n\n tk = fts5.make_fts5_tokenizer(SimpleTokenizer())\n fts5.register_tokenizer(conn, 'simple_tokenizer', tk)\n\nRequirements\n============\n\n * Python 2.7, Python 3.3+, and PyPy2.7, PyPy3.2+\n * CFFI_\n * FTS3/4 and/or FTS5 enabled SQLite3 or APSW_ (for Windows, you may need to download and replace sqlite3.dll)\n\n * SQLite 3.11.x have to be compiled with -DSQLITE_ENABLE_FTS3_TOKENIZER to enable 2-arg fts3_tokenizer\n * SQLite 3.10.2 and older versions do not have extra requirements. 2-arg fts3_tokenizer is always avaiable.\n * SQLite 3.12.0 and later vesrions do not have extra requirements. 2-arg fts3_tokenizer can be enabled dynamically.\n\nNote for APSW users: An APSW Amalgamation build does not expose SQLite APIs used in this module, so libsqlite3.so/sqlite3.dll is also required even it has no runtime library dependencies on SQLite. An APSW local build already depends on the shared library. Detail: sqlite3_db_config can be invoked via Connection.config, but it rejects SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER to register a new tokenizer. tested at APSW 3.21.0-r1.\n\nLicence\n=======\n\nThis software is released under the MIT License, see LICENSE.\n\n\nThanks\n======\n\n * https://github.com/saaj\n\n\n.. _SQLite Full-Text search(FTS3/4): https://www.sqlite.org/fts3.html\n.. _FTS5: https://www.sqlite.org/fts5.html\n.. _predefined tokenizers for FTS3/4: https://www.sqlite.org/fts3.html#tokenizer\n.. _predefined tokenizers for FTS5: https://www.sqlite.org/fts5.html#section_4_3\n.. _peewee: https://github.com/coleifer/peewee\n.. _CFFI: https://cffi.readthedocs.io/en/latest/\n.. _ctypes: https://docs.python.org/library/ctypes.html\n.. |travisci build status| image:: https://api.travis-ci.org/hideaki-t/sqlite-fts-python.svg?branch=master\n.. _travisci build status: https://travis-ci.org/hideaki-t/sqlite-fts-python\n.. |appveyor build status| image:: https://ci.appveyor.com/api/projects/status/github/hideaki-t/sqlite-fts-python?svg=true\n.. _appveyor build status: https://ci.appveyor.com/project/hideaki-t/sqlite-fts-python\n.. _APSW: https://github.com/rogerbinns/apsw\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/hideaki-t/sqlite-fts-python/", "keywords": "SQLite,Full-text search,FTS", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "sqlitefts", "package_url": "https://pypi.org/project/sqlitefts/", "platform": "", "project_url": "https://pypi.org/project/sqlitefts/", "project_urls": { "Homepage": "https://github.com/hideaki-t/sqlite-fts-python/" }, "release_url": "https://pypi.org/project/sqlitefts/0.5.1/", "requires_dist": [ "cffi" ], "requires_python": "", "summary": "A Python binding for tokenizers of SQLite Full Text Search", "version": "0.5.1" }, "last_serial": 4306673, "releases": { "0.2": [ { "comment_text": "", "digests": { "md5": "285fd60c6b4d6d803171a06f4fa41866", "sha256": "99ed3d32f9e207e09049a0d569cd394b440b08dfdb3fe92c876577dbff7843df" }, "downloads": -1, "filename": "sqlitefts-0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "285fd60c6b4d6d803171a06f4fa41866", "packagetype": "bdist_wheel", "python_version": "3.4", "requires_python": null, "size": 6149, "upload_time": "2014-06-21T22:44:03", "url": "https://files.pythonhosted.org/packages/89/20/206f4aa5b195433dfc08a81be4951a88fef862aeef334e485f133aa80786/sqlitefts-0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ddd004cf7ae542ef7a0d486ddda6093e", "sha256": "c24f71ecee32165d6a277d5c6155c5ebf35b631bbc72bc648356a88ccbbd3c55" }, "downloads": -1, "filename": "sqlitefts-0.2.tar.gz", "has_sig": false, "md5_digest": "ddd004cf7ae542ef7a0d486ddda6093e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9375, "upload_time": "2014-06-21T22:43:53", "url": "https://files.pythonhosted.org/packages/f5/e1/e8a1961c4d0ba7d1894b95dff0566d27375156e7b24cffdaa4d01e9d1b38/sqlitefts-0.2.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "ee901aeef276476dc1c5ca883f82a444", "sha256": "583e29a275317b877c08dc45d3e1b7983143acf328f5d058028e4613e42400b5" }, "downloads": -1, "filename": "sqlitefts-0.3.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "ee901aeef276476dc1c5ca883f82a444", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 6528, "upload_time": "2016-07-16T23:48:52", "url": "https://files.pythonhosted.org/packages/d5/7d/bc096425d28a0c4b7aa4f1bc9ddf8b6b08f090ec0f4818fa6499d72ef9e9/sqlitefts-0.3.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "231e01a06d02699e2b5665a4d3d70779", "sha256": "ab874056c2e0a30725e75a811250133e462285d57ed0f79e648a371b8e211386" }, "downloads": -1, "filename": "sqlitefts-0.3.0.tar.gz", "has_sig": false, "md5_digest": "231e01a06d02699e2b5665a4d3d70779", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12913, "upload_time": "2016-07-16T23:48:54", "url": "https://files.pythonhosted.org/packages/9c/df/50b8c2fa33381183f6fc3a8859bfcfab85918807e2a2164b3b70d8e7696c/sqlitefts-0.3.0.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "00be693474f1635b03c8860f492b6897", "sha256": "777933b2b6aebdb0709de9dbb632a41b0145c9d91ebe1eec72b0bf8d2a9e663c" }, "downloads": -1, "filename": "sqlitefts-0.3.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "00be693474f1635b03c8860f492b6897", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 6504, "upload_time": "2016-07-17T00:47:16", "url": "https://files.pythonhosted.org/packages/05/0f/7f700c9dd340da9dc39f4e4281eb9a5eb79280e6742411446ca88c9bdbab/sqlitefts-0.3.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e658b21e1b280f8d100ff3182b3e6f41", "sha256": "4573578d35ad9fc453b4ff210b913263208169e5ec5a0d6d741058d22381cf4f" }, "downloads": -1, "filename": "sqlitefts-0.3.1.tar.gz", "has_sig": false, "md5_digest": "e658b21e1b280f8d100ff3182b3e6f41", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12890, "upload_time": "2016-07-17T00:47:19", "url": "https://files.pythonhosted.org/packages/04/58/7329266162bff79069c9c6d070dbada8a1ed56d152374f60a10885aa0cb3/sqlitefts-0.3.1.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "efb6d39c4a408298214f82d920026c3c", "sha256": "4351d8df8faf996f3aeabee7ddb1fc29ea66f452c442ef76977365f7a8623652" }, "downloads": -1, "filename": "sqlitefts-0.4.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "efb6d39c4a408298214f82d920026c3c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7334, "upload_time": "2016-08-21T13:44:29", "url": "https://files.pythonhosted.org/packages/2b/ed/cb3bcf1673f562c938cd7aa6801e7edbdbb43d260e167d6b79af9a6a4976/sqlitefts-0.4.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8d158b19f9cc514fd0618ec9a2066c4e", "sha256": "7cdf2494290903fbc98f5a2d27138aa76a57cb278c285bc255738216db7bced8" }, "downloads": -1, "filename": "sqlitefts-0.4.0.tar.gz", "has_sig": false, "md5_digest": "8d158b19f9cc514fd0618ec9a2066c4e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13333, "upload_time": "2016-08-21T13:44:31", "url": "https://files.pythonhosted.org/packages/ff/bd/63295e840b9c6af288020e007daa46b8abcac63eb7fe48d827a5ce01dd12/sqlitefts-0.4.0.tar.gz" } ], "0.4.9": [ { "comment_text": "", "digests": { "md5": "26a25c38bb3988328bd6b9b0bd4afec4", "sha256": "53cac208509ac2012d20cbd585db1146c2c2bb26659634e4441dea96b5fc986a" }, "downloads": -1, "filename": "sqlitefts-0.4.9-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "26a25c38bb3988328bd6b9b0bd4afec4", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8027, "upload_time": "2017-07-12T10:55:42", "url": "https://files.pythonhosted.org/packages/e5/1d/7bc517cfc1427e7c41ff4ae0a9d328bdd2e9e20b2b69d33327d11f250efc/sqlitefts-0.4.9-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "7a36f4848c7964f695a35914d813fde6", "sha256": "e24df21ac5b9678ee918e26162c3ac1c67a0e1fcdc95790d4cba412c1dd1c375" }, "downloads": -1, "filename": "sqlitefts-0.4.9.tar.gz", "has_sig": false, "md5_digest": "7a36f4848c7964f695a35914d813fde6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15144, "upload_time": "2017-07-12T10:55:43", "url": "https://files.pythonhosted.org/packages/a4/19/3e578176941b48dfa90d6a16f597f49bdd1c6aa26f4875bfd04410c4c428/sqlitefts-0.4.9.tar.gz" } ], "0.4.9.1": [ { "comment_text": "", "digests": { "md5": "13999d3a5d3ecfd3e6d9ccf77fd04cb5", "sha256": "b38020562e1a0c68abbce6fbc885d8a9162de123777c05df52401b8f6c05f33c" }, "downloads": -1, "filename": "sqlitefts-0.4.9.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "13999d3a5d3ecfd3e6d9ccf77fd04cb5", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8088, "upload_time": "2017-07-12T11:12:15", "url": "https://files.pythonhosted.org/packages/67/55/05653dc4283d0e0a0f122d3ec4f96fed93798353a3437702e0c5a1d54b60/sqlitefts-0.4.9.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "5026b5302aa9986553b5cc36125c88af", "sha256": "7099fea5e5a038445b9ea8919025d35283397be4fc0a172972204df7c576b612" }, "downloads": -1, "filename": "sqlitefts-0.4.9.1.tar.gz", "has_sig": false, "md5_digest": "5026b5302aa9986553b5cc36125c88af", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15223, "upload_time": "2017-07-12T11:12:16", "url": "https://files.pythonhosted.org/packages/db/e6/96c350d899bbcd9462e25d9c611791bd4085fcc5bb8e98091579c4bc9728/sqlitefts-0.4.9.1.tar.gz" } ], "0.4.9.2": [ { "comment_text": "", "digests": { "md5": "03462b14d967801d78fe900f7290f416", "sha256": "6bfe2525ec2e7438768df102dace5173fbe8c7538005b2fe87de40fe107380e3" }, "downloads": -1, "filename": "sqlitefts-0.4.9.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "03462b14d967801d78fe900f7290f416", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8178, "upload_time": "2017-08-11T12:36:36", "url": "https://files.pythonhosted.org/packages/37/62/fa534ffe774f1e1cb986adc4a1c51c9566ddae22b0d8d8f9d7c78b367fa8/sqlitefts-0.4.9.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "fafd7239dc150c27dec3132446d7ba5a", "sha256": "cd69ea7f5ac8449da0ba0c2fbd3e8613144e22b596fd02a5f43894745fc8011f" }, "downloads": -1, "filename": "sqlitefts-0.4.9.2.tar.gz", "has_sig": false, "md5_digest": "fafd7239dc150c27dec3132446d7ba5a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14555, "upload_time": "2017-08-11T12:36:37", "url": "https://files.pythonhosted.org/packages/c3/f5/9865d2a96830878ee8b8e9c281d329ffac2d1ee076c21ce0aff2c8f86886/sqlitefts-0.4.9.2.tar.gz" } ], "0.5.1": [ { "comment_text": "", "digests": { "md5": "f96617501ef7bf2f93a71e78f914bdb3", "sha256": "59b5713c3d26305eb212eeccc902e0651ea10016746bd0865b12f3f8b95c38b3" }, "downloads": -1, "filename": "sqlitefts-0.5.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f96617501ef7bf2f93a71e78f914bdb3", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10707, "upload_time": "2018-09-24T23:59:07", "url": "https://files.pythonhosted.org/packages/e2/41/e4572adf1f5c3a72511174fd4668ba5019b35d922e015bf25399f62a17fd/sqlitefts-0.5.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "011dc82901dba2ef20437bbd1c15db95", "sha256": "01690a49b6878da24039eec8aed4741f4be44696af64560ea7c3ab8c7ca53737" }, "downloads": -1, "filename": "sqlitefts-0.5.1.tar.gz", "has_sig": false, "md5_digest": "011dc82901dba2ef20437bbd1c15db95", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17864, "upload_time": "2018-09-24T23:59:08", "url": "https://files.pythonhosted.org/packages/eb/7a/4ecb9ba105104064391377fab36315bc9660782174be85182adbbdaefd4b/sqlitefts-0.5.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "f96617501ef7bf2f93a71e78f914bdb3", "sha256": "59b5713c3d26305eb212eeccc902e0651ea10016746bd0865b12f3f8b95c38b3" }, "downloads": -1, "filename": "sqlitefts-0.5.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f96617501ef7bf2f93a71e78f914bdb3", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10707, "upload_time": "2018-09-24T23:59:07", "url": "https://files.pythonhosted.org/packages/e2/41/e4572adf1f5c3a72511174fd4668ba5019b35d922e015bf25399f62a17fd/sqlitefts-0.5.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "011dc82901dba2ef20437bbd1c15db95", "sha256": "01690a49b6878da24039eec8aed4741f4be44696af64560ea7c3ab8c7ca53737" }, "downloads": -1, "filename": "sqlitefts-0.5.1.tar.gz", "has_sig": false, "md5_digest": "011dc82901dba2ef20437bbd1c15db95", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17864, "upload_time": "2018-09-24T23:59:08", "url": "https://files.pythonhosted.org/packages/eb/7a/4ecb9ba105104064391377fab36315bc9660782174be85182adbbdaefd4b/sqlitefts-0.5.1.tar.gz" } ] }