{ "info": { "author": "Yusuke Matsui", "author_email": "matsui528@gmail.com", "bugtrack_url": null, "classifiers": [], "description": "# nanopq\n\n[![Build Status](https://travis-ci.org/matsui528/nanopq.svg?branch=master)](https://travis-ci.org/matsui528/nanopq)\n[![Documentation Status](https://readthedocs.org/projects/nanopq/badge/?version=latest)](https://nanopq.readthedocs.io/en/latest/?badge=latest)\n[![PyPI version](https://badge.fury.io/py/nanopq.svg)](https://badge.fury.io/py/nanopq)\n[![Downloads](https://pepy.tech/badge/nanopq)](https://pepy.tech/project/nanopq)\n\nNano Product Quantization (nanopq): a vanilla implementation of Product Quantization (PQ) and Optimized Product Quantization (OPQ) written in pure python without any third party dependencies.\n\n\n## Installing\nYou can install the package via pip. This library works with Python 3.5+ on linux.\n```\npip install nanopq\n```\n\n## [Documentation](https://nanopq.readthedocs.io/en/latest/index.html)\n- [Tutorial](https://nanopq.readthedocs.io/en/latest/source/tutorial.html)\n- [API](https://nanopq.readthedocs.io/en/latest/source/api.html)\n\n## Example\n\n```python\nimport nanopq\nimport numpy as np\n\nN, Nt, D = 10000, 2000, 128\nX = np.random.random((N, D)).astype(np.float32) # 10,000 128-dim vectors to be indexed\nXt = np.random.random((Nt, D)).astype(np.float32) # 2,000 128-dim vectors for training\nquery = np.random.random((D,)).astype(np.float32) # a 128-dim query vector\n\n# Instantiate with M=8 sub-spaces\npq = nanopq.PQ(M=8)\n\n# Train codewords\npq.fit(Xt)\n\n# Encode to PQ-codes\nX_code = pq.encode(X) # (10000, 8) with dtype=np.uint8\n\n# Results: create a distance table online, and compute Asymmetric Distance to each PQ-code \ndists = pq.dtable(query).adist(X_code) # (10000, ) \n```\n\n## Author\n- [Yusuke Matsui](http://yusukematsui.me)\n\n\n## Reference\n- [H. Jegou, M. Douze, and C. Schmid, \"Product Quantization for Nearest Neighbor Search\", IEEE TPAMI 2011](https://ieeexplore.ieee.org/document/5432202/) (the original paper of PQ)\n- [T. Ge, K. He, Q. Ke, and J. Sun, \"Optimized Product Quantization\", IEEE TPAMI 2014](https://ieeexplore.ieee.org/document/6678503/) (the original paper of OPQ)\n- [Y. Matsui, Y. Uchida, H. Jegou, and S. Satoh, \"A Survey of Product Quantization\", ITE MTA 2018](https://www.jstage.jst.go.jp/article/mta/6/1/6_2/_pdf/) (a survey paper of PQ) \n- [PQ in faiss](https://github.com/facebookresearch/faiss/wiki/Faiss-building-blocks:-clustering,-PCA,-quantization#pq-encoding--decoding) (Faiss contains an optimized implementation of PQ. [See the difference to ours here](https://nanopq.readthedocs.io/en/latest/source/tutorial.html#difference-from-pq-in-faiss))\n- [Rayuela.jl](https://github.com/una-dinosauria/Rayuela.jl) (Julia implementation of several encoding algorithms including PQ and OPQ)\n- [PQk-means](https://github.com/DwangoMediaVillage/pqkmeans) (clustering on PQ-codes. The implementation of nanopq is compatible to [that of PQk-means](https://github.com/DwangoMediaVillage/pqkmeans/blob/master/tutorial/1_pqkmeans.ipynb))\n- [Rii](https://github.com/matsui528/rii) (IVFPQ-based ANN algorithm using nanopq)\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/matsui528/nanopq", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "nanopq", "package_url": "https://pypi.org/project/nanopq/", "platform": "", "project_url": "https://pypi.org/project/nanopq/", "project_urls": { "Homepage": "https://github.com/matsui528/nanopq" }, "release_url": "https://pypi.org/project/nanopq/0.1.8/", "requires_dist": [ "numpy", "scipy" ], "requires_python": "", "summary": "Pure python implementation of product quantization for nearest neighbor search", "version": "0.1.8" }, "last_serial": 5577554, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "9431911fbb9d1158d1c216362a4fbede", "sha256": "a34103162f04259d2429d138be9ebf0cff791bc49c8c18140d6d54f627cc14e3" }, "downloads": -1, "filename": "nanopq-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "9431911fbb9d1158d1c216362a4fbede", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7169, "upload_time": "2018-07-23T16:33:16", "url": "https://files.pythonhosted.org/packages/56/06/63b07d3e867d8d5425b73e42ab37df5e30c9e3008fb37a6f8bb899621894/nanopq-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b339b59703549572b1853f54f3c65646", "sha256": "2172938cb311c19d798308b91ce71e84cf9f4cbf0928014b54bda32b3b14e197" }, "downloads": -1, "filename": "nanopq-0.1.0.tar.gz", "has_sig": false, "md5_digest": "b339b59703549572b1853f54f3c65646", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5983, "upload_time": "2018-07-23T16:33:17", "url": "https://files.pythonhosted.org/packages/09/89/0c131ae5351fb4fbb07cd1e3ba2a1a0f5eb5b27e1ae1faa63ffa8a25f580/nanopq-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "b9200e70f85e147df9215ade934def7b", "sha256": "30810864cb2730462503463fed20a71cab9088d8d004beb182a807e66a10a730" }, "downloads": -1, "filename": "nanopq-0.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "b9200e70f85e147df9215ade934def7b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7205, "upload_time": "2018-07-23T17:19:29", "url": "https://files.pythonhosted.org/packages/0c/30/64c722f50e1b2b53905f36d24abfc172e89387a58fb814adb760cc5d48e4/nanopq-0.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0640893b5e762c3fee3d3ddec073e79a", "sha256": "0c061b5b80ce255030b439b0b25d3445ebede24aaff175e47d77ec29646848ac" }, "downloads": -1, "filename": "nanopq-0.1.1.tar.gz", "has_sig": false, "md5_digest": "0640893b5e762c3fee3d3ddec073e79a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6040, "upload_time": "2018-07-23T17:19:30", "url": "https://files.pythonhosted.org/packages/f4/e9/a16c8cbc9578d0471d4f376b1141764e077c4cdc88905db9b0b2a840b463/nanopq-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "fcdefee45813610145d66e1560f638dc", "sha256": "074505ac868b42f8f0b9a0aad31f2cb294076c35b4c280ef2b63fd7d2371d578" }, "downloads": -1, "filename": "nanopq-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "fcdefee45813610145d66e1560f638dc", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7250, "upload_time": "2018-07-23T18:48:43", "url": "https://files.pythonhosted.org/packages/18/97/8ca7257f0574252db5cad618650215dbf7d5d73807040b422a50f69314db/nanopq-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9cd2484a49b749245da057be42189010", "sha256": "eb33c1c069c70304ef02955b9fb3b32ac974926b77e7e99ac87d215b49276c95" }, "downloads": -1, "filename": "nanopq-0.1.2.tar.gz", "has_sig": false, "md5_digest": "9cd2484a49b749245da057be42189010", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6185, "upload_time": "2018-07-23T18:48:44", "url": "https://files.pythonhosted.org/packages/cf/aa/b52c01bb42253a22df4ea218d05d38de27ba6b6091a42d8f27959c040d78/nanopq-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "277dba56e2357276f55349aab12f5601", "sha256": "dcdf6d45b622b0049b1af314301e0a2cb9b838b3b5d17cb0992a7bd9921b8479" }, "downloads": -1, "filename": "nanopq-0.1.3-py3-none-any.whl", "has_sig": false, "md5_digest": "277dba56e2357276f55349aab12f5601", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7525, "upload_time": "2018-07-24T07:52:56", "url": "https://files.pythonhosted.org/packages/7b/1d/4d2b857f525f422d86e90db3c960efd294d2209bb128d7249bc8c4bf8aec/nanopq-0.1.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f00fc23fa0c1bab1276da2ef732cd28e", "sha256": "d9532361b6c1f9c017d510c5b368718f8ada7bc8d6d8819b75305667b2e30cd2" }, "downloads": -1, "filename": "nanopq-0.1.3.tar.gz", "has_sig": false, "md5_digest": "f00fc23fa0c1bab1276da2ef732cd28e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6384, "upload_time": "2018-07-24T07:52:58", "url": "https://files.pythonhosted.org/packages/81/cd/8251a04c5b4f266833f2204e07f50e0e93f9b09b12f1e2b7cf3988dbc97b/nanopq-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "ae9c3ac78666e43dd5e89306115c40e7", "sha256": "bdc21f84479d14ced32d118413b10f3f1713be1b363afdd124391e9b78ba1f93" }, "downloads": -1, "filename": "nanopq-0.1.4-py3-none-any.whl", "has_sig": false, "md5_digest": "ae9c3ac78666e43dd5e89306115c40e7", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 6874, "upload_time": "2018-07-25T12:11:28", "url": "https://files.pythonhosted.org/packages/a0/b8/cdfc4daabac90cfbf5470b428baf455977557f7776111c817bb1e4f93a0d/nanopq-0.1.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0b6934212e8effc734f0a50164245385", "sha256": "06dae84cb7805e0cd6055373a203c06ca4a6539c95a9a8cf108a29b0ea3dee92" }, "downloads": -1, "filename": "nanopq-0.1.4.tar.gz", "has_sig": false, "md5_digest": "0b6934212e8effc734f0a50164245385", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6319, "upload_time": "2018-07-25T12:11:29", "url": "https://files.pythonhosted.org/packages/c7/28/5a42ecfdba370488daaed1df93f7362fda5581ca6e0bdb45aa0014cf3e7a/nanopq-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "91291b71f20e6a7afe4cc5b25194149d", "sha256": "da4251f3cad6e9baed5d6f4b731389bb8a758ce2471febb1fe2919b704d32c66" }, "downloads": -1, "filename": "nanopq-0.1.5-py3-none-any.whl", "has_sig": false, "md5_digest": "91291b71f20e6a7afe4cc5b25194149d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 6941, "upload_time": "2018-08-11T12:36:21", "url": "https://files.pythonhosted.org/packages/d0/00/68ad7c2124f490a133e15b593f03e7b6f825ee9e8e05e86d2c7f6d0818e1/nanopq-0.1.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "fec1dda573b0dba786db43254bcbc5ea", "sha256": "c179116907f43acdd9c5f78fefa1c181a790e7c718d10e6d31eae7a6fedc640e" }, "downloads": -1, "filename": "nanopq-0.1.5.tar.gz", "has_sig": false, "md5_digest": "fec1dda573b0dba786db43254bcbc5ea", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6377, "upload_time": "2018-08-11T12:36:23", "url": "https://files.pythonhosted.org/packages/3f/2c/c951f2040b10a47cf4318ac76cf4320f0343e12a5962de5c02dfb5272c0d/nanopq-0.1.5.tar.gz" } ], "0.1.6": [ { "comment_text": "", "digests": { "md5": "f6a2f9f435f65fb1e8d550537b75905d", "sha256": "f69d668b4017e26c6962bec8a9efa2ae1111b5974adbed14250bb0e9f06481a3" }, "downloads": -1, "filename": "nanopq-0.1.6-py3-none-any.whl", "has_sig": false, "md5_digest": "f6a2f9f435f65fb1e8d550537b75905d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7160, "upload_time": "2018-08-24T03:43:35", "url": "https://files.pythonhosted.org/packages/25/2b/afadc20d14bffe91560543826fee41c4d87ab540b2191a9d5dfdeb783304/nanopq-0.1.6-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a7deea2da39abcac715872c645d9d5aa", "sha256": "6f1feeb008760a3abff1953d4c023556eb3a708c0adcdce3fb0812efa8f0e047" }, "downloads": -1, "filename": "nanopq-0.1.6.tar.gz", "has_sig": false, "md5_digest": "a7deea2da39abcac715872c645d9d5aa", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6563, "upload_time": "2018-08-24T03:43:36", "url": "https://files.pythonhosted.org/packages/67/3b/978102fe4f5fe5f8a3a74504c54eb391d07a56217208dfb751c60f209c76/nanopq-0.1.6.tar.gz" } ], "0.1.7": [ { "comment_text": "", "digests": { "md5": "33d5e53543954b716d6c122e0dbcf4b9", "sha256": "b69ed35c56aaf300663e03d3722f04c65672c85c2acd08a35c02fc02cfefbc10" }, "downloads": -1, "filename": "nanopq-0.1.7-py3-none-any.whl", "has_sig": false, "md5_digest": "33d5e53543954b716d6c122e0dbcf4b9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9233, "upload_time": "2019-06-05T09:34:25", "url": "https://files.pythonhosted.org/packages/8f/4b/0e39d32c4de5ca8fc92bbf111ff2771d34e296815163adc7d1892af0c498/nanopq-0.1.7-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4f5f7eb7836132aee516d73ff9e3a6b7", "sha256": "0acd3031a01bb0b1f3163d92231b042a5412ed4c98b2b5aef8e7e48a2ee61a42" }, "downloads": -1, "filename": "nanopq-0.1.7.tar.gz", "has_sig": false, "md5_digest": "4f5f7eb7836132aee516d73ff9e3a6b7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7549, "upload_time": "2019-06-05T09:34:27", "url": "https://files.pythonhosted.org/packages/47/eb/3c8910a435c0e473da39398d17cf46156b1b31ab0c581f06e58f749ce6cd/nanopq-0.1.7.tar.gz" } ], "0.1.8": [ { "comment_text": "", "digests": { "md5": "a090970c07d2df98bad399923dc6d34e", "sha256": "fb82883be8146c4fd8149ed8f304d0a354c99c2be5f61af3cb53ff57039760ba" }, "downloads": -1, "filename": "nanopq-0.1.8-py3-none-any.whl", "has_sig": false, "md5_digest": "a090970c07d2df98bad399923dc6d34e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9235, "upload_time": "2019-07-24T13:16:46", "url": "https://files.pythonhosted.org/packages/d9/17/465feffcfcd84b0a800adcf4edfb2ede0b748a559154db6438f22625967e/nanopq-0.1.8-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1d75760d5c3f9f33dee138b1ad79e87e", "sha256": "a7f99bda8db95626b8937f1c9e59b231369a3f236347fe88eded94fc793af295" }, "downloads": -1, "filename": "nanopq-0.1.8.tar.gz", "has_sig": false, "md5_digest": "1d75760d5c3f9f33dee138b1ad79e87e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7591, "upload_time": "2019-07-24T13:16:47", "url": "https://files.pythonhosted.org/packages/f9/38/d53c342209bf0ba5304ee949152010830ca1f53e8c870559504a35bf6d52/nanopq-0.1.8.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "a090970c07d2df98bad399923dc6d34e", "sha256": "fb82883be8146c4fd8149ed8f304d0a354c99c2be5f61af3cb53ff57039760ba" }, "downloads": -1, "filename": "nanopq-0.1.8-py3-none-any.whl", "has_sig": false, "md5_digest": "a090970c07d2df98bad399923dc6d34e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9235, "upload_time": "2019-07-24T13:16:46", "url": "https://files.pythonhosted.org/packages/d9/17/465feffcfcd84b0a800adcf4edfb2ede0b748a559154db6438f22625967e/nanopq-0.1.8-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1d75760d5c3f9f33dee138b1ad79e87e", "sha256": "a7f99bda8db95626b8937f1c9e59b231369a3f236347fe88eded94fc793af295" }, "downloads": -1, "filename": "nanopq-0.1.8.tar.gz", "has_sig": false, "md5_digest": "1d75760d5c3f9f33dee138b1ad79e87e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7591, "upload_time": "2019-07-24T13:16:47", "url": "https://files.pythonhosted.org/packages/f9/38/d53c342209bf0ba5304ee949152010830ca1f53e8c870559504a35bf6d52/nanopq-0.1.8.tar.gz" } ] }