{ "info": { "author": "Andreas van Cranenburgh", "author_email": "A.W.van.Cranenburgh@rug.nl", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Science/Research", "License :: OSI Approved :: GNU General Public License (GPL)", "Operating System :: POSIX", "Programming Language :: Cython", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.3" ], "description": "Roaring Bitmap in Cython\n========================\n\nA roaring bitmap is an efficient compressed datastructure to store a set\nof integers. A Roaring bitmap stores a set of 32-bit integers in a series of\narrays and bitmaps, whichever takes the least space (which is always\n``2 ** 16`` bits or less).\n\nThis datastructure is useful for storing a large number of integers, e.g., for\nan inverted index used by search engines and databases. In particular, it is\npossible to quickly compute the intersection of a series of sets, which can be\nused to implement a query as the conjunction of subqueries.\n\nThis implementation is based on the Java and C implementations at\nhttps://github.com/lemire/RoaringBitmap\nand https://github.com/lemire/CRoaring\n\nAdditional features of this implementation:\n\n- Inverted list representation: blocks that are mostly full are stored\n compactly as an array of non-members (instead of as an array of members or a\n fixed-size bitmap).\n- Collections of immutable roaring bitmaps can be efficiently serialized with\n ``mmap`` in a single file.\n\nMissing features w.r.t. CRoaring:\n\n- Run-length encoded blocks\n- Various AVX2 / SSE optimizations\n\nSee also PyRoaringBitmap, a Python wrapper of CRoaring:\nhttps://github.com/Ezibenroc/PyRoaringBitMap\n\nLicense, requirements\n---------------------\nThe code is licensed under GNU GPL v2, or any later version at your option.\n\n- Python 2.7+/3.3+ http://www.python.org (headers required, e.g. python-dev package)\n- Cython 0.20+ http://www.cython.org\n\nInstallation, usage\n-------------------\n\n::\n\n $ git clone https://github.com/andreasvc/roaringbitmap.git\n $ cd roaringbitmap\n $ make\n\n(or ``make py2`` for Python 2)\n\nA ``RoaringBitmap()`` can be used as a replacement for a normal (mutable)\nPython set containing (unsigned) 32-bit integers:\n\n.. code-block:: python\n\n >>> from roaringbitmap import RoaringBitmap\n >>> RoaringBitmap(range(10)) & RoaringBitmap(range(5, 15))\n RoaringBitmap({5, 6, 7, 8, 9})\n\n``ImmutableRoaringBitmap`` is an immutable variant (analogous to ``frozenset``)\nwhich is stored compactly as a contiguous block of memory.\n\nA sequence of immutable RoaringBitmaps can be stored in a single file and\naccessed efficiently with ``mmap``, without needing to copy or deserialize:\n\n.. code-block:: python\n\n >>> from roaringbitmap import MultiRoaringBitmap\n >>> mrb = MultiRoaringBitmap([range(n, n + 5) for n in range(10)], filename='index')\n\n >>> mrb = MultiRoaringBitmap.fromfile('index')\n >>> mrb[5]\n ImmutableRoaringBitmap({5, 6, 7, 8, 9})\n\nFor API documentation cf. http://roaringbitmap.readthedocs.io\n\nBenchmarks\n----------\nOutput of ``$ make bench``::\n\n small sparse set\n 100 runs with sets of 200 random elements n s.t. 0 <= n < 40000\n set() RoaringBitmap() ratio\n init 0.000834 0.00138 0.603\n initsort 0.00085 0.000394 2.16\n and 0.00102 8.49e-05 12.1\n or 0.00171 0.000169 10.1\n xor 0.00152 0.000213 7.11\n sub 0.000934 0.000197 4.74\n iand 1.29e-05 2.97e-06 4.35\n ior 9.7e-06 3.26e-06 2.98\n ixor 8.98e-06 3.43e-06 2.62\n isub 6.83e-06 3.3e-06 2.07\n eq 0.000438 1.17e-05 37.6\n neq 6.37e-06 7.81e-06 0.816\n jaccard 0.0029 0.000126 23.1\n\n medium load factor\n 100 runs with sets of 59392 random elements n s.t. 0 <= n < 118784\n set() RoaringBitmap() ratio\n init 0.564 0.324 1.74\n initsort 0.696 0.273 2.55\n and 0.613 0.000418 1466\n or 0.976 0.000292 3344\n xor 0.955 0.000294 3250\n sub 0.346 0.000316 1092\n iand 0.00658 1.14e-05 575\n ior 0.00594 1.08e-05 548\n ixor 0.00434 1.12e-05 385\n isub 0.00431 1.09e-05 397\n eq 0.0991 0.000116 851\n neq 9.62e-06 1.29e-05 0.743\n jaccard 1.62 0.00025 6476\n\n dense set / high load factor\n 100 runs with sets of 39800 random elements n s.t. 0 <= n < 40000\n set() RoaringBitmap() ratio\n init 0.33 0.0775 4.26\n initsort 0.352 0.148 2.38\n and 0.24 0.000223 1078\n or 0.45 0.000165 2734\n xor 0.404 0.000161 2514\n sub 0.169 0.000173 973\n iand 0.00287 6.02e-06 477\n ior 0.00179 6.34e-06 282\n ixor 0.00195 5.53e-06 353\n isub 0.0017 6.35e-06 267\n eq 0.0486 4.65e-05 1045\n neq 1.01e-05 1.13e-05 0.888\n jaccard 0.722 0.000118 6136\n\nSee https://github.com/Ezibenroc/roaring_analysis/ for a performance comparison\nof PyRoaringBitmap and this library.\n\nReferences\n----------\n- http://roaringbitmap.org/\n- Chambi, S., Lemire, D., Kaser, O., & Godin, R. (2016). Better bitmap\n performance with Roaring bitmaps. Software: practice and experience, 46(5),\n pp. 709-719. http://arxiv.org/abs/1402.6407\n- The idea of using the inverted list representation is based on\n https://issues.apache.org/jira/browse/LUCENE-5983", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://roaringbitmap.readthedocs.io", "keywords": "", "license": "GPL", "maintainer": "", "maintainer_email": "", "name": "roaringbitmap", "package_url": "https://pypi.org/project/roaringbitmap/", "platform": "Many", "project_url": "https://pypi.org/project/roaringbitmap/", "project_urls": { "Homepage": "http://roaringbitmap.readthedocs.io" }, "release_url": "https://pypi.org/project/roaringbitmap/0.7/", "requires_dist": null, "requires_python": "", "summary": "Roaring Bitmap", "version": "0.7" }, "last_serial": 5286117, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "d91f34a59235834c5e26701c57211298", "sha256": "c6f0686761ca93c5078e0c1e775b1501e91d766c26be0c7860b6f386937ede51" }, "downloads": -1, "filename": "roaringbitmap-0.1.tar.gz", "has_sig": false, "md5_digest": "d91f34a59235834c5e26701c57211298", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22871, "upload_time": "2015-01-30T02:08:00", "url": "https://files.pythonhosted.org/packages/83/90/aa8c6918c69bfa0e74c337e20d968eb4a82583a9ecdf3de1af6fa2f7f7b3/roaringbitmap-0.1.tar.gz" } ], "0.2": [], "0.3": [ { "comment_text": "", "digests": { "md5": "060edc8b8996284adb0a5cc2ca3c3342", "sha256": "aac342239dacc0fa5661e63a68ecd3619cb3bf6f47257f68a975baa1555009c0" }, "downloads": -1, "filename": "roaringbitmap-0.3.0.tar.gz", "has_sig": false, "md5_digest": "060edc8b8996284adb0a5cc2ca3c3342", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 117684, "upload_time": "2015-08-11T12:26:37", "url": "https://files.pythonhosted.org/packages/57/82/e3cfc44fd7f9ad0f448db289e199bcbeb29a05cb9933ae3489441c9874b1/roaringbitmap-0.3.0.tar.gz" } ], "0.4": [ { "comment_text": "", "digests": { "md5": "32a4c865cb63844e58d7fcb1c681a26f", "sha256": "530d4f6954666ec03c35127c335e5aaf02f700ce462cf47239edf99970032e06" }, "downloads": -1, "filename": "roaringbitmap-0.4.tar.gz", "has_sig": false, "md5_digest": "32a4c865cb63844e58d7fcb1c681a26f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 217753, "upload_time": "2016-05-05T21:20:05", "url": "https://files.pythonhosted.org/packages/89/17/277b2fe0a3570650363fe395aa170f24a2a2d9327718153eeb6e01635076/roaringbitmap-0.4.tar.gz" } ], "0.5": [ { "comment_text": "", "digests": { "md5": "ba8c8835a57e64961c314196afa52758", "sha256": "10dc4eca3fe05104ea1ed381fa8dbc59c9718c59a2f8613ab2e3e3b3e16b16c7" }, "downloads": -1, "filename": "roaringbitmap-0.5.tar.gz", "has_sig": false, "md5_digest": "ba8c8835a57e64961c314196afa52758", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 224205, "upload_time": "2016-12-16T13:36:36", "url": "https://files.pythonhosted.org/packages/2c/12/7a38b7647f31065e47d093c2917d241750499229bd41cf029ea5ba3ab422/roaringbitmap-0.5.tar.gz" } ], "0.5.1": [ { "comment_text": "", "digests": { "md5": "8e7431281d821a99a60170ef0126f507", "sha256": "e34b768f293a243e5996a46cb7c2fe6575fd0634108ddeb5ac2e1efa1425ea9e" }, "downloads": -1, "filename": "roaringbitmap-0.5.1.tar.gz", "has_sig": false, "md5_digest": "8e7431281d821a99a60170ef0126f507", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 228538, "upload_time": "2017-03-07T09:49:55", "url": "https://files.pythonhosted.org/packages/c7/dd/dc8210f2f719279af7f4586d20962c4e4e1cc4eecadf7228f113889efbf5/roaringbitmap-0.5.1.tar.gz" } ], "0.5.2": [ { "comment_text": "", "digests": { "md5": "8112863f5c5f3cb1455f316d709203a2", "sha256": "42be64fc47026c5bc1d4d815a06c02c041def03e44127656ab5c9216a9537a9f" }, "downloads": -1, "filename": "roaringbitmap-0.5.2.tar.gz", "has_sig": false, "md5_digest": "8112863f5c5f3cb1455f316d709203a2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 200438, "upload_time": "2017-03-24T12:44:16", "url": "https://files.pythonhosted.org/packages/d4/87/c49d88da2f611173777188b6c42d9100c15bd115239a2e8d8408c584b43f/roaringbitmap-0.5.2.tar.gz" } ], "0.6": [ { "comment_text": "", "digests": { "md5": "d7139a969bd4b9cb502f9f840359542b", "sha256": "88a8cd3df557a60b582920afe340b0ce925e88b522d61486600a6d6c33a538a2" }, "downloads": -1, "filename": "roaringbitmap-0.6.tar.gz", "has_sig": false, "md5_digest": "d7139a969bd4b9cb502f9f840359542b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 226494, "upload_time": "2018-07-21T11:05:42", "url": "https://files.pythonhosted.org/packages/ca/bd/197af6f4c77013ad252f8afb59b884c8807beffe0621435baf3255612d4f/roaringbitmap-0.6.tar.gz" } ], "0.7": [ { "comment_text": "", "digests": { "md5": "899de2783f60239385a8468f6125fbf3", "sha256": "9b6fd44555eccd252bdbb3615105d9d6e9774f8ea508dc652d710c360a19bbe6" }, "downloads": -1, "filename": "roaringbitmap-0.7.tar.gz", "has_sig": false, "md5_digest": "899de2783f60239385a8468f6125fbf3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 224736, "upload_time": "2019-05-18T16:23:48", "url": "https://files.pythonhosted.org/packages/55/d8/1236c5fe75ba69f7f16009c0304d91e3e39f39470f168284ac406f35e5eb/roaringbitmap-0.7.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "899de2783f60239385a8468f6125fbf3", "sha256": "9b6fd44555eccd252bdbb3615105d9d6e9774f8ea508dc652d710c360a19bbe6" }, "downloads": -1, "filename": "roaringbitmap-0.7.tar.gz", "has_sig": false, "md5_digest": "899de2783f60239385a8468f6125fbf3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 224736, "upload_time": "2019-05-18T16:23:48", "url": "https://files.pythonhosted.org/packages/55/d8/1236c5fe75ba69f7f16009c0304d91e3e39f39470f168284ac406f35e5eb/roaringbitmap-0.7.tar.gz" } ] }