{ "info": { "author": "Daniel Povey, Meixu Song, Soroush Zargar, Mahsa Yarmohammadi, Jian Wu", "author_email": "dpovey@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Topic :: System :: Archiving :: Compression" ], "description": "# lilcom\n\n\nThis package lossily compresses floating-point NumPy arrays\ninto byte strings, with an accuracy specified by the user.\nThe main anticipated use is in machine learning applications, for\nstoring things like training data and models.\n\nThis package requires Python 3 and is not compatible with Python 2.\n\n## Installation with PyPi\n\nFrom PyPi you can install this with just\n```\npip3 install lilcom\n```\n\n### How to use\n\nThe most common usage pattern will be as follows (showing Python code):\n```\nimport numpy as np\nimport lilcom\n\na = np.random.randn(300,500)\na_compressed = lilcom.compress(a)\n# a_compressed is of type `bytes`, a byte string.\n# In this case it will use about 1.3 bytes per element.\n\n# decompress a\na_decompressed = lilcom.decompress(a_compressed)\n```\nThe compression is lossy so `a_decompressed` will not be exactly the same\nas `a`. The amount of error (absolute, not relative!) is determined by the\noptional `tick_power` argument to lilcom.compress() (default: -8), which is the\npower of 2 used for the step size between discretized values. The maximum error\nper element is 2**(tick_power-1), e.g. for tick_power=-8, it is 1/512.\n\n\n\n### Installation from Github\n\nTo install lilcom from github, first clone the repository;\n```\ngit clone git@github.com:danpovey/lilcom.git\n```\nthen run setup with `install` argument.\n```\npython3 setup.py install\n```\n(you may need to add the `--user` flag if you don't have system privileges).\nYou need to make sure a C++ compiler is installed, e.g. g++ or clang.\nTo test it, you can then cd to `test` and run:\n\n```\npython3 test_lilcom.py\n```\n\n\n## Technical details\n\nThe algorithm regresses each element on the previous element (for a 1-d array)\nor, for general n-d arrays, it regresses on the previous elements along each of\nthe axes, i.e. we regress element `a[i,j]` on `a[i-1,j]` and `a[i,j-1]`. The\nregression coefficients are global and written as part of the header in the\nstring.\n\nThe elements are then integerized and the integers are compressed using\nan algorithm that gives good compression when successive elements tend to\nhave about the same magnitude (the number of bits we're transmitting\nvaries dynamically acccording to the magnitudes of the elements).\n\nThe core parts of the code are implemented in C++.", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/danpovey/lilcom", "keywords": "compression numpy", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "lilcom", "package_url": "https://pypi.org/project/lilcom/", "platform": "", "project_url": "https://pypi.org/project/lilcom/", "project_urls": { "Homepage": "https://github.com/danpovey/lilcom" }, "release_url": "https://pypi.org/project/lilcom/1.1.1/", "requires_dist": null, "requires_python": ">=3.5", "summary": "Lossy-compression utility for sequence data in NumPy", "version": "1.1.1", "yanked": false, "yanked_reason": null }, "last_serial": 8857297, "releases": { "0.0.0": [ { "comment_text": "", "digests": { "md5": "b7d98a670d3f241011801e3e3895980b", "sha256": "46d278a1ec8c4e478a188a174c7578163e905200a2c2a5cfdcb59a64f02f0304" }, "downloads": -1, "filename": "lilcom-0.0.0-py3.7.egg", "has_sig": false, "md5_digest": "b7d98a670d3f241011801e3e3895980b", "packagetype": "bdist_egg", "python_version": "3.7", "requires_python": ">=3.5", "size": 13634, "upload_time": "2019-10-22T17:48:19", "upload_time_iso_8601": "2019-10-22T17:48:19.998888Z", "url": "https://files.pythonhosted.org/packages/6d/9c/f23f72582343f1a5eb47dd11416b21315e9327e778100a662371d3cfeffb/lilcom-0.0.0-py3.7.egg", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "5336f7796fc466835226fe3c245353ff", "sha256": "ae84590bf35385477b866de92f6c10c07e205a1584487d6af8debed1ebe88c0a" }, "downloads": -1, "filename": "lilcom-0.0.0-py3.7-macosx-10.14-x86_64.egg", "has_sig": false, "md5_digest": "5336f7796fc466835226fe3c245353ff", "packagetype": "bdist_egg", "python_version": "3.7", "requires_python": ">=3.5", "size": 23848, "upload_time": "2019-10-22T17:48:17", "upload_time_iso_8601": "2019-10-22T17:48:17.564830Z", "url": "https://files.pythonhosted.org/packages/17/7f/7e7a41668c61c41b2e4bea5fe340237ed01a1d07e8515e30d9a9dd26dd86/lilcom-0.0.0-py3.7-macosx-10.14-x86_64.egg", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "5ef52b2b7f61dfd10e2fff6cebd25b5a", "sha256": "f5bf14fc8671bced4ca98c73cb5f42ec521270bce47cd5f30cc73ff554d6f096" }, "downloads": -1, "filename": "lilcom-0.0.0.tar.gz", "has_sig": false, "md5_digest": "5ef52b2b7f61dfd10e2fff6cebd25b5a", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 52467, "upload_time": "2019-10-22T17:48:21", "upload_time_iso_8601": "2019-10-22T17:48:21.760848Z", "url": "https://files.pythonhosted.org/packages/74/be/05927e43b78b9ed156b3dff397a15c413510311d6d8756abb1729efeba65/lilcom-0.0.0.tar.gz", "yanked": false, "yanked_reason": null } ], "0.1.0": [ { "comment_text": "", "digests": { "md5": "dfd70fd2c1dcbf356988132a464eba60", "sha256": "65449b02955608889288e617a040d3b862d8e22dca38726f07159cabbee45f4d" }, "downloads": -1, "filename": "lilcom-0.1.0-py3.7-macosx-10.14-x86_64.egg", "has_sig": false, "md5_digest": "dfd70fd2c1dcbf356988132a464eba60", "packagetype": "bdist_egg", "python_version": "3.7", "requires_python": ">=3.5", "size": 21875, "upload_time": "2020-04-23T06:59:34", "upload_time_iso_8601": "2020-04-23T06:59:34.853802Z", "url": "https://files.pythonhosted.org/packages/61/9d/b46661f79e74aae7872040ca344312a4e0656b82f6431477331f6be933c2/lilcom-0.1.0-py3.7-macosx-10.14-x86_64.egg", "yanked": false, "yanked_reason": null } ], "1.0.0": [ { "comment_text": "", "digests": { "md5": "034aa92c5f5808fd3865e9601a7ddb81", "sha256": "43b41bbeb7ed9229436ca69200f011b5b85a92f90caec9ce867b8e05144ccc2a" }, "downloads": -1, "filename": "lilcom-1.0.0-py3.7-macosx-10.14-x86_64.egg", "has_sig": false, "md5_digest": "034aa92c5f5808fd3865e9601a7ddb81", "packagetype": "bdist_egg", "python_version": "3.7", "requires_python": ">=3.5", "size": 20922, "upload_time": "2020-04-23T06:59:36", "upload_time_iso_8601": "2020-04-23T06:59:36.111388Z", "url": "https://files.pythonhosted.org/packages/96/cf/6d4437cb9f58c8272dfe3e432bd32799cd13d8e505766d507182a4ab2da9/lilcom-1.0.0-py3.7-macosx-10.14-x86_64.egg", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "242b936168b707359ec7edf0c885624d", "sha256": "487f076783c11ed84d822250011310563ab9b5bc71314370549feee96e9aa8d2" }, "downloads": -1, "filename": "lilcom-1.0.0.tar.gz", "has_sig": false, "md5_digest": "242b936168b707359ec7edf0c885624d", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 24988, "upload_time": "2020-04-23T06:59:37", "upload_time_iso_8601": "2020-04-23T06:59:37.669592Z", "url": "https://files.pythonhosted.org/packages/01/5b/db6478a825d300a6a42beee6b8e31ed7ce48105cd73b87c73cce52aa8bd1/lilcom-1.0.0.tar.gz", "yanked": false, "yanked_reason": null } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "52e11a50425f3b7db1d9b949479bb38c", "sha256": "9b97b23565ed55a288fec6d9c2b0eeb6e737b4dfeb5cfdcce1eefccefd3ae3d1" }, "downloads": -1, "filename": "lilcom-1.1.0.tar.gz", "has_sig": false, "md5_digest": "52e11a50425f3b7db1d9b949479bb38c", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 30093, "upload_time": "2020-05-09T05:31:36", "upload_time_iso_8601": "2020-05-09T05:31:36.700588Z", "url": "https://files.pythonhosted.org/packages/8b/53/43f601d93e9f33da1c9d6909c1bf94390666726dbd2d18e76c377a5a1a9a/lilcom-1.1.0.tar.gz", "yanked": false, "yanked_reason": null } ], "1.1.1": [ { "comment_text": "", "digests": { "md5": "b5906351fe3bf9250c220b0d97b052e7", "sha256": "c0f114658d6408fd45267ca466fe1a5b09a3e4f0b5f902ef0ab6835bd98c3b01" }, "downloads": -1, "filename": "lilcom-1.1.1.tar.gz", "has_sig": false, "md5_digest": "b5906351fe3bf9250c220b0d97b052e7", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 30895, "upload_time": "2020-12-09T10:56:48", "upload_time_iso_8601": "2020-12-09T10:56:48.925250Z", "url": "https://files.pythonhosted.org/packages/46/78/9605e17af46a789747544c66f69bcfba17d43ae12f7caeb82f10bbd3fbd5/lilcom-1.1.1.tar.gz", "yanked": false, "yanked_reason": null } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "b5906351fe3bf9250c220b0d97b052e7", "sha256": "c0f114658d6408fd45267ca466fe1a5b09a3e4f0b5f902ef0ab6835bd98c3b01" }, "downloads": -1, "filename": "lilcom-1.1.1.tar.gz", "has_sig": false, "md5_digest": "b5906351fe3bf9250c220b0d97b052e7", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 30895, "upload_time": "2020-12-09T10:56:48", "upload_time_iso_8601": "2020-12-09T10:56:48.925250Z", "url": "https://files.pythonhosted.org/packages/46/78/9605e17af46a789747544c66f69bcfba17d43ae12f7caeb82f10bbd3fbd5/lilcom-1.1.1.tar.gz", "yanked": false, "yanked_reason": null } ], "vulnerabilities": [] }