{ "info": { "author": "Janis Lesinskis", "author_email": "janis@jaggedverge.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "Programming Language :: Python", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5" ], "description": "mmap_backed_array\n=================\n.. image:: https://travis-ci.org/JaggedVerge/mmap_backed_array.svg\n :target: https://travis-ci.org/JaggedVerge/mmap_backed_array\n\n.. image:: https://coveralls.io/repos/JaggedVerge/mmap_backed_array/badge.svg?branch=master&service=github\n :target: https://coveralls.io/github/JaggedVerge/mmap_backed_array?branch=master\n\n.. image:: https://badge.fury.io/py/mmap_backed_array.png\n :target: https://badge.fury.io/py/mmap_backed_array\n\nThis library lets you create arrays with mmap backing.\nThe interface is designed to be very similar to the array_ module in the standard library.\n\n.. _array: https://docs.python.org/3/library/array.html\n\nUse case\n--------\nThis library gives you the ability to have a shared memory space between different processes.\nThis can be helpful to work around some of the concurrency limitations present in some Python implementations.\n\nThis library was originally created to support the usage of very large precomputed lookup tables across multiple threads/processes.\nBecause of the large size of these tables creating copies for each process would have been very expensive in terms of memory usage.\nBy creating a mmap backing and accessing read-only multiple processes could read from the same memory hence leading to memory savings.\n\nThere's also another use case when doing interprocess communications, if you want a quick-and-dirty shared memory between\nprocesses in python and processes using other languages you can use this library to share a flat memory space between them.\n\nIf you just need some simple shared memory and don't want, or can't, bring in a more complicated dependency this might be what you need.\nFor more complicated concurrency tasks there may be more suitable libraries.\n\nUsage\n-----\n\nInstantiation:\n~~~~~~~~~~~~~~\n\nIf you don't provide a file for mmap backing an anonymous mmap is created to back the array.\n\n.. code:: python\n\n import mmap_backed_array\n arr = mmap_backed_array.mmaparray('I', [1, 2, 3, 4])\n\nYou can also provide a mmap file as backing.\n\n.. code:: python\n\n import mmap_backed_array\n import mmap\n with open(\"mmap_file\", 'rb') as fd:\n mmap_backing = mmap.mmap(\n fd.fileno(), 0, access=mmap.ACCESS_READ\n )\n arr = mmap_backed_array.mmaparray('I', mmap=mmap_backing)\n\nNote that this file can be shared with other processes, including ones\nthat are not python.\n\nAPI\n~~~\nThe API is designed to be as close to the standard library array_ module API as possible.\nUsing mmaparray's is designed to be similar to array.array.\nAnother goal is to make interoperating with array.array as easy as possible.\n\nMajor functionality including (but not limited) to ``append``, ``extend``, ``pop`` and ``tobytes`` is supported.\nThere may be some slight API incompatibilities currently, if there is anything substantial in the array API that is not implemented please open an issue.\n\nFor example:\n\n.. code:: python\n\n arr = mmaparray('I')\n >>> arr.append(1)\n >>> arr\n array('I', [1])\n >>> arr.extend([2,3,4])\n >>> arr\n array('I', [1, 2, 3, 4])\n\nYou can also use the standard library arrays easily with the mmap backed arrays:\n\n.. code:: python\n\n >>> from mmap_backed_array import mmaparray\n >>> mmap_array = mmaparray('I', (1, 1, 1, 1))\n >>> mmap_array\n array('I', [1, 1, 1, 1])\n >>> import array\n >>> mmap_array[2:4] = array.array('I', (2, 2))\n >>> mmap_array\n array('I', [1, 1, 2, 2])\n\n\nDue to the way in which we are storing direct to arrays, just like in the standard library array_ the typecodes must match up:\n\n.. code:: python\n\n >>> mmap_array.typecode\n 'I'\n >>> mmap_array[2:4] = array.array('b', (3, 3))\n Traceback (most recent call last):\n File \"\", line 1, in \n File \"/home/janis/mmap_backed_array/mmap_backed_array/mmap_array.py\", line 302, in __setitem__\n 'Can only assign array of same type to array slice'\n TypeError: Can only assign array of same type to array slice", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/JaggedVerge/mmap_backed_array", "keywords": "mmap array", "license": "GPLv3", "maintainer": "", "maintainer_email": "", "name": "mmap_backed_array", "package_url": "https://pypi.org/project/mmap_backed_array/", "platform": "", "project_url": "https://pypi.org/project/mmap_backed_array/", "project_urls": { "Homepage": "https://github.com/JaggedVerge/mmap_backed_array" }, "release_url": "https://pypi.org/project/mmap_backed_array/0.4.1/", "requires_dist": null, "requires_python": "", "summary": "Arrays with mmap backing", "version": "0.4.1" }, "last_serial": 2704377, "releases": { "0.3.0": [ { "comment_text": "", "digests": { "md5": "135128bec1dc59229cf9d4cfd58ff05b", "sha256": "f1991c8b7649816d336efffdd847df32f5c92f3d2647d1de474d82b8e720b32b" }, "downloads": -1, "filename": "mmap_backed_array-0.3.0.tar.gz", "has_sig": false, "md5_digest": "135128bec1dc59229cf9d4cfd58ff05b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10055, "upload_time": "2016-10-11T10:20:35", "url": "https://files.pythonhosted.org/packages/96/f3/3803a5c1696889b64395a99fe98dc8b773d9531b5ac59c7f69dc59007d25/mmap_backed_array-0.3.0.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "b7f293d017e9a71c29cd841b3f1b867b", "sha256": "b48d6f2f99fcee53c1b93f68487a47d30ca959f4a444ea4f3b57fe49e7c3e8a4" }, "downloads": -1, "filename": "mmap_backed_array-0.3.1.tar.gz", "has_sig": false, "md5_digest": "b7f293d017e9a71c29cd841b3f1b867b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10111, "upload_time": "2016-10-11T10:39:02", "url": "https://files.pythonhosted.org/packages/40/f5/f937a3427e0b79cd00d9478587806c58206677707ee147d1020fb65e3fd4/mmap_backed_array-0.3.1.tar.gz" } ], "0.4.1": [ { "comment_text": "", "digests": { "md5": "61b24b4c40daf51c1c31c879fd9be6cb", "sha256": "ec42ab195c42f11a692849576c2b308745eb35f468aa860e8423f097448cd513" }, "downloads": -1, "filename": "mmap_backed_array-0.4.1.tar.gz", "has_sig": false, "md5_digest": "61b24b4c40daf51c1c31c879fd9be6cb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8727, "upload_time": "2017-03-14T09:33:10", "url": "https://files.pythonhosted.org/packages/57/69/528863327fe03ba3c8db497f833b3e37bd6f03eac667b92d5150fabd5d78/mmap_backed_array-0.4.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "61b24b4c40daf51c1c31c879fd9be6cb", "sha256": "ec42ab195c42f11a692849576c2b308745eb35f468aa860e8423f097448cd513" }, "downloads": -1, "filename": "mmap_backed_array-0.4.1.tar.gz", "has_sig": false, "md5_digest": "61b24b4c40daf51c1c31c879fd9be6cb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8727, "upload_time": "2017-03-14T09:33:10", "url": "https://files.pythonhosted.org/packages/57/69/528863327fe03ba3c8db497f833b3e37bd6f03eac667b92d5150fabd5d78/mmap_backed_array-0.4.1.tar.gz" } ] }