{ "info": { "author": "Oleg Pudeyev", "author_email": "oleg@bsdpower.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: System Administrators", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3", "Topic :: Database", "Topic :: System :: Archiving" ], "description": "redis-dump-load\n===============\n\n.. image:: https://api.travis-ci.org/p/redis-dump-load.png\n :target: https://travis-ci.org/p/redis-dump-load\n\nDumps Redis data sets into a format suitable for long-term storage\n(currently JSON) and loads data from such dump files back into Redis.\n\nFeatures\n--------\n\nredis-dump-load:\n\n- Supports all Redis data types;\n- Dumps TTL and expiration times;\n- Can load TTL OR original expiration time for expiring keys;\n- Can create pretty/human-readable dumps (keys dumped in sorted order,\n output indented);\n- Can stream data when dumping and loading;\n- Can be used as a module in a larger program or as a standalone utility;\n- Uses an output format compatible with redis-dump_.\n\nUsage\n-----\n\nredis-dump-load may be used as a module and also as a command line tool.\n\nModule Usage\n^^^^^^^^^^^^\n\nredis-dump-load exports a pickle_-like interface, namely ``load``,\n``loads``, ``dump`` and ``dumps`` functions. For example::\n\n import redisdl\n\n json_text = redisdl.dumps()\n\n with open('path/to/dump.json', 'w') as f:\n # streams data\n redisdl.dump(f)\n\n json_text = '...'\n redisdl.loads(json_text)\n\n with open('path/to/dump.json') as f:\n # streams data if ijson or jsaone are installed\n redisdl.load(f)\n\nSee the streaming section below for more information about streaming.\n\nDump and load methods accept options as keyword arguments::\n\n json_text = redisdl.dumps(encoding='iso-8859-1', pretty=True)\n\nThe arguments should always be passed in as keywords, i.e, do not rely\non the order in which the parameters are currently listed.\nOptions take string values unless otherwise noted. The options are as follows:\n\n- ``host``: host name or IP address for redis server\n- ``port``: port number for redis server\n- ``unix_socket_path``: connect to redis via a Unix socket instead of TCP/IP;\n specify the path to the socket\n- ``password``: specify password to connect to redis\n- ``db`` (integer): redis database to connect to\n- ``encoding``: encoding to use for encoding or decoding the data, see\n Unicode section below\n- ``pretty`` (boolean, dump only): produce a pretty-printed JSON which is\n easier to read; currently this makes ``dump`` load entire data set into\n memory rather than stream it\n- ``keys`` (dump only): only dump keys matching specified pattern\n- ``use_expireat`` (boolean, load only): use ``expireat`` in preference to ``ttl`` when loading expiring keys\n- ``empty`` (boolean, load only): empty the redis data set before loading the\n data\n- ``streaming_backend`` (string): streaming backend to use when loading via\n ``load`` method, if ijson_ or jsaone_ is installed and streaming is thus used\n\nCommand Line Usage\n^^^^^^^^^^^^^^^^^^\n\n``redisdl.py`` can be used as a command line tool as follows::\n\n # dump database 0\n ./redisdl.py > dump.json\n ./redisdl.py -o dump.json\n\n # load into database 0\n ./redisdl.py -l < dump.json\n ./redisdl.py -l dump.json\n\nFor convenience, ``redisdl.py`` can be hard or soft linked as follows::\n\n ln redisdl.py redis-dump\n ln redisdl.py redis-load\n\nNow it can be used thusly::\n\n # dump database 0\n ./redis-dump > dump.json\n ./redis-dump -o dump.json\n\n # load into database 0\n ./redis-load < dump.json\n ./redis-load dump.json\n\nSymlinks work as well. \"load\" in the executable name triggers the loading\nmode, \"dump\" triggers the dumping mode, otherwise the default is to dump\nand ``-l`` option switches into the loading mode.\n\nAll options supported by the module API are accepted when redisdl is invoked\nas a command line tool. The command line options are:\n\n- ``-h``/``--help``: help text\n- ``-H HOST``/``--host HOST``: specify redis host\n- ``-p PORT``/``--port PORT``: specify redis port\n- ``-s SOCKET_PATH``/``--socket SOCKET_PATH``: connect to Unix socket at\n the specified path\n- ``-w PASSWORD``/``--password PASSWORD``: password to use when connecting to redis\n- ``-d DATABASE``/``--db DATABASE``: redis database to connect to (integer)\n- ``-k PATTERN``/``--keys PATTERN`` (dumping only): dump only keys matching specified glob-style pattern\n- ``-E ENCODING``/``-encoding ENCODING``: specify encoding to use\n- ``-o PATH``/``--output PATH``: write dump to PATH rather than standard output\n- ``-y``/``--pretty`` (dumping only): pretty-print JSON\n- ``-A``/``--use-expireat`` (loading only): use ``expireat`` rather than ``ttl`` values in the dump\n- ``-e``/``--empty`` (loading only): empty redis data set before loading\n- ``-B BACKEND``/``--backend BACKEND`` (loading only): streaming backend to use\n\nStreaming\n---------\n\n``dump`` will stream data unless ``pretty`` is given and ``True``.\n\n``load`` will stream data if ijson_ or jsaone_ is installed. To determine whether\nredis-dump-load supports streaming data load, examine\n``redisdl.have_streaming_load`` variable. There are also\n``redisdl.have_ijson`` and ``redisdl.have_jsaone`` variables indicating\npresence of the respective library.\n\nredis-dump-load prefers ijson over jsaone and does not specify a backend for\nijson by default, which as of this writing means that ijson's pure Python\nbackend will be used. To request a specific backend either pass it as\nfollows to the load methods::\n\n redisdl.load(io, streaming_backend='ijson-yajl2')\n\n... or set the desired backend globally as follows::\n\n redisdl.streaming_backend = 'ijson-yajl2'\n\nThe backend argument takes form of \"library-library backend\", e.g.:\n- ``ijson`` selects the default backend of ijson, which currently is the pure Python one.\n- ``ijson-yajl2`` selects ijson with yajl2 backend.\n- ``yajl2`` means the same things as ``ijson-yajl2`` for compatibility with older redis-dump-load versions.\n- ``jsaone`` selects the jsaone backend.\n\nNote: Streaming loading is substantially slower than lump loading.\nTo force lump loading of files, read the files in memory and invoke ``loads``\nrather than ``load``.\n\njsaone support was added in redis-dump-load version 1.0.\n\nTTL, EXPIRE and EXPIREAT\n------------------------\n\nWhen dumping, redis-dump-load dumps the TTL values for expiring keys\nas well as calculated time when the keys will expire (``expireat``).\nAs Redis does not provide a command to retrieve absolute expiration time of\na key, the expiration time is calculated using the current time on the\n*client*'s system. As such, if the time on the client system is not in sync\nwith time on the system where the Redis server is running, ``expireat``\nvalues will be incorrect.\n\nWhen loading, redis-dump-load by default uses the TTL values in the dump\n(``ttl`` key) to set expiration times on the keys in preference to\n``expireat`` values. This will maintain the expiration times of the keys\nrelative to the dump/load time but will change the absolute expiration time\nof the keys. Using ``-A``/``--use-expireat`` command line option or\n``use_expireat`` parameter to module functions will make redis-dump-load\nuse ``expireat`` values in preference to ``ttl`` values, setting expiring\nkeys to expire at the same absolute time as they had before they were dumped\n(as long as system times are in sync on all machines involved).\n\nDumping and loading of TTL values and expiration times was added in\nredis-dump-load version 1.0.\n\nUnicode\n-------\n\nRedis operates on bytes and has no concept of Unicode or encodings.\nJSON operates on (Unicode) strings and cannot serialize binary data. Therefore,\nredis-dump-load has to encode Unicode strings into byte strings when\nloading data into Redis and decode byte strings into Unicode strings\nwhen dumping data from Redis.\nBy default redis-dump-load uses utf-8 for encoding data sent to Redis\nand decoding data received from Redis.\nThis behavior matches redis-py, whose default encoding is utf-8.\nA different encoding can be specified.\n\n``dumps`` returns strings, that is, instances of ``str`` on Python 2\nand instances of ``unicode`` on Python 3.\n\nWhen dumping to an IO object using ``dump``, and the IO object accepts\nbyte strings (such as when a file is opened in binary mode),\nredis-dump-load will ``.encode()`` the dumped data using the default\nencoding in effect.\n\nijson's yajl2 backend can only decode ``bytes`` instances, not ``str``.\nWhen loading data from a file opened in text mode and using ijson-yajl2,\nredis-dump-load will encode the file data using utf-8 encoding before\npassing the data to ijson. If this fails, try opening the file/stream in\nbinary mode.\n\njsaone can only decode text strings (``str`` instances), not ``bytes``.\nWhen loading data from a file opened in binary mode and using jsaone,\nredis-dump-load will decode the file data using the default encoding.\nIf this fails, you can change the default encoding or open the files in text\nmode with the encoding appropriately specified in the ``open()`` call.\n\nConcurrent Modifications\n------------------------\n\nredis-dump-load does not lock the entire data set it is dumping,\nbecause Redis does not provide a way to do so.\nAs a result, modifications to the data set made while a dump is in progress\naffect the contents of the dump.\n\nDependencies\n------------\n\n- redis-py_\n- ijson_ or jsaone_ (optional, for streaming load)\n- simplejson_ (Python 2.5 only)\n\nTests\n-----\n\nredis-dump-load has a test suite. To run it, install nose_ and run::\n\n nosetests\n\nThere are several tests that check for race conditions and as such take\na long time to run. To skip them, invoke nose thusly::\n\n nosetests -a '!slow'\n\nLicense\n-------\n\nReleased under the 2 clause BSD license.\n\n.. _redis-dump: https://github.com/delano/redis-dump\n.. _redis-py: https://github.com/andymccurdy/redis-py\n.. _simplejson: http://pypi.python.org/pypi/simplejson/\n.. _pickle: http://docs.python.org/library/pickle.html\n.. _nose: https://nose.readthedocs.org/en/latest/\n.. _ijson: https://pypi.python.org/pypi/ijson\n.. _jsaone: http://pietrobattiston.it/jsaone\n", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/p/redis-dump-load", "keywords": null, "license": "BSD", "maintainer": null, "maintainer_email": null, "name": "redis-dump-load", "package_url": "https://pypi.org/project/redis-dump-load/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/redis-dump-load/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.com/p/redis-dump-load" }, "release_url": "https://pypi.org/project/redis-dump-load/1.1/", "requires_dist": null, "requires_python": null, "summary": "Dump and load redis databases", "version": "1.1" }, "last_serial": 2334645, "releases": { "0.2.0": [ { "comment_text": "", "digests": { "md5": "0e07b47cbb6ed8587c5cfc86b311b429", "sha256": "7972ae453ac6f19abbdbbcb3bee95ba717b44a42d68b803d77f9997d0b1cffd8" }, "downloads": -1, "filename": "redis-dump-load-0.2.0.tar.gz", "has_sig": false, "md5_digest": "0e07b47cbb6ed8587c5cfc86b311b429", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2702, "upload_time": "2013-12-18T02:34:50", "url": "https://files.pythonhosted.org/packages/91/ed/21bad5fcf534f79ae92793e074ef044346e57c88a286dd687973cc3aa3c0/redis-dump-load-0.2.0.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "39f933a44772029f1d51236f3d25cdc5", "sha256": "8c7bcd16e6a1ed73f60ea0388c5f2d60dcee8909b44c7629251fd532a9d8de3f" }, "downloads": -1, "filename": "redis-dump-load-0.3.0.tar.gz", "has_sig": false, "md5_digest": "39f933a44772029f1d51236f3d25cdc5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6197, "upload_time": "2014-02-03T02:13:31", "url": "https://files.pythonhosted.org/packages/49/b6/140f12420037df4c15885872cf8345e4ef241c9ac92a50cd285e2209b91b/redis-dump-load-0.3.0.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "5bd937a1b5ebe9235b3f63662f12a696", "sha256": "c03b9ee8e50ba5bc549408838ed474825069e6d8a0df9e6a89531edda9f9c949" }, "downloads": -1, "filename": "redis-dump-load-0.4.0.tar.gz", "has_sig": false, "md5_digest": "5bd937a1b5ebe9235b3f63662f12a696", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7428, "upload_time": "2015-11-30T06:27:13", "url": "https://files.pythonhosted.org/packages/7b/ec/57efea5bb2fd7c31859b856989a7ef3c9209587faa63537d7f3d100dae1c/redis-dump-load-0.4.0.tar.gz" } ], "1.0": [ { "comment_text": "", "digests": { "md5": "425a9271254adad08e516c9ad832718f", "sha256": "fb20e7341d0405d6287f03385b4b395756873922799605e57afd657d5f84bbc9" }, "downloads": -1, "filename": "redis-dump-load-1.0.tar.gz", "has_sig": false, "md5_digest": "425a9271254adad08e516c9ad832718f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9595, "upload_time": "2016-08-27T02:03:11", "url": "https://files.pythonhosted.org/packages/27/68/3353ce288c578025137cd09e75e1552b38f6438a41665bf07eb359d1c31e/redis-dump-load-1.0.tar.gz" } ], "1.1": [ { "comment_text": "", "digests": { "md5": "967d0ce0812cb94f84777f133ffc1a52", "sha256": "af4832c4db849fe060abb2053d1652b6cd91ea115058a98c8348af906fe5b6c9" }, "downloads": -1, "filename": "redis-dump-load-1.1.tar.gz", "has_sig": false, "md5_digest": "967d0ce0812cb94f84777f133ffc1a52", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9780, "upload_time": "2016-09-10T02:12:18", "url": "https://files.pythonhosted.org/packages/7a/6b/aca5babcdf62980a2bfae7430302e7e40610be7ff1abe7fa0214d2a0c2e5/redis-dump-load-1.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "967d0ce0812cb94f84777f133ffc1a52", "sha256": "af4832c4db849fe060abb2053d1652b6cd91ea115058a98c8348af906fe5b6c9" }, "downloads": -1, "filename": "redis-dump-load-1.1.tar.gz", "has_sig": false, "md5_digest": "967d0ce0812cb94f84777f133ffc1a52", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9780, "upload_time": "2016-09-10T02:12:18", "url": "https://files.pythonhosted.org/packages/7a/6b/aca5babcdf62980a2bfae7430302e7e40610be7ff1abe7fa0214d2a0c2e5/redis-dump-load-1.1.tar.gz" } ] }