{ "info": { "author": "Matthew Moisen", "author_email": "mmoisen@cisco.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: POSIX :: Linux", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: Implementation :: CPython", "Topic :: Database" ], "description": "pychbase\n========\n\nThis is a Python C wrapper for MapRDB and HBase using the `libhbase C\nAPI `__.\n\n``pychbase`` is modeled after the HappyBase API, but it does not use\n``thrift``, and is ideal for MapRDB.\n\n``pychbase`` is tested on Python 2.7 and MapR 5.1.\n\nLD\\_LIBRARY\\_PATH\n=================\n\nTo compile as well as import ``pychbase``, your ``LD_LIBRARY_PATH`` must\nhave the directory with ``libjvm.so`` on it, normally in either:\n\n- $JAVA\\_HOME/lib/amd64/server\n- $JAVA\\_HOME/jre/lib/amd64/server\n\nIf you are using this with MapR, you must also have ``/opt/mapr/lib`` on\nyour ``LD_LIBRARY_PATH``\n\nE.g,\n\n::\n\n export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/lib/amd64/server:/opt/mapr/lib\n\nInstallation\n============\n\nInstallation on a MapR environment\n----------------------------------\n\nNormally, the only environment variable to worry about on a MapR\nenvironment is $PYCHBASE\\_LIBJVM\\_DIR\n\n::\n\n export PYCHBASE_LIBJVM_DIR=/usr/lib/jvm/jre-1.7.0/lib/amd64/server\n virtualenv pychbase\n cd pychbase\n source bin/activate\n pip install pychbase\n\n # Or build it from source\n git clone https://github.com/mkmoisen/pychbase.git\n cd pychbase\n python setup.py install\n\nInstallation on a Non-MapR environment\n--------------------------------------\n\nPlease see the end of the readme for Cloudera intallation notes.\n\nRun the tests\n=============\n\nThe ``config.py`` file in the ``tests`` directory has two constants,\n``ZOOKEEPER`` and ``TABLE_NAME``, that probably won't work if you run\nthe tests without modification\n\nCreate a ``tests/local_config.py`` file like the following:\n\n::\n\n ZOOKEEPERS = 'localhost:7222'\n TABLE_NAME = 'testpychbase'\n\nTo run the tests, make sure to be in the ``tests`` directory, or else\nyou will face import problems:\n\n::\n\n cd tests\n python tests.py\n\nCurrently ``nosetests`` will not work without facing an import issue.\n\nUsage\n=====\n\nI have attempted to mimic the great HappyBase API as closely as\npossible.\n\nMake sure to set the LD\\_LIBRARY\\_PATH environment variable:\n\n::\n\n # MapR\n export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/jre/lib/amd64/server:/opt/mapr/lib\n\n # Non MapR\n export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/jvm/jre-1.7.0/lib/amd64/server::/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/native\n export HBASE_LIB_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/\n # I've only gotten it to work on CDH4. If you are on CDH5 you'll need to mess around with the classpath some more\n\nImports:\n\n::\n\n from pychbase import Connection, Table, Batch\n\nTo create a connection:\n\n::\n\n # On MapR, you don't need to specify the CLDBS/zookeepers:\n connection = Connection()\n\n # On Non-MapR environments, you'll need to specify the zookeepers:\n connection = Connection('zookeeper-one:2181\",zookeeper-two:2181\",zookeeper-three:2181\"')\n\nTo create and delete tables:\n\n::\n\n # Create a table named 'test_table' with a single column family named 'f' with no additional attributes\n connection.create_table('test_table', {'f': {}})\n connection.delete_table('test_table')\n\nTo get a table to operate on:\n\n::\n\n table = connection.table('test_table')\n\nTo put, delete, and get from a table:\n\n::\n\n # Put a row with the current timestamp\n table.put('rowkey1', {'f:foo': 'bar', 'f:hai': 'bai'})\n data = table.row('rowkey1')\n assert data == {'f:foo': 'bar', 'f:hai': 'bai'}\n\n # Get a row and only include a single column\n data = table.row('rowkey1', columns=('f:foo',))\n assert data == {'f:foo': 'bar'}\n\n # Delete the row\n table.delete('rowkey1')\n data = table.row('rowkey1')\n assert data == {}\n\n # Put a row with a given timestamp\n table.put('rowkey1', {'f:foo': 'bar'}, timestamp=1000)\n table.put('rowkey1', {'f:foo': 'BAR'}, timestamp=10000)\n\n # Get a row with a given timestamp and include its timestamp\n data = table.row('rowkey1', timestamp=1000, include_timestamp=True)\n assert data == {'f:foo': ('bar', 1000)}\n\nTo scan:\n\n::\n\n # Full table scan:\n for row, data in table.scan():\n pass\n\n # Scan with a start and stop:\n for row, data in table.scan('foo', 'bar'):\n pass\n\n # Scan with a row prefix:\n for row, data in table.scan(row_prefix='baz'): # E.g., start='baz', stop='baz~'\n pass\n\n # Scan with a filter: # Check out tests.py on how to use all of the filters\n for row, data in table.scan(filter=\"SingleColumnValueFilter('f', 'foo', =, 'binary:foo')\":\n pass\n\n # Scan a table but return only row keys, no rows:\n table.put('foo', {'f:foo': 'foo'}\n table.put('foo1', {'f:foo': 'foo'}\n table.put('foo2', {'f:foo': 'foo'}\n\n rows = list(table.scan(only_rowkeys=True))\n assert rows == ['foo', 'foo1', 'foo2']\n\nTo count the number of rows in a table:\n\n::\n\n # Full table count:\n count = table.count()\n\n # Count all rows with a start and stop\n count = table.count('foo', 'bar')\n\n # Count all rows whose row key starts with a row prefix:\n count = table.count(row_prefix='baz') # E.g., start='baz', stop='baz~'\n\n # Count all rows with a filter:\n count = table.count(filter=\"SingleColumnValueFilter('f', 'foo', =, 'binary:foo')\")\n\nTo batch put:\n\n::\n\n batch = table.batch()\n datas = [\n ('foo', 'a', 'b'),\n ('foo1', 'a1', 'b1'),\n ('foo2', 'a2', 'b2'),\n ]\n\n for data in datas:\n batch.put(data[0], {'f:foo': data[1], 'f:bar': data[2]})\n\n errors = batch.send()\n\n assert errors == 0\n\nTo batch delete:\n\n::\n\n batch = table.batch()\n rows = ['foo', 'foo1', 'foo2']\n for row in rows:\n batch.delete(row)\n\n errors = batch.send()\n\n assert errors == 0\n\nNote that ``batch.send()`` returns the number of errors that occurred,\nif any. It is up to the client to ignore this or raise an exception.\n\n::\n\n batch = table.batch()\n batch.put('foo', {'f:foo', 'bar'})\n batch.put('foo', 'invalid')\n\n errors = batch.send()\n assert errors == 1\n\nAn additional helper method is ``table.delete_prefix(row_prefix)``,\nwhich deletes all rows containing starting with the prefix.\n\n::\n\n table.put('foo', {'f:foo', 'foo'}\n table.put('foo1', {'f:foo', 'foo'}\n table.put('foo2', {'f:foo', 'foo'}\n table.put('foo3', {'f:foo', 'foo'}\n table.put('bar', {'f:bar', 'bar'}\n\n number_deleted = table.delete_prefix('foo')\n assert number_deleted == 3\n\n assert table.count() == 2\n\nNote that attempting to batch/put unescaped null terminators will result\nin them being stripped. Attempting to use a row key with an unescaped\nnull terminator will raise a TypeException. It is the user's duty to\nescape null terminators before attempting to batch/put data.\n\n::\n\n table.put('foo', {'f:foo\\0bar': 'baz\\0bak'})\n data = table.row('foo')\n assert data == {'f:foo': 'baz'}\n\n table.put('bar', {'f:foo\\\\0bar': 'baz\\00bak'})\n data = table.row('foo')\n assert data == {'f:foo\\\\0bar': 'baz\\00bak'}\n\nHappyBase compatibility\n=======================\n\nOne goal of this library is to maintain compatibility with the APIs in\nHappyBase.\n\nCheck out ``__init__.py`` to understand which features of HappyBase I\nhave not yet implemented.\n\nIn the future, I will force print warnings to stderr in the event a user\nuses an unimplemented feature.\n\nNon-MapR Installation and Environment Variables Guide\n=====================================================\n\nI have not tested ``pychbase`` heavily on Cloudera. I couldn't get it\nworking on CDH5 due to a classpath issue with ``libhbase``, and and\nwhile I was able to get it up and running with CDH4, some of the tests\nare failing. It seems to me that ``libhbase`` is not fully compatible\noutside of MapR.\n\n::\n\n export PYCHBASE_IS_MAPR=FALSE\n export PYCHBASE_LIBJVM_DIR=/usr/lib/jvm/jre-1.7.0/lib/amd64/server\n export PYCHBASE_INCLUDE_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/include\n export PYCHBASE_LIBRARY_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/native\n virtualenv pychbase\n cd pychbase\n source bin/activate\n pip install pychbase\n\n # Or build it from source\n git clone https://github.com/mkmoisen/pychbase.git\n cd pychbase\n python setup.py install\n\nPlease note that the following environment variables must be set in\norder to install ``pychbase`` correctly:\n\n- PYCHBASE\\_IS\\_MAPR\n- PYCHBASE\\_LIBJVM\\_DIR\n- PYCHBASE\\_INCLUDE\\_DIR\n- PYCHBASE\\_LIBRARY\\_DIR\n\nPYCHBASE\\_IS\\_MAPR\n------------------\n\nThis defaults to ``TRUE``. IF you are using Cloudera/etc, make sure to:\n\n::\n\n export PYCHBASE_IS_MAPR=FALSE\n\nPYCHBASE\\_LIBJVM\\_DIR\n---------------------\n\nThis is the directory that houses the ``libjvm.so`` file. Normally it is\nin either:\n\n- $JAVA\\_HOME/lib/amd64/server\n- $JAVA\\_HOME/jre/lib/amd64/server\n\nIf ``PYCHBASE_LIBJVM_DIR`` is not set, the installer will check if\n``JAVA_HOME`` has been set, and then try each of the above directories.\nIf ``JAVA_HOME`` is not set, it will attempt to default to\n``/usr/lib/jvm/jre-1.7.0/``.\n\nExample:\n\n::\n\n export PYCHBASE_LIBJVM_DIR=/usr/lib/jvm/jre-1.7.0/lib/amd64/server\n\nPYCHBASE\\_INCLUDE\\_DIR\n----------------------\n\nThis houses the ``/hbase/hbase.h`` and other ``libhbase`` C header\nfiles.\n\nIf ``PYCHBASE_IS_MAPR`` is true, this defaults to ``/opt/mapr/include``.\n\nFor Non-MapR environments, this must be set or the installation will\nfail.\n\nExample on Cloudera:\n\n::\n\n export PYCHBASE_INCLUDE_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/include\n\nPYCHBASE\\_LIBRARY\\_DIR\n----------------------\n\nThis houses either the ``libMapRClient.so`` file on MapR environments,\nor the ``libhbase.so`` file on Non-MapR environments.\n\nIf ``PYCHBASE_IS_MAPR`` is true, this defaults to ``/opt/mapr/lib``.\n\nFor Non-MapR environments, this must be set or the installation will\nfail.\n\nExample on Cloudera:\n\n::\n\n export PYCHBASE_LIBRARY_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/native\n\nAuthor\n======\n\n`Matthew Moisen `__\n\nLicense\n=======\n\nMIT", "description_content_type": null, "docs_url": null, "download_url": "https://github.com/mkmoisen/pychbase/tarball/0.1", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/mkmoisen/pychbase", "keywords": "hbase libhbase", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "pychbase", "package_url": "https://pypi.org/project/pychbase/", "platform": "", "project_url": "https://pypi.org/project/pychbase/", "project_urls": { "Download": "https://github.com/mkmoisen/pychbase/tarball/0.1", "Homepage": "https://github.com/mkmoisen/pychbase" }, "release_url": "https://pypi.org/project/pychbase/0.1.8/", "requires_dist": null, "requires_python": "", "summary": "A Python wrapper for the libhbase C API to HBase", "version": "0.1.8" }, "last_serial": 2668071, "releases": { "0.1.3": [ { "comment_text": "", "digests": { "md5": "026f9ba201f3128035a28bfc6b2d3ff0", "sha256": "25558601ee875f00389f4d70c2b06b8e7748bee289dc6cb31d1093b998576269" }, "downloads": -1, "filename": "pychbase-0.1.3.tar.gz", "has_sig": false, "md5_digest": "026f9ba201f3128035a28bfc6b2d3ff0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 23190, "upload_time": "2017-02-06T07:05:09", "url": "https://files.pythonhosted.org/packages/a6/90/212db3fdd2d6442b5bcbff7f271f6d0bb74fe83b2aaea4c583bc0061dc03/pychbase-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "dd662751a1381b213150f2d9f78d582c", "sha256": "6e778bdc8e328c80593ecf78a380b2e9a2f420d64bc312930c5e4e66ab29201a" }, "downloads": -1, "filename": "pychbase-0.1.4.tar.gz", "has_sig": false, "md5_digest": "dd662751a1381b213150f2d9f78d582c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 23106, "upload_time": "2017-02-06T20:23:36", "url": "https://files.pythonhosted.org/packages/e5/42/004c385ac84ffe7950969046a174cfe40c3a94399df023febbd55b3e7f04/pychbase-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "f09e6d99c374353ef7e8a1e20c9bbcc5", "sha256": "309236d89537e0c81f1f3f4727af3689a67dcb84ee57d73f70abb09eaa7b3ac1" }, "downloads": -1, "filename": "pychbase-0.1.5.tar.gz", "has_sig": false, "md5_digest": "f09e6d99c374353ef7e8a1e20c9bbcc5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 23315, "upload_time": "2017-02-07T02:14:29", "url": "https://files.pythonhosted.org/packages/7d/50/17dba2cd1d500f55e66e494c06c562c2a9ef155dae0c58b0380b9547c1e9/pychbase-0.1.5.tar.gz" } ], "0.1.6": [ { "comment_text": "", "digests": { "md5": "08c4ae010a7d6ac5b8dd38bd2f07e9a8", "sha256": "49a5795f1a99146a60c39e23a4f280571070191fee6d07f8a297fdc8d32dea0b" }, "downloads": -1, "filename": "pychbase-0.1.6.tar.gz", "has_sig": false, "md5_digest": "08c4ae010a7d6ac5b8dd38bd2f07e9a8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 30033, "upload_time": "2017-02-16T08:15:29", "url": "https://files.pythonhosted.org/packages/81/c3/d613ad11063d8b901d12cc13fae61500ee794d841047d002183f08d37874/pychbase-0.1.6.tar.gz" } ], "0.1.7": [ { "comment_text": "", "digests": { "md5": "0a10415a20d689458d5ab9f00229a2be", "sha256": "aa27d8a8e40b054c2667f259115057ea307863a0c3735d1ca3ac12fea3059a74" }, "downloads": -1, "filename": "pychbase-0.1.7.tar.gz", "has_sig": false, "md5_digest": "0a10415a20d689458d5ab9f00229a2be", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 30461, "upload_time": "2017-02-18T01:48:23", "url": "https://files.pythonhosted.org/packages/4b/b8/7fcd39d721f776f7cfda2717bf824439d87f0d4b35f9069f4105f2ab7816/pychbase-0.1.7.tar.gz" } ], "0.1.8": [ { "comment_text": "", "digests": { "md5": "911b883c9d35fd482b9674e2fa775026", "sha256": "482554510470e96672d07770744215ba982d200ea6fab73f482831dfc6b86717" }, "downloads": -1, "filename": "pychbase-0.1.8.tar.gz", "has_sig": false, "md5_digest": "911b883c9d35fd482b9674e2fa775026", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32629, "upload_time": "2017-02-25T20:39:26", "url": "https://files.pythonhosted.org/packages/eb/3d/d848a1b9363ff9614544b35affef5af10b020ecdfbd162511d2a2a507487/pychbase-0.1.8.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "911b883c9d35fd482b9674e2fa775026", "sha256": "482554510470e96672d07770744215ba982d200ea6fab73f482831dfc6b86717" }, "downloads": -1, "filename": "pychbase-0.1.8.tar.gz", "has_sig": false, "md5_digest": "911b883c9d35fd482b9674e2fa775026", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32629, "upload_time": "2017-02-25T20:39:26", "url": "https://files.pythonhosted.org/packages/eb/3d/d848a1b9363ff9614544b35affef5af10b020ecdfbd162511d2a2a507487/pychbase-0.1.8.tar.gz" } ] }