{ "info": { "author": "Yassine Azzouz", "author_email": "yassine.azzouz@agmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "Intended Audience :: System Administrators", "License :: OSI Approved :: MIT License", "Programming Language :: Python", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4" ], "description": "PyWHDFS\n==================================\n\nAPI and interactive command line interface for Web HDFS.\n\n```bash\n $ pywhdfs -c prod\n\n ---------------------------------------------------\n ------------------- PYWEBHDFS 1.0.0 ------------------\n -- Welcome to WEB HDFS interactive python shell. --\n -- The WEB HDFS client is available as `CLIENT`. --\n --------------------- Enjoy ! ---------------------\n ---------------------------------------------------\n\n\n >>> CLIENT.list(\"/\")\n [u'admin', u'data', u'group', u'solr', u'system', u'tmp', u'user']\n```\n\nFunctionalities\n--------\n\n* Python Library for interacting with WebHDFS and HTTFS Rest API\n* Support both secure (Kerberos,Token) and insecure clusters\n* Supports HA cluster and handle namenode failover\n* Supports HDFS federation with multiple nameservices and mount points.\n* Json format clusters configuration. \n* Command line interface to interactively interact with WebHDFS API\n on a python shell.\n* Support concurency on uploads and downloads.\n\n\nGetting started\n---------------\n\n```bash\n $ easy_install pywhdfs\n```\n\n\nConfiguration\n---------------\n\nPyWHDFS uses a json configuration file that define the connection parameters\nfor the different clusters.\nA simple configuration file looks like:\n\n```json\n {\n \"clusters\": [\n {\n \"name\": \"prod\",\n \"auth_mechanism\": \"GSSAPI\",\n \"verify\": false,\n \"truststore\": \"trust/store/path.jks\",\n \"nameservices\": [\n {\n \"urls\": [\"http://first_namenode_url:50070\" , \"http://second_namenode_url:50070\"],\n \"mounts\": [\"/\"]\n }\n ]\n }\n ]\n }\n```\n\nThe configuration file is validated against a [ schema file ](pywhdfs/resources/config_schema.json)\n\nThe default location of configuration file is \"~/.webhdfs.cfg\" but can can be overwritten using\nWEBHDFS_CONFIG environement variable.\n\nUSAGE\n-------\n\nThe interactive python shell client is the easiest way to use pywhdfs, but you can also instanciate\nthe client manually :\n\n```python \n >>>import pywhdfs.client as pywhdfs\n >>>CLIENT = pywhdfs.WebHDFSClient(nameservices=[{'urls':[ \"http://host1.hadoop.domain:50070\" , \"http://host2.hadoop.domain:50070\"],'mounts':['/']}], auth_mechanism=\"GSSAPI\", verify=False)\n >>>CLIENT.list(\"/\")\n```\n\nThe interacctive shell requires the connection parameters for the cluster to be setup in the configuration file,\nand the cluster name needs to match the name you pass as argument. \n\nContributing\n------------\n\nFeedback and Pull requests are very welcome!", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://bitbucket.org/suty/pyhdfs", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "pywhdfs", "package_url": "https://pypi.org/project/pywhdfs/", "platform": "", "project_url": "https://pypi.org/project/pywhdfs/", "project_urls": { "Homepage": "https://bitbucket.org/suty/pyhdfs" }, "release_url": "https://pypi.org/project/pywhdfs/1.0.3/", "requires_dist": null, "requires_python": "", "summary": "pywhdfs: python Web HDFS Client.", "version": "1.0.3" }, "last_serial": 2670354, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "d9b9aeaec460686da80d8d0fb8c46a5b", "sha256": "6d0b24bf706e3dde3909440394d8816e5d5322a0dca30253a2f932233e8b4870" }, "downloads": -1, "filename": "pywhdfs-1.0.0.tar.gz", "has_sig": false, "md5_digest": "d9b9aeaec460686da80d8d0fb8c46a5b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 35373, "upload_time": "2017-02-22T11:18:50", "url": "https://files.pythonhosted.org/packages/22/f9/103e462431f090d6b7d1bb9ccf59421906f618c9e1c5229f1d9956372dc5/pywhdfs-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "92f239d03aea2d4ba3640ddbeb05e789", "sha256": "45b2cbf2545e4abfbcc2957e85932decedd4a007ef80b645aec417ce84db013a" }, "downloads": -1, "filename": "pywhdfs-1.0.1.tar.gz", "has_sig": false, "md5_digest": "92f239d03aea2d4ba3640ddbeb05e789", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 36407, "upload_time": "2017-02-22T23:51:09", "url": "https://files.pythonhosted.org/packages/47/60/7dc70b2dc5eabf58bebf48d64a3dcf6ce29b230a3996bd34ce567572c012/pywhdfs-1.0.1.tar.gz" } ], "1.0.2": [ { "comment_text": "", "digests": { "md5": "28e37cda70883e42c89d2296ad8053ee", "sha256": "a280741c7f7b9b1191d7a82c06e9e5adc4d5f4455c7b14ce3ab18a34523dcb8c" }, "downloads": -1, "filename": "pywhdfs-1.0.2.tar.gz", "has_sig": false, "md5_digest": "28e37cda70883e42c89d2296ad8053ee", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 35505, "upload_time": "2017-02-26T21:57:26", "url": "https://files.pythonhosted.org/packages/71/35/1c7743a33fed4f5be43353822940b7423944a9dcc87100050a5034868d03/pywhdfs-1.0.2.tar.gz" } ], "1.0.3": [ { "comment_text": "", "digests": { "md5": "bf155d86b8295cd93a6c4e609b14cd24", "sha256": "c9706b55d2bd78579679608437ed4c0202af241299546fc8e036aef7f9036ae0" }, "downloads": -1, "filename": "pywhdfs-1.0.3.tar.gz", "has_sig": false, "md5_digest": "bf155d86b8295cd93a6c4e609b14cd24", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 36570, "upload_time": "2017-02-27T08:34:14", "url": "https://files.pythonhosted.org/packages/bc/5d/5f6ced647da2e89f7fd191babaf08b9f23522094e50c8d0327d59cbc065e/pywhdfs-1.0.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "bf155d86b8295cd93a6c4e609b14cd24", "sha256": "c9706b55d2bd78579679608437ed4c0202af241299546fc8e036aef7f9036ae0" }, "downloads": -1, "filename": "pywhdfs-1.0.3.tar.gz", "has_sig": false, "md5_digest": "bf155d86b8295cd93a6c4e609b14cd24", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 36570, "upload_time": "2017-02-27T08:34:14", "url": "https://files.pythonhosted.org/packages/bc/5d/5f6ced647da2e89f7fd191babaf08b9f23522094e50c8d0327d59cbc065e/pywhdfs-1.0.3.tar.gz" } ] }