{ "info": { "author": "Stone Zhong", "author_email": "stonezhong@hotmail.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3" ], "description": "# dfs_tool\n\nIt is a HDFS cli tool. You can use it to manage your HDFS file system.\n\nIt calls the [WebHDFS](https://hadoop.apache.org/docs/r1.0.4/webhdfs.html) API.\n\n# Configuration\nYou need to put a config file. By default, the config file is at `~/.dfs_tool/config.json`, however you can change it's location by setting environment variable `DFS_TOOL_CFG`\n\nThe configuration looks like below:\n```json\n{\n \"api_base_uri\": \"https://my_hdfs_cluster.com/gateway/ui/webhdfs/v1/\",\n \"username\": \"superman\",\n \"password\": \"mypassword\",\n \"io_chunk_size\": 16777216\n}\n```\n\nIn some case, server uses certificate to authenticate client, then you can config like below:\n```json\n{\n \"api_base_uri\": \"https://my_hdfs_cluster.com/gateway/ui/webhdfs/v1/\",\n \"auth_cert\": \"/Users/shizhong/.dfs_tool/sso_client.cer\",\n \"auth_key\" : \"/Users/shizhong/.dfs_tool/sso_client.key\",\n \"ca_cert\" : \"/Users/shizhong/.dfs_tool/CombinedDigicertCA.cer\"\n}\n```\n\n* `api_base_url`: You need to put your WebHDFS endpoint here\n* `username`: You need to specify your HDFS account username\n* `password`: You need to specify your HDFS account password\n* `io_chunk_size`: optional, if not set, the default value is 1048576. It is the chunk size for downloading data from HDFS or uploading data to HDFS, you may want to bump this value if your bandwidth is high\n\n# Command supported\n```\ndfs_tool ls [-R] -- list directory or file\ndfs_tool download [-R] -- download file\ndfs_tool cat -- cat a file\ndfs_tool mkdir -- make a directory\ndfs_tool rm [-RM] -- remove a file or directory\ndfs_tool upload [-R] [-F] -- upload file\ndfs_tool mv -- move file or directory\n```\n\n# Options\nSome command support options, here are options:\n* `-R`\nIt means \"recursive\"\n\n* `-F`\nIt means \"force\", in upload command, when `-F` is specified, it will override the file already exist there.\n\n* `-M`\nIn \"rm\" command, you can specify a pattern to match the files you want to delete, for example:\n```\ndfs_tool rm -M \"/tmp/*.parquet\"\n```\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/stonezhong/dfs_tool", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "dfs-tool", "package_url": "https://pypi.org/project/dfs-tool/", "platform": "", "project_url": "https://pypi.org/project/dfs-tool/", "project_urls": { "Homepage": "https://github.com/stonezhong/dfs_tool" }, "release_url": "https://pypi.org/project/dfs-tool/0.0.10/", "requires_dist": [ "pywebhdfs" ], "requires_python": "", "summary": "Hadoop HDFS cli", "version": "0.0.10" }, "last_serial": 5882910, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "ce746035d2ca45ce807fc0c1a24a76d9", "sha256": "49a64e9a34c33bff11f8800dfef67c58175eaed5b6b6560685616ca4b6fdebe1" }, "downloads": -1, "filename": "dfs_tool-0.0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "ce746035d2ca45ce807fc0c1a24a76d9", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7087, "upload_time": "2019-07-15T06:25:00", "url": "https://files.pythonhosted.org/packages/bd/53/32a4f2fef28e0773a9eeaa195ffcc18fa5e2d00b8e2509fbf56719dafc11/dfs_tool-0.0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f48ddc73ce56681194f02c15b48eb6fc", "sha256": "41ea925b9c6cff38cb1484367ef0d3d859e6d4d9c730113bcc6ba7b06a4cfffe" }, "downloads": -1, "filename": "dfs_tool-0.0.1.tar.gz", "has_sig": false, "md5_digest": "f48ddc73ce56681194f02c15b48eb6fc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5130, "upload_time": "2019-07-15T06:25:02", "url": "https://files.pythonhosted.org/packages/a0/15/e4daa89f7dfb35ca657af7ff75b8cfe95ef5f7ee945e55db8aec8017c880/dfs_tool-0.0.1.tar.gz" } ], "0.0.10": [ { "comment_text": "", "digests": { "md5": "9354668c1562836464c3a41b2ab48250", "sha256": "8f131f4e944b7a0225d5e08282edb1a068efd25be741263f7725cb8def810b27" }, "downloads": -1, "filename": "dfs_tool-0.0.10-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9354668c1562836464c3a41b2ab48250", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8131, "upload_time": "2019-09-25T03:38:32", "url": "https://files.pythonhosted.org/packages/87/2e/1f6a7b71c8d266c68f8d2f8e50e653cbdb8b9e49394f4b907b731023d511/dfs_tool-0.0.10-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "96a41008a43462313ebca5bf1480f7f8", "sha256": "5ee8bdbbe3346939d99fd9591668096524839a2e04f1af344cb8ebac3d66888a" }, "downloads": -1, "filename": "dfs_tool-0.0.10.tar.gz", "has_sig": false, "md5_digest": "96a41008a43462313ebca5bf1480f7f8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6223, "upload_time": "2019-09-25T03:38:34", "url": "https://files.pythonhosted.org/packages/16/13/3e0cc46beb07ea3f386b1f4f0d72738e04a80bd0e20deedc3647f5cb9dae/dfs_tool-0.0.10.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "13f6ee8dc25c8b23b62ed23944b52d40", "sha256": "549452f21c4ca5ac4d01b2acbf6546ef7dc39828d9d041d56053e41778d0fbd1" }, "downloads": -1, "filename": "dfs_tool-0.0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "13f6ee8dc25c8b23b62ed23944b52d40", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7087, "upload_time": "2019-07-15T06:33:55", "url": "https://files.pythonhosted.org/packages/d0/d0/fca837688bde010de941cab2a657aaa80e30a87c1f2d4ceb519af5f69dfe/dfs_tool-0.0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "406fb46dad111f7ce9f01c65d55233d3", "sha256": "1a884b5b00ade8a9ed3a61027a98fd15b03a6fb5f060cf85b9de6cc64fb24e6e" }, "downloads": -1, "filename": "dfs_tool-0.0.2.tar.gz", "has_sig": false, "md5_digest": "406fb46dad111f7ce9f01c65d55233d3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5144, "upload_time": "2019-07-15T06:33:56", "url": "https://files.pythonhosted.org/packages/a9/36/082e7b8174af67cb94b66fce5b6173a01588cf8044c40c988dd8cbe14e3b/dfs_tool-0.0.2.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "a7a8f7fff957f202fcfa028d660e69a1", "sha256": "1009e4c78f7157ff5373ca057aaee80aecaffc2288f8fe29f669a21cb69aa2b3" }, "downloads": -1, "filename": "dfs_tool-0.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a7a8f7fff957f202fcfa028d660e69a1", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7771, "upload_time": "2019-09-06T00:03:13", "url": "https://files.pythonhosted.org/packages/74/24/41ff48e3955a5117d2a7a31e2bfd2034841f33d5cb5e2a5400f8ca3f5a43/dfs_tool-0.0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a873292d2a84a235f4c8f2c667148188", "sha256": "948e48cc1d7425854ea0fa9a20d880f014176ef0cdf226a81d3ad53fe7f82379" }, "downloads": -1, "filename": "dfs_tool-0.0.5.tar.gz", "has_sig": false, "md5_digest": "a873292d2a84a235f4c8f2c667148188", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5830, "upload_time": "2019-09-06T00:03:14", "url": "https://files.pythonhosted.org/packages/1a/21/6aa298df3fe554e572a6d728f83a012d85bcfea62a0ff3f45ea73b70a6d1/dfs_tool-0.0.5.tar.gz" } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "f7a72d980be8df3609e573b955d376d5", "sha256": "258c353e50d64d8fcd293449e24f9471b544379277db9c1beb0016cf05fbf626" }, "downloads": -1, "filename": "dfs_tool-0.0.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f7a72d980be8df3609e573b955d376d5", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7963, "upload_time": "2019-09-06T00:09:59", "url": "https://files.pythonhosted.org/packages/03/0b/7c40808c77cafeee883de9b7e8916770856dde78e21db8b0b5e2b8088987/dfs_tool-0.0.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "088ca169d9ef9f27d082ff0dfbd76746", "sha256": "572f30fc1329b3e796875ef2613f659aaf50820b561fd0b148cc2b458a2b393b" }, "downloads": -1, "filename": "dfs_tool-0.0.6.tar.gz", "has_sig": false, "md5_digest": "088ca169d9ef9f27d082ff0dfbd76746", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5996, "upload_time": "2019-09-06T00:10:01", "url": "https://files.pythonhosted.org/packages/11/53/23787dfc2c95f3b81b1aacbb5ea7b73b54c5c7c2894eed5ef8304474ed17/dfs_tool-0.0.6.tar.gz" } ], "0.0.7": [ { "comment_text": "", "digests": { "md5": "9df1e9d086a84042cc54124af5c7281d", "sha256": "cb9cb7bce809f1ebc9998d78854a931fd9b8eae2199601305189730556ef3c22" }, "downloads": -1, "filename": "dfs_tool-0.0.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9df1e9d086a84042cc54124af5c7281d", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7952, "upload_time": "2019-09-06T00:13:20", "url": "https://files.pythonhosted.org/packages/7b/f0/95fc59a2605624c99aa92ed80dfbd5db6274f2fa5665c2f825b1f9fca1bb/dfs_tool-0.0.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3dc91d9afbad5d6f38fbc0b87560cfae", "sha256": "51652dcac0bbeb41c829906b99e6e0bd79ce26a699804f088a00f26d69609094" }, "downloads": -1, "filename": "dfs_tool-0.0.7.tar.gz", "has_sig": false, "md5_digest": "3dc91d9afbad5d6f38fbc0b87560cfae", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5991, "upload_time": "2019-09-06T00:13:21", "url": "https://files.pythonhosted.org/packages/a7/99/89f5ec4c77c1e1d800d935320be28218608fade9afc7ae29d7a4d8f9f286/dfs_tool-0.0.7.tar.gz" } ], "0.0.8": [ { "comment_text": "", "digests": { "md5": "6334abaf1e70a465ca37391798cffe4b", "sha256": "d525b4a0bf819062711998ecfc5c88633d8712608e7c29e338629eb7abf1e003" }, "downloads": -1, "filename": "dfs_tool-0.0.8-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "6334abaf1e70a465ca37391798cffe4b", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7977, "upload_time": "2019-09-06T06:37:31", "url": "https://files.pythonhosted.org/packages/ae/92/3ec1738769c191dc6cac4f73b1fc70a110d3965ab23cd4ed47ba0f04feaa/dfs_tool-0.0.8-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ea1c1eebb8f720a53b3c1c3e9f00d8bd", "sha256": "3f2396a372b01df5624c190a411c91774bcfe79213b5999c9408abf2e8e2ba7b" }, "downloads": -1, "filename": "dfs_tool-0.0.8.tar.gz", "has_sig": false, "md5_digest": "ea1c1eebb8f720a53b3c1c3e9f00d8bd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6021, "upload_time": "2019-09-06T06:37:32", "url": "https://files.pythonhosted.org/packages/2e/11/2b47573501a2838787e42859721a59d2b9f45bcbfb252948c2b1d78c1c41/dfs_tool-0.0.8.tar.gz" } ], "0.0.9": [ { "comment_text": "", "digests": { "md5": "49982914377eb99478cccc13046c40a3", "sha256": "eae57eef581fcba574903ebb8806834d5540cc9d78826cea0e88bb13f25dc815" }, "downloads": -1, "filename": "dfs_tool-0.0.9-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "49982914377eb99478cccc13046c40a3", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7959, "upload_time": "2019-09-06T06:43:03", "url": "https://files.pythonhosted.org/packages/17/21/a4a30b18665dc0e828fa25540b6191e4e6fea9428db2e0870cbccf7b978a/dfs_tool-0.0.9-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b473aaa4e1e42ce403d208cee6a2b989", "sha256": "5fe58ac322f61c8feb0dea5243ea36ffcf8af7708fdde22ec6a9a72a4201ceb3" }, "downloads": -1, "filename": "dfs_tool-0.0.9.tar.gz", "has_sig": false, "md5_digest": "b473aaa4e1e42ce403d208cee6a2b989", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5996, "upload_time": "2019-09-06T06:43:05", "url": "https://files.pythonhosted.org/packages/5b/73/33bcabca757ee7e0703c0a4f4a6001d056583bd62ab3b792cd07ae3d0436/dfs_tool-0.0.9.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "9354668c1562836464c3a41b2ab48250", "sha256": "8f131f4e944b7a0225d5e08282edb1a068efd25be741263f7725cb8def810b27" }, "downloads": -1, "filename": "dfs_tool-0.0.10-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9354668c1562836464c3a41b2ab48250", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8131, "upload_time": "2019-09-25T03:38:32", "url": "https://files.pythonhosted.org/packages/87/2e/1f6a7b71c8d266c68f8d2f8e50e653cbdb8b9e49394f4b907b731023d511/dfs_tool-0.0.10-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "96a41008a43462313ebca5bf1480f7f8", "sha256": "5ee8bdbbe3346939d99fd9591668096524839a2e04f1af344cb8ebac3d66888a" }, "downloads": -1, "filename": "dfs_tool-0.0.10.tar.gz", "has_sig": false, "md5_digest": "96a41008a43462313ebca5bf1480f7f8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6223, "upload_time": "2019-09-25T03:38:34", "url": "https://files.pythonhosted.org/packages/16/13/3e0cc46beb07ea3f386b1f4f0d72738e04a80bd0e20deedc3647f5cb9dae/dfs_tool-0.0.10.tar.gz" } ] }