{ "info": { "author": "IBM", "author_email": "abdullah.alger@ibm.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Programming Language :: Python :: 3.6", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "# Convert CSV object files to Apache Parquet with IBM Cloud Object Storage\n\nThis tool was developed to help users on IBM Cloud convert their CSV objects in IBM Cloud Object Storage (COS) to Apache Parquet objects. It's developed using Python 3.6.6 and will work with Python 3 versions up to 3.6.6.\n\n### Installation\nTo install the tool, run `pip` with:\n\n```\npip install csvtoparquet\n```\n\nAfter the tool's installed, you must have an IBM Cloud API Key and IBM COS service to make the command line tool work. It requires that you insert your IBM Cloud API Key and a IBM COS service. You can find the API Key from your IBM Cloud management panel: **Manage > Security > Platform API Keys**. If you don't have IBM COS as a service, you can find it in the cloud **Catalog** under Object Storage, which has a lite tier (free).\n\nIf you already have the COS service, you'll need the name of the bucket where your CSV objects are located. Right now, the tool doesn't support multiple buckets, so you can't convert objects from one bucket and store them in another. Nonetheless, you can rename your converted objects to use prefixes such as:\n\n[object name] - `mycsvfile.csv`\n[renamed object stored as parquet] - `new/prefix/mycsvfile.parquet`\n\nThe file extension `.parquet` will be automatically added to your new object name.\n\n### Usage\n\nRun `csvtoparquet` on the command line using the following required flags:\n\n```\ncsvtoparquet -a -e -b \n```\n\n- `-a` or `--apikey` - IBM Cloud API Key\n- `-e` or `--endpoint` - COS bucket endpoint\n- `-b` or `--bucket` - COS bucket name where the CSV objects are stored\n\nAfter using the flag you can append the following flags to the command:\n\n- `-l` or `--list` - Lists all the objects in the bucket\n- `-c` or `--csv` - Lists all CSV objects in the bucket\n- `-cn` or `--csv-names` - Lists only the names of CSV objects in the bucket\n- `-f` or `--file` - Name of the CSV object you want to convert - used with `-n`\n- `-n` or `--name` - Name of the **new** object - can include prefixes - used with `-f`\n\n#### Converting objects\n\n##### Convert one object\n\nInput:\n\n```\ncsvtoparquet -a -e -b \\\n-f csvfile.csv -n csvfile\n```\n\nOutput:\n\n```\nNow Converting: csvfile.csv --> csvfile.parquet\n```\n\n##### Convert more than one object\n\nInput:\n\n```\ncsvtoparquet -a -e -b \\ \n-f csvfile.csv anothercsvfile.csv -n csvfile new/csvfile\n```\n\nOutput:\n\n```\nNow Converting: csvfile.csv --> csvfile.parquet\nNow Converting: anothercsvfile.csv --> new/csvfile.parquet\n```\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "", "keywords": "cloud csv parquet object_storage IBM", "license": "Apache 2.0", "maintainer": "", "maintainer_email": "", "name": "csvtoparquet", "package_url": "https://pypi.org/project/csvtoparquet/", "platform": "", "project_url": "https://pypi.org/project/csvtoparquet/", "project_urls": null, "release_url": "https://pypi.org/project/csvtoparquet/0.1.5/", "requires_dist": [ "pandas", "pyarrow", "ibm-cos-sdk", "simplejson" ], "requires_python": "", "summary": "", "version": "0.1.5" }, "last_serial": 4196595, "releases": { "0.1.2": [ { "comment_text": "", "digests": { "md5": "8fb53a69067c5989270cd597cec08f69", "sha256": "eb268cb91cfc3aae5b5b7ad9c79fefdfe588f7a6fd24e2abcb67bf71a8ebd40a" }, "downloads": -1, "filename": "csvtoparquet-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "8fb53a69067c5989270cd597cec08f69", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 12286, "upload_time": "2018-08-03T18:27:20", "url": "https://files.pythonhosted.org/packages/46/fb/daed8ffac9e526cea4e8907463de26999a0af680b0fdd1bc21d1ec109031/csvtoparquet-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ab76211c0ddf2719b40b6b37bbbafb1a", "sha256": "1d1a1a8d59844e4e4800d4d7aa893b1c34c5aa6f231af66e7223f921ca2e2415" }, "downloads": -1, "filename": "csvtoparquet-0.1.2.tar.gz", "has_sig": false, "md5_digest": "ab76211c0ddf2719b40b6b37bbbafb1a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5069, "upload_time": "2018-08-03T18:27:22", "url": "https://files.pythonhosted.org/packages/5d/5d/55713cbbad55901f95530e484bfe850bbb7b48117e864c9c3a88924d60a3/csvtoparquet-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "09f9b2ebab101365d4f16cd6cdc2f95f", "sha256": "33600a065609432bb42df3eaf39cd662c4e3184cf89e79f637c37393463b224d" }, "downloads": -1, "filename": "csvtoparquet-0.1.3-py3-none-any.whl", "has_sig": false, "md5_digest": "09f9b2ebab101365d4f16cd6cdc2f95f", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 13788, "upload_time": "2018-08-08T17:23:31", "url": "https://files.pythonhosted.org/packages/fd/7f/155af4c7e88c31e4ae5e34879880bd327ff85c466e047458a51792448971/csvtoparquet-0.1.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "476802cd65f9fdc38a4e17b71abdce9e", "sha256": "7ff9cce7083eecd8de2b6dedabb067ed92b15fc68fcfc4d8e5999b2d9137b4b7" }, "downloads": -1, "filename": "csvtoparquet-0.1.3.tar.gz", "has_sig": false, "md5_digest": "476802cd65f9fdc38a4e17b71abdce9e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6089, "upload_time": "2018-08-08T17:23:33", "url": "https://files.pythonhosted.org/packages/e6/c4/d71d53e68c2aa652c57169966e229f37958c527ffc23ee1a32c56a2b8165/csvtoparquet-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "4f5c0a1de09680b2130e6af6a60398e5", "sha256": "6c20bd24019005e0251af7fb7ac25d8b14ac427d22319ba2ca9c692d4db8728d" }, "downloads": -1, "filename": "csvtoparquet-0.1.4-py3-none-any.whl", "has_sig": false, "md5_digest": "4f5c0a1de09680b2130e6af6a60398e5", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 13885, "upload_time": "2018-08-14T07:37:34", "url": "https://files.pythonhosted.org/packages/bb/54/259875a454c9875e2a876c6afa84d9df2d4985ae00ca7560e557bfc5a5de/csvtoparquet-0.1.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "45b2ce0661b7d570f339550a074999ac", "sha256": "ca0b12d0dd14b251345fd9f5be10060e013fd29c5d1597c05bd63200286f23ee" }, "downloads": -1, "filename": "csvtoparquet-0.1.4.tar.gz", "has_sig": false, "md5_digest": "45b2ce0661b7d570f339550a074999ac", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6205, "upload_time": "2018-08-14T07:37:36", "url": "https://files.pythonhosted.org/packages/fd/a7/3d335a42f8f85724e90b7bc56cc7daa08cd2c1eda3ad4953c8feffdf57f4/csvtoparquet-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "3bee9c22d59c535758be1f8839605fd5", "sha256": "25a3b8d393843aa3cbb68e4a6d82482c527efce7752cffb0ce2c8fe5b85c3eac" }, "downloads": -1, "filename": "csvtoparquet-0.1.5-py3-none-any.whl", "has_sig": false, "md5_digest": "3bee9c22d59c535758be1f8839605fd5", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 13873, "upload_time": "2018-08-22T15:31:14", "url": "https://files.pythonhosted.org/packages/8a/16/5e9042359af822f66bc0c9c85d6c440bd08db428ff9aba987f89cba8ecee/csvtoparquet-0.1.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "baa8c334ed0b100d1a138d245e688ba2", "sha256": "b6d1a1a6037c1351f549339843140cd0bf814eab34fee91338d953ad073274a5" }, "downloads": -1, "filename": "csvtoparquet-0.1.5.tar.gz", "has_sig": false, "md5_digest": "baa8c334ed0b100d1a138d245e688ba2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6213, "upload_time": "2018-08-22T15:31:15", "url": "https://files.pythonhosted.org/packages/fc/24/e390619ed522ce8b1e7f10cbe710bc2fa6e6bbd5c54071a6dc5e78ce8d7c/csvtoparquet-0.1.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "3bee9c22d59c535758be1f8839605fd5", "sha256": "25a3b8d393843aa3cbb68e4a6d82482c527efce7752cffb0ce2c8fe5b85c3eac" }, "downloads": -1, "filename": "csvtoparquet-0.1.5-py3-none-any.whl", "has_sig": false, "md5_digest": "3bee9c22d59c535758be1f8839605fd5", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 13873, "upload_time": "2018-08-22T15:31:14", "url": "https://files.pythonhosted.org/packages/8a/16/5e9042359af822f66bc0c9c85d6c440bd08db428ff9aba987f89cba8ecee/csvtoparquet-0.1.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "baa8c334ed0b100d1a138d245e688ba2", "sha256": "b6d1a1a6037c1351f549339843140cd0bf814eab34fee91338d953ad073274a5" }, "downloads": -1, "filename": "csvtoparquet-0.1.5.tar.gz", "has_sig": false, "md5_digest": "baa8c334ed0b100d1a138d245e688ba2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6213, "upload_time": "2018-08-22T15:31:15", "url": "https://files.pythonhosted.org/packages/fc/24/e390619ed522ce8b1e7f10cbe710bc2fa6e6bbd5c54071a6dc5e78ce8d7c/csvtoparquet-0.1.5.tar.gz" } ] }