{ "info": { "author": "Colin Dellow", "author_email": "cldellow@cldellow.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "# csv2parquet\n\n[![Build Status](https://travis-ci.org/cldellow/csv2parquet.svg?branch=master)](https://travis-ci.org/cldellow/csv2parquet)\n[![codecov](https://codecov.io/gh/cldellow/csv2parquet/branch/master/graph/badge.svg)](https://codecov.io/gh/cldellow/csv2parquet)\n\nConvert a CSV to a parquet file. You may also find [sqlite-parquet-vtable](https://github.com/cldellow/sqlite-parquet-vtable) useful.\n\n## Installing\n\nIf you just want to use the tool:\n\n```\nsudo pip install pyarrow csv2parquet\n```\n\nIf you want to clone the repo and work on the tool, install its dependencies via pipenv:\n\n```\npipenv install\n```\n\n## Usage\n\nNext, create some Parquet files. The tool supports CSV and TSV files.\n\n```\nusage: csv2parquet [-h] [-n ROWS] [-r ROW_GROUP_SIZE] [-o OUTPUT] [-c CODEC]\n [-i INCLUDE [INCLUDE ...] | -x EXCLUDE [EXCLUDE ...]]\n [-R RENAME [RENAME ...]] [-t TYPE [TYPE ...]]\n csv_file\n\npositional arguments:\n csv_file input file, can be CSV or TSV\n\noptional arguments:\n -h, --help show this help message and exit\n -n ROWS, --rows ROWS The number of rows to include, useful for testing.\n -r ROW_GROUP_SIZE, --row-group-size ROW_GROUP_SIZE\n The number of rows per row group.\n -o OUTPUT, --output OUTPUT\n The parquet file\n -c CODEC, --codec CODEC\n The compression codec to use (brotli, gzip, snappy,\n zstd, none)\n -i INCLUDE [INCLUDE ...], --include INCLUDE [INCLUDE ...]\n Include the given columns (by index or name)\n -x EXCLUDE [EXCLUDE ...], --exclude EXCLUDE [EXCLUDE ...]\n Exclude the given columns (by index or name)\n -R RENAME [RENAME ...], --rename RENAME [RENAME ...]\n Rename a column. Specify the column to be renamed and\n its new name, eg: 0=age or person_age=age\n -t TYPE [TYPE ...], --type TYPE [TYPE ...]\n Parse a column as a given type. Specify the column and\n its type, eg: 0=bool? or person_age=int8. Parse errors\n are fatal unless the type is followed by a question\n mark. Valid types are string (default), bool, int8,\n int16, int32, int64, float32, float64, timestamp\n```\n\n## Testing\n\n```\npylint csv2parquet\npytest\n```\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/cldellow/csv2parquet", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "csv2parquet", "package_url": "https://pypi.org/project/csv2parquet/", "platform": "", "project_url": "https://pypi.org/project/csv2parquet/", "project_urls": { "Homepage": "https://github.com/cldellow/csv2parquet" }, "release_url": "https://pypi.org/project/csv2parquet/0.0.6/", "requires_dist": null, "requires_python": "", "summary": "A tool to convert CSVs to Parquet files", "version": "0.0.6" }, "last_serial": 4252127, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "09ba5f11d3e91762fefcc2b4b5a68a0c", "sha256": "d3c4e56b0d5b37edf1372fe67963f4f8616ba7eb7c7bab0e843ed187af988401" }, "downloads": -1, "filename": "csv2parquet-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "09ba5f11d3e91762fefcc2b4b5a68a0c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 3419, "upload_time": "2018-07-08T15:23:48", "url": "https://files.pythonhosted.org/packages/e9/fd/5bf9b7922eb5aebc964182d80f6d14aabc27714d42dcde7d3df5553193c4/csv2parquet-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "eebf80e63c8b00e06726b7edbda54321", "sha256": "6ae87ec9d82de06ebf5c541f10cf1e2fbee4988cf185caed4aae7c8e678b8c84" }, "downloads": -1, "filename": "csv2parquet-0.0.1.tar.gz", "has_sig": false, "md5_digest": "eebf80e63c8b00e06726b7edbda54321", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2665, "upload_time": "2018-07-08T15:23:49", "url": "https://files.pythonhosted.org/packages/d9/b2/ee1046348d0cc742460fe14630435faf9a39b76e2cab9124d70dd7bc943b/csv2parquet-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "3a62af9e3bc4423ca8ccd1a1c4acae9b", "sha256": "3799e5c246a35b096a95acc2dfc4f9379d333659603ab671cea68d26beffcd02" }, "downloads": -1, "filename": "csv2parquet-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "3a62af9e3bc4423ca8ccd1a1c4acae9b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 3238, "upload_time": "2018-07-08T16:02:30", "url": "https://files.pythonhosted.org/packages/60/68/4e5279b64adf0e80ec76ca81decdc066d556092711a154e7902f8e61acce/csv2parquet-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ab7edbaa913772e8233aca15be809573", "sha256": "ac4d6a8c8ef53103af7fa52c9342ecefd5bd8b0ac9be67e6fe37c83c20f12b85" }, "downloads": -1, "filename": "csv2parquet-0.0.2.tar.gz", "has_sig": false, "md5_digest": "ab7edbaa913772e8233aca15be809573", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2381, "upload_time": "2018-07-08T16:02:32", "url": "https://files.pythonhosted.org/packages/2a/ac/b02de827bfed30b02dcdd6fbc77c3a5727ad7a8a7359a9cc93319ba07383/csv2parquet-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "4ea45f09375b99c9ad11598eb50052ad", "sha256": "79675ac65752f16321de267368e06a1a721ae1df1643c41bb498a7601ff9c756" }, "downloads": -1, "filename": "csv2parquet-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "4ea45f09375b99c9ad11598eb50052ad", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 4650, "upload_time": "2018-07-08T19:54:27", "url": "https://files.pythonhosted.org/packages/24/1c/ad0ee89fd632ecf177adb28509f96563478817afee4ae8df284e47fe9c49/csv2parquet-0.0.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2834b4dfe706bb492400f4135b9160dd", "sha256": "ad77caf0dfe0a65c7dff1df31521960cd3e31cc04d667c18115481ff5b811530" }, "downloads": -1, "filename": "csv2parquet-0.0.3.tar.gz", "has_sig": false, "md5_digest": "2834b4dfe706bb492400f4135b9160dd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3330, "upload_time": "2018-07-08T19:54:39", "url": "https://files.pythonhosted.org/packages/4c/ac/42b9eacc3246cf66c745bf4611e49a5edee611fc6773c0fc64e24e50465c/csv2parquet-0.0.3.tar.gz" } ], "0.0.4.post4": [ { "comment_text": "", "digests": { "md5": "c99df398e2f1fa7c51c3d28e170fcb4c", "sha256": "925ec7265240f6255678c30e0656ab44aeedf0523882268a6300fc74b232a7b1" }, "downloads": -1, "filename": "csv2parquet-0.0.4.post4-py3-none-any.whl", "has_sig": false, "md5_digest": "c99df398e2f1fa7c51c3d28e170fcb4c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 5906, "upload_time": "2018-07-09T01:07:47", "url": "https://files.pythonhosted.org/packages/e7/0f/7b8df3f4a1737077f6b6c939cec4a18e6d348ca079343b9b39a915b87a23/csv2parquet-0.0.4.post4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "96820d9f0b29df900db13bc973f4b804", "sha256": "429d2036f795b4316dd182dcf6f0dd02b0f1d7cf047f6a3113fe91816fdcdbbd" }, "downloads": -1, "filename": "csv2parquet-0.0.4.post4.tar.gz", "has_sig": false, "md5_digest": "96820d9f0b29df900db13bc973f4b804", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4505, "upload_time": "2018-07-09T01:07:48", "url": "https://files.pythonhosted.org/packages/0a/f7/7e00a9093f7c3ee592fc27ba05eca17a9efe755628271436fba845212ef2/csv2parquet-0.0.4.post4.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "67dc7dc424efbc5d6182175f1781fdce", "sha256": "7a1a83cea53159275eda27cb2fc53d7d5a8bf1853579efe972d32e48a0572ceb" }, "downloads": -1, "filename": "csv2parquet-0.0.5-py3-none-any.whl", "has_sig": false, "md5_digest": "67dc7dc424efbc5d6182175f1781fdce", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7717, "upload_time": "2018-07-09T03:55:27", "url": "https://files.pythonhosted.org/packages/6b/20/bc440aac3f7d099bd03fdb69ea89a118e39bc80ffe811e0048edc436a857/csv2parquet-0.0.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1bdfeb765bbaff257ceb64929a789146", "sha256": "cca14aaeafd453743e2c6a195041325c26b09872de7602f68d396c661518e635" }, "downloads": -1, "filename": "csv2parquet-0.0.5.tar.gz", "has_sig": false, "md5_digest": "1bdfeb765bbaff257ceb64929a789146", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6225, "upload_time": "2018-07-09T03:55:28", "url": "https://files.pythonhosted.org/packages/cd/fe/1dc51a591801e8a667e1d095e4f93b6b59fabd8b7b4df034093dca05159f/csv2parquet-0.0.5.tar.gz" } ], "0.0.5.post1": [ { "comment_text": "", "digests": { "md5": "0aa3fe731c90fcbf8514375e05deb6a3", "sha256": "155c44aaebcb4600d2b1fc3044a561ae52a812e1fc9ee2ade89be67923071cc4" }, "downloads": -1, "filename": "csv2parquet-0.0.5.post1-py3-none-any.whl", "has_sig": false, "md5_digest": "0aa3fe731c90fcbf8514375e05deb6a3", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7765, "upload_time": "2018-07-09T04:00:35", "url": "https://files.pythonhosted.org/packages/6f/1a/cc1dbefe921617b6a9bf8ea595640b9d111048f51f0f713f1dc0b2cb1528/csv2parquet-0.0.5.post1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "69785f0c86e3664332c757c00499eb75", "sha256": "bc8887bb33e1e92d66b142243daee89bcf30193698047691e3b911f64224fb37" }, "downloads": -1, "filename": "csv2parquet-0.0.5.post1.tar.gz", "has_sig": false, "md5_digest": "69785f0c86e3664332c757c00499eb75", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6212, "upload_time": "2018-07-09T04:00:36", "url": "https://files.pythonhosted.org/packages/f0/b4/1f8c687d71f58965d3977a56c69451c764a32dd9619c23bc99d047dd735c/csv2parquet-0.0.5.post1.tar.gz" } ], "0.0.5.post2": [ { "comment_text": "", "digests": { "md5": "6cb6b4327d7315885d7e8761303d7cf5", "sha256": "095105ce1ef10f36522ded595632afcc9189486c9d43a0c06f1e35c9d971bbfa" }, "downloads": -1, "filename": "csv2parquet-0.0.5.post2-py3-none-any.whl", "has_sig": false, "md5_digest": "6cb6b4327d7315885d7e8761303d7cf5", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7760, "upload_time": "2018-07-09T04:03:41", "url": "https://files.pythonhosted.org/packages/96/72/42a37399f4c121dd4f46af0acb1b00792c8ae66b111f86a4727befed6698/csv2parquet-0.0.5.post2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "568a8949157ea7af3e9beae47e2209f3", "sha256": "eac8b3eae4fd3e221cbd6d2eb19207d4b2f2809f3240057f1ab73a083d167613" }, "downloads": -1, "filename": "csv2parquet-0.0.5.post2.tar.gz", "has_sig": false, "md5_digest": "568a8949157ea7af3e9beae47e2209f3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6207, "upload_time": "2018-07-09T04:03:43", "url": "https://files.pythonhosted.org/packages/96/aa/d794e02752a1e950e07a95a2d1b5f0ddc7e3dbd8b2edaeafec0e70fd8853/csv2parquet-0.0.5.post2.tar.gz" } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "4d8479af9b399e4cd1215baea3ffebd0", "sha256": "88e86e32b7384c650cd0046fe7ca1981e2bb39b38f3050b440190be8f8a3c6fa" }, "downloads": -1, "filename": "csv2parquet-0.0.6-py3-none-any.whl", "has_sig": false, "md5_digest": "4d8479af9b399e4cd1215baea3ffebd0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7720, "upload_time": "2018-09-08T19:58:45", "url": "https://files.pythonhosted.org/packages/90/6f/12e802f1cf2ea3c0a170b074eb86212e2849996b4fd9e096385822175134/csv2parquet-0.0.6-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "899a007a9858c4a990642c075888d650", "sha256": "df661b170c86deca6af45afa17b4358a54eccd047688d6f93de3a81e4c0fb50e" }, "downloads": -1, "filename": "csv2parquet-0.0.6.tar.gz", "has_sig": false, "md5_digest": "899a007a9858c4a990642c075888d650", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6226, "upload_time": "2018-09-08T19:58:46", "url": "https://files.pythonhosted.org/packages/bc/cf/1de8113258809b7ab809a773b52267318d3e3d2904c34d2a838640b19e55/csv2parquet-0.0.6.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "4d8479af9b399e4cd1215baea3ffebd0", "sha256": "88e86e32b7384c650cd0046fe7ca1981e2bb39b38f3050b440190be8f8a3c6fa" }, "downloads": -1, "filename": "csv2parquet-0.0.6-py3-none-any.whl", "has_sig": false, "md5_digest": "4d8479af9b399e4cd1215baea3ffebd0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7720, "upload_time": "2018-09-08T19:58:45", "url": "https://files.pythonhosted.org/packages/90/6f/12e802f1cf2ea3c0a170b074eb86212e2849996b4fd9e096385822175134/csv2parquet-0.0.6-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "899a007a9858c4a990642c075888d650", "sha256": "df661b170c86deca6af45afa17b4358a54eccd047688d6f93de3a81e4c0fb50e" }, "downloads": -1, "filename": "csv2parquet-0.0.6.tar.gz", "has_sig": false, "md5_digest": "899a007a9858c4a990642c075888d650", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6226, "upload_time": "2018-09-08T19:58:46", "url": "https://files.pythonhosted.org/packages/bc/cf/1de8113258809b7ab809a773b52267318d3e3d2904c34d2a838640b19e55/csv2parquet-0.0.6.tar.gz" } ] }