{ "info": { "author": "Jay Young", "author_email": "dev@yjmade.net", "bugtrack_url": null, "classifiers": [], "description": "# jlpipe\n\n## Introduction\nProject owner: Jay Young \n\n## Installation\n\n```bash\npip install jlpipe\n```\n\n## Usage\nfull commands list:\n\n* dynamojson: convert json from dynamodb format to normal\n* jsonkey: return json keys of each line\n* jsonallkeys: show statistics of keys of json lines\n* jsonselect: select fields from json\n* json2csv: convert json to csv\n* json2pgtext: convert json to postgres plain text then can use pgcopy to import to database\n* parquet2json: convert parquet file to json format\n* jsondecompress: decompress gzip compressed field\n* arrayunpack: similiar to postgres unnest, unnest a array to lines\n* pgcopy: similiar to pgfutter, but it use plain text to copy\n* parallel_split: similiar to `parallel --pipe`, start multiple worker ahead, and cycle each line of input and route to each worker, then forword workers output to its stdout, not garentee order.\n\n\n\n```bash\n$ cat *.json|parallel_split dynamojson|jsonselect data=. patent_id description=patent_description|jsondecompress description|json2pgtext -a|pgcopg all_patent\n\n$ cat *.json|jsonallkeys -l 10000 -r >/dev/null\n$ ls *.parquet|parquet2json|json2pgtext|pgcopg all_patent\n```\n\nMore usage reference to the command help text\n\n```bash\ncommand --help\n```", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/yjmade/jlpipe", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "jlpipe", "package_url": "https://pypi.org/project/jlpipe/", "platform": "", "project_url": "https://pypi.org/project/jlpipe/", "project_urls": { "Homepage": "https://github.com/yjmade/jlpipe" }, "release_url": "https://pypi.org/project/jlpipe/1.1.0/", "requires_dist": null, "requires_python": "", "summary": "jlpipe", "version": "1.1.0" }, "last_serial": 5642940, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "e5f7ecc335eb061b062070efceb99f15", "sha256": "99044cfe4c5da8d6ca5e9b5e713756674a6f85364552e5aa6b1e57c12172a371" }, "downloads": -1, "filename": "jlpipe-1.0.0.tar.gz", "has_sig": false, "md5_digest": "e5f7ecc335eb061b062070efceb99f15", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6556, "upload_time": "2019-07-26T09:53:49", "url": "https://files.pythonhosted.org/packages/71/3a/6db0581c065e89a8555bb7b385724d4a8c085719caa4ff16cf288e9963bc/jlpipe-1.0.0.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "1cd003cf446ff93701ea675b37bfd62b", "sha256": "f349fb14f590b88644d301a4ef6a28459604812fe656784ad546c35d657ab96f" }, "downloads": -1, "filename": "jlpipe-1.1.0.tar.gz", "has_sig": false, "md5_digest": "1cd003cf446ff93701ea675b37bfd62b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7846, "upload_time": "2019-08-07T03:21:48", "url": "https://files.pythonhosted.org/packages/a8/dd/a574e7daa42abc67c147d276dbdc46035e5b6b4e9b0da8ff11576e1faf55/jlpipe-1.1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1cd003cf446ff93701ea675b37bfd62b", "sha256": "f349fb14f590b88644d301a4ef6a28459604812fe656784ad546c35d657ab96f" }, "downloads": -1, "filename": "jlpipe-1.1.0.tar.gz", "has_sig": false, "md5_digest": "1cd003cf446ff93701ea675b37bfd62b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7846, "upload_time": "2019-08-07T03:21:48", "url": "https://files.pythonhosted.org/packages/a8/dd/a574e7daa42abc67c147d276dbdc46035e5b6b4e9b0da8ff11576e1faf55/jlpipe-1.1.0.tar.gz" } ] }