{ "info": { "author": "metadeta96", "author_email": "metadeta96@gmail.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "# PyBite\n\nChunk by chunk iteration made easier\n\n## Installation\n\n pip install pybite\n\n## Methods\n\n### iterate_by\n\nReturn a iterator of chunks for the iterable\n\n#### Parameters\n\n**iterable**: *iter*\n\nAny iterable data e.g. list, tuple, dict, iter, ...\n\n**chunk_size**: *int*\n\nThe size of each chunk\n\n**map**: *callable* optional, defaults *None*\n\nA map function for transform the data before dividing in chunks\n\n**persist_header**: *bool* optional, defaults *False*\n\nWhether to persist a header on each chunk or not. \nIf persist_header is True and header is None, \nthe firt line will be read as the actual header.\n\n**header**: *str* optional, defaults *None*\n\nA header to be written at the start of each file.\nIf persist_header is True and header is None, \nthe firt line will be read as the actual header.\n\n#### Returns\n\n**iter**\n\nNew iterable for the chunked data\n\n#### Examples\n\n```python\n>>> iterate_by([1, 2, 3, 4, 5], 2)\niter([[1, 2], [3, 4], [5]])\n>>> iterate_by([1, 2, 3, 4, 5], 2, map=lambda x: x * 2)\niter([[2, 4], [6, 8], [10]])\n>>> iterate_by([\"numbers\", 1, 2, 3, 4, 5], 2, persist_header=True)\niter([[\"numbers\", 1, 2], [\"numbers\", 3, 4], [\"numbers\", 5]])\n```\n\n### iterate_file_by_lines\n\nReturn a iterator of file lines.\n\nEach line is read on-demand untill there is no more line to\nbe read.\n\n#### Parameters\n\n**iterable**: *iter*\n\nAny iterable data e.g. list, tuple, dict, iter, ...\n\n**file_stream** : *str*, *io.StringIO*\n\nA file path or io.StringIO instance to the file to be read.\n\n**encoding** : *str* optional, defaults *None*\n\nThe input file encoding. \nThe chunks will be saved using the same encoding.\n\n**strip_end** : *bool* optional, defaults *False*\n\nFlag for strip the end of each line.\nSame as calling `line.rstrip()`.\n\n#### Returns\n\n**iter**\n\nNew iterable for the chunked data\n\n#### Examples\n\n```python\nwith open(\"test.txt\", \"w\", encoding=\"utf-8\") as f:\n f.write(\"Symbols\\nAyp\\nBx\\nCC\\nDt\")\n\n>>> iterate_file_by_lines(\"test.txt\", encoding=\"utf-8\")\niter([\"Symbols\\n\", \"Ayp\\n\", \"Bx\\n\", \"CC\\n\", \"Dt\"])\n>>> iterate_file_by_lines(\"test.txt\", encoding=\"utf-8\", strip_end=True)\niter([\"Symbols\", \"Ayp\", \"Bx\", \"CC\", \"Dt\"])\n```\n\n### split_file_by_lines\n\nSplit a file into multiple files and store them in the output path.\n\nThe file is divided b lines into chunk files that later can be read individually\nor joined back.\n\n#### Parameters\n\n**file_stream**: *str*, *io.StringIO*\n\nA file path or io.StringIO instance to the file you want to split.\n\n**output_path**: *str*\n\nThe path to the directory where the chunks will be stored.\n\n**lines**: *int*\n\nThe quantity of lines on each chunk file. \nIf included the header the total will be lines + 1.\n\n**encoding**: *str* optional, defaults *None*\n\nThe input file encoding. \nThe chunks will be saved using the same encoding.\n\n**persist_header**: *bool* optional, defaults *False*\n\nWhether to persist a header on each chunk or not. \nIf persist_header is True and header is None, \nthe firt line will be read as the actual header.\n\n**header**: *str* optional, defaults *None*\n\nA header to be written at the start of each file.\nIf persist_header is True and header is None, \nthe firt line will be read as the actual header.\n\n**chunk_name_format**: *str* optional, defaults *\"04d\"*\n\nThe format string for the chunk numbers in the output file names.\n\n#### Returns\n\n**list**\n\nList of paths to the written files.\n\n#### Examples\n\n```python\nwith open(\"test.txt\", \"w\", encoding=\"utf-8\") as f:\n f.write(\"Symbols\\nAyp\\nBx\\nCC\\nDt\")\n\n>>> split_by_lines(\"test.txt\", \"out\", 2, encoding=\"utf-8\")\n[\"out/test.chunk0000.txt\", \"out/test.chunk0001.txt\", \n\"out/test.chunk0002.txt\"]\n>>> split_by_lines(\"test.txt\", \"out\", 2, encoding=\"utf-8\", persist_header=True)\n[\"out/test.chunk0000.txt\", \"out/test.chunk0001.txt\"]\n```\n\n### join_file_chunks\n\nJoin chunk files into a single line stream.\n\nJoin the files created by split_by_lines into a iterable of\nstr lines and read ordered by name. \nChunks not found will throw an error if ignore_missing_chunks is not False.\n\nAvoid saving different files chunks into the same directory.\n\n#### Parameters\n\n**files_path**: *str*\n\nThe path to a directory containing the chunk files or a list of the path to the files.\n\n**encoding**: *str* optional, defaults *None*\n\nThe input file encoding. \nThe chunks will be saved using the same encoding.\n\n**persisted_header**: *bool* optional, defaults *False*\n\nWhether a header was persisted on each chunk or not. \nIf persisted_header is True and header is None, \nthe firt line of teh first file will be read as the actual header.\n\n**header**: *str* optional, defaults *None*\n\nA header to be read at the start of the first file.\nIf persisted_header is True and header is None, \nthe firt line of teh first file will be read as the actual header.\n\n**ignore_missing_chunks**: *bool* optional, defaults *\"False*\n\nFlag to ignore missing chunks.\n\n#### Returns\n\n**iter**\n\nAn iterable to the lines of the files (read in the order).\n\n### split_file_by_lines\n\nSlice file by lines into an iterable of the sliced lines.\n\nReturns only cut lines in iterable str format. \nIt does not work with negative numbers.\n\n#### Parameters\n\n**files_stream**: *str*\n\nThe path and file to be cut.\n\n***start***: *int*\n\nInitial position of the cut.\nUse positive Number.\n\n***end***: *int*\n\nEnd position of the cut.\nUse Positive Number.\n\n**encoding**: *str* optional, defaults *None*\n\nThe input file encoding. \nThe chunks will be saved using the same encoding.\n\n**persisted_header**: *bool* optional, defaults *False*\n\nWhether a header was persisted on each chunk or not. \nIf persisted_header is True and header is None, \nthe firt line of teh first file will be read as the actual header.\n\n**header**: *str* optional, defaults *None*\n\nA header to be read at the start of the first file.\nIf persisted_header is True and header is None, \nthe firt line of teh first file will be read as the actual header.\n\n#### Returns\n\n**iter**\n\nAn iterable with the cut portion of the file\n\n### Examples\n\n```python\ntest_data = [\n \"Name;Age;\",\n \"Test;00;\",\n \"Test;11;\",\n \"John;22;\",\n \"Test;33;\",\n \"Test;44;\",\n \"Test;55;\",\n \"Test;66;\",\n \"Test;77;\",\n \"Test;88;\",\n]\nwith open(\"./text.txt\", \"w\") as f:\n f.write(\"\\n\".join(test_data))\n\n>>> slice_file_by_lines(input_path, 1,2, persist_header=True)\niter([\"Name;Age;\\n\", \"Test;11;\\n\", \"John;22;\\n\"])\n>>> slice_file_by_lines(input_path, 1,2)\niter([\"Test;11;\\n\", \"John;22;\\n\"])\n```\n\n## Test\n\nTest are handled by [PyTest](https://pypi.org/project/pytest/) and are included inside the folder `test`.\n\nTo run the test execute the command:\n\n pytest\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "https://github.com/metadeta96/pybite", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/metadeta96/pybite", "keywords": "iteration,iter,iterable,sequence,list,tuple,dict,array,chunk,block,processing,data,split,slice,file,buffer", "license": "", "maintainer": "", "maintainer_email": "", "name": "pybite", "package_url": "https://pypi.org/project/pybite/", "platform": "", "project_url": "https://pypi.org/project/pybite/", "project_urls": { "Download": "https://github.com/metadeta96/pybite", "Homepage": "https://github.com/metadeta96/pybite" }, "release_url": "https://pypi.org/project/pybite/1.4.2/", "requires_dist": null, "requires_python": ">=3.5", "summary": "Chunk by chunk iteration made easier", "version": "1.4.2" }, "last_serial": 5805183, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "a5fc2bb77de5d47cba56cc8175d0050e", "sha256": "0aeab73eedd371177d06b04c5271c2e314a2f863eb0ead00dec0c51f8131cabc" }, "downloads": -1, "filename": "pybite-1.0.0.tar.gz", "has_sig": false, "md5_digest": "a5fc2bb77de5d47cba56cc8175d0050e", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 1995, "upload_time": "2019-08-20T19:23:30", "url": "https://files.pythonhosted.org/packages/da/9a/03319f6c0d3ea42ccecfdf618d78e5f589c262d7ad6d1cc00ad0ed05abed/pybite-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "efc5fa1a4ed615346923ca06d1bf0f12", "sha256": "24450bed4803126f707890f9cb88989d3968830ca88c405b0ef35b0d33cf78cb" }, "downloads": -1, "filename": "pybite-1.0.1.tar.gz", "has_sig": false, "md5_digest": "efc5fa1a4ed615346923ca06d1bf0f12", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 1998, "upload_time": "2019-08-20T19:29:33", "url": "https://files.pythonhosted.org/packages/15/89/8b0ea50fa29af94924f64c403373895bbfa19cbb1ee10fd10b0715b949eb/pybite-1.0.1.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "d41ab72005686428e78ddcd48cebae3d", "sha256": "5ef3ba3467238a126610594f1abee7ceff1e4364462e9dc06f778ffad0918cca" }, "downloads": -1, "filename": "pybite-1.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "d41ab72005686428e78ddcd48cebae3d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 16919, "upload_time": "2019-08-27T12:57:10", "url": "https://files.pythonhosted.org/packages/6f/bc/5c5604288906b6d9116a87fbe6f24c3c0618561aa72760ebd7c41b007ac0/pybite-1.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "79e8d356a46b1b72879bb8d0c652d160", "sha256": "93951d66237aeff4afbf7ffb9be82af290afd9d3538539db22889d443dd8adc8" }, "downloads": -1, "filename": "pybite-1.1.0.tar.gz", "has_sig": false, "md5_digest": "79e8d356a46b1b72879bb8d0c652d160", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 4214, "upload_time": "2019-08-27T12:57:30", "url": "https://files.pythonhosted.org/packages/e3/e5/7a2bdf86a584323b464f22b2202f206e958a826adea631ece2455c5abf9a/pybite-1.1.0.tar.gz" } ], "1.2.0": [ { "comment_text": "", "digests": { "md5": "04715d08b23f4025b7df7f9b38beea02", "sha256": "04c37d3979cfc48c30b47e27fdf977549fa7d1ca9c9b7dfed9fda1527be65dd7" }, "downloads": -1, "filename": "pybite-1.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "04715d08b23f4025b7df7f9b38beea02", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 17898, "upload_time": "2019-08-27T15:52:48", "url": "https://files.pythonhosted.org/packages/9d/db/d09fb8309c4a3053dada50778f464be683f232f5844ed4a5e402b55b776a/pybite-1.2.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "213fd6e7f68613af79bec0a2923fc00f", "sha256": "53db35ca6c0e9fb15cefa06e24b1976f0e6fa5e2b3640b694fa6db06d895c9ca" }, "downloads": -1, "filename": "pybite-1.2.0.tar.gz", "has_sig": false, "md5_digest": "213fd6e7f68613af79bec0a2923fc00f", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 5157, "upload_time": "2019-08-27T15:52:54", "url": "https://files.pythonhosted.org/packages/cf/a2/7d78fbcc8df80f6cd6c60c99c72429e2cd8113b43411d22395440208d075/pybite-1.2.0.tar.gz" } ], "1.2.1": [ { "comment_text": "", "digests": { "md5": "80b9fcf23f9ae60eac55c91c3f99667d", "sha256": "c8279d5685755c1407654af6d317b223337a4850058cdc1f64ba1556780cc0a8" }, "downloads": -1, "filename": "pybite-1.2.1-py3-none-any.whl", "has_sig": false, "md5_digest": "80b9fcf23f9ae60eac55c91c3f99667d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 17914, "upload_time": "2019-08-27T16:07:30", "url": "https://files.pythonhosted.org/packages/ed/05/78bca2b080ed4d0a42667e864031abe70bd3c26f4d82bb1aa7dcef475c85/pybite-1.2.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "75a2c1cca9b0fa336678fd8760a69d16", "sha256": "c43a83919e38cc8d8f8f02f5eb4e633f9be655f279814b948a915b6d4e6938c1" }, "downloads": -1, "filename": "pybite-1.2.1.tar.gz", "has_sig": false, "md5_digest": "75a2c1cca9b0fa336678fd8760a69d16", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 5169, "upload_time": "2019-08-27T16:07:38", "url": "https://files.pythonhosted.org/packages/27/84/f80bebcf90a754c68f1b033b77cc66e0dd0aa99d1e2699c31891e0bff3b9/pybite-1.2.1.tar.gz" } ], "1.3.0": [ { "comment_text": "", "digests": { "md5": "43a9b3f9f76df84222641d8b34b26ce4", "sha256": "7871b8d42ef61496e5eeec43953af4b35660e204c0153b0dd682dd42f29ebb5a" }, "downloads": -1, "filename": "pybite-1.3.0-py3-none-any.whl", "has_sig": false, "md5_digest": "43a9b3f9f76df84222641d8b34b26ce4", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 18220, "upload_time": "2019-09-06T14:37:27", "url": "https://files.pythonhosted.org/packages/de/40/4fb02ad1f20bbc755161c488fe773670e12707f242c02fd83adc3463549c/pybite-1.3.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "64dbb204974f2bfe7c4ae23d7e201fe9", "sha256": "24e45eaf7d05852dc4c8ddad93b16ac4c9cd27dadda3366c62fb7c4e7342e00f" }, "downloads": -1, "filename": "pybite-1.3.0.tar.gz", "has_sig": false, "md5_digest": "64dbb204974f2bfe7c4ae23d7e201fe9", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 4173, "upload_time": "2019-09-06T14:37:35", "url": "https://files.pythonhosted.org/packages/a6/db/625925bdcf6f17a60a21f7076b383edef985c734071058be7a2084cbe826/pybite-1.3.0.tar.gz" } ], "1.3.1": [ { "comment_text": "", "digests": { "md5": "34d821b4dafb14635c9ed19cd5c9e25f", "sha256": "0dca070003f149423bea51553c3c21671a66cd9fdcfd5039af184c0f6de0042f" }, "downloads": -1, "filename": "pybite-1.3.1-py3-none-any.whl", "has_sig": false, "md5_digest": "34d821b4dafb14635c9ed19cd5c9e25f", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 19630, "upload_time": "2019-09-06T17:19:09", "url": "https://files.pythonhosted.org/packages/1d/fe/af7f46bd64ba0826784c77d370bbdac759887ee0cd4bc23777261ef3a73b/pybite-1.3.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e08bb619755d92c455433cae1e05fc73", "sha256": "e4cf21c07c6e5e517550b09574fd488a3cc916fd233fb7db9e3c5288df377155" }, "downloads": -1, "filename": "pybite-1.3.1.tar.gz", "has_sig": false, "md5_digest": "e08bb619755d92c455433cae1e05fc73", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 6175, "upload_time": "2019-09-06T17:19:11", "url": "https://files.pythonhosted.org/packages/7f/95/de623bca7ebff3131dc1bf6e119b9867a12a9455503cfde917c2858be057/pybite-1.3.1.tar.gz" } ], "1.3.2": [ { "comment_text": "", "digests": { "md5": "c235699380fea3bc25a26eb83aa78c50", "sha256": "482692007f8f01fce52e56c0ef07fda24a1aaa81bffbc1e29eb561a22143c541" }, "downloads": -1, "filename": "pybite-1.3.2-py3-none-any.whl", "has_sig": false, "md5_digest": "c235699380fea3bc25a26eb83aa78c50", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 19654, "upload_time": "2019-09-06T17:59:52", "url": "https://files.pythonhosted.org/packages/37/51/6f653feaed164e66ba493ae16e3ca184ddeca742e0cbebcc8ccd4c74c403/pybite-1.3.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9023df2cc64f9cc80789bc7f938c7379", "sha256": "1443b7dd201d5ef02f0c497a97274a81fadc1fe5c9dc9a115e21d8fa82f33d26" }, "downloads": -1, "filename": "pybite-1.3.2.tar.gz", "has_sig": false, "md5_digest": "9023df2cc64f9cc80789bc7f938c7379", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 6195, "upload_time": "2019-09-06T17:59:55", "url": "https://files.pythonhosted.org/packages/a9/86/c67eee690000ebb0f2c234e5af9451c59c49a660116e44f6e9f499a95cfa/pybite-1.3.2.tar.gz" } ], "1.4.1": [ { "comment_text": "", "digests": { "md5": "cdd34280d27df7de148d8a3f320e6dfa", "sha256": "377c543323f367d3af2d6b8b76ce8b6cb66bbb6a3ab17ab4171a4c6f463b9355" }, "downloads": -1, "filename": "pybite-1.4.1-py3-none-any.whl", "has_sig": false, "md5_digest": "cdd34280d27df7de148d8a3f320e6dfa", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 20367, "upload_time": "2019-09-09T15:05:20", "url": "https://files.pythonhosted.org/packages/5f/70/aabbdca8d21237833a4c7b23f1181b7ba2d77f06d6486241e41624b55184/pybite-1.4.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b3c65306da5e08153f59831be11ca1ac", "sha256": "070e3bd249e8fee3bc7c2df8b7d926e3cc33aac0f3e036ed78f592518ca718a4" }, "downloads": -1, "filename": "pybite-1.4.1.tar.gz", "has_sig": false, "md5_digest": "b3c65306da5e08153f59831be11ca1ac", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 6947, "upload_time": "2019-09-09T15:05:26", "url": "https://files.pythonhosted.org/packages/e2/54/ab2ce95b1a996865c5d8e0ff9f39a9b45616fafd69b56113502ece1c7224/pybite-1.4.1.tar.gz" } ], "1.4.2": [ { "comment_text": "", "digests": { "md5": "df0963eed1e3dd5a9e43b5d46219cc9d", "sha256": "3befcc162de0b61c771edddfeb16a3fca2862bd725f0c2b9438bf122e76b64af" }, "downloads": -1, "filename": "pybite-1.4.2-py3-none-any.whl", "has_sig": false, "md5_digest": "df0963eed1e3dd5a9e43b5d46219cc9d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 20376, "upload_time": "2019-09-09T19:39:16", "url": "https://files.pythonhosted.org/packages/27/b9/8456e8413c1940c1ff7e3a827c43c75d2b3d0468687e842a8fc2dafcfa8d/pybite-1.4.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "68865b8237c0d04b381e61a670a411e9", "sha256": "277a0dc99b08920d6d9f4964464a8d9bb0629cb91b89fe0c1de3b2a4d5deac7d" }, "downloads": -1, "filename": "pybite-1.4.2.tar.gz", "has_sig": false, "md5_digest": "68865b8237c0d04b381e61a670a411e9", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 6945, "upload_time": "2019-09-09T19:39:21", "url": "https://files.pythonhosted.org/packages/4d/e6/d8196b323353b99ada5e53e1680ae21ae1086cec52285c6e59009e9d29e8/pybite-1.4.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "df0963eed1e3dd5a9e43b5d46219cc9d", "sha256": "3befcc162de0b61c771edddfeb16a3fca2862bd725f0c2b9438bf122e76b64af" }, "downloads": -1, "filename": "pybite-1.4.2-py3-none-any.whl", "has_sig": false, "md5_digest": "df0963eed1e3dd5a9e43b5d46219cc9d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 20376, "upload_time": "2019-09-09T19:39:16", "url": "https://files.pythonhosted.org/packages/27/b9/8456e8413c1940c1ff7e3a827c43c75d2b3d0468687e842a8fc2dafcfa8d/pybite-1.4.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "68865b8237c0d04b381e61a670a411e9", "sha256": "277a0dc99b08920d6d9f4964464a8d9bb0629cb91b89fe0c1de3b2a4d5deac7d" }, "downloads": -1, "filename": "pybite-1.4.2.tar.gz", "has_sig": false, "md5_digest": "68865b8237c0d04b381e61a670a411e9", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 6945, "upload_time": "2019-09-09T19:39:21", "url": "https://files.pythonhosted.org/packages/4d/e6/d8196b323353b99ada5e53e1680ae21ae1086cec52285c6e59009e9d29e8/pybite-1.4.2.tar.gz" } ] }