{ "info": { "author": "Data Monkey, LLC", "author_email": "help@data-monkey.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6" ], "description": "![PyPI - Python Version](https://img.shields.io/pypi/pyversions/datamonkey.svg)\n![PyPI - License](https://img.shields.io/pypi/l/datamonkey.svg?color=green)\n![PyPI](https://img.shields.io/pypi/v/datamonkey.svg?color=orange&label=version)\n\nData Monkey\n==========\n\n**Easy-to-use file transformation library** \n\n--- \n\n## What is it?\n\n**Data Monkey** is a Python file transformation library designed to take the stress\nout of manipulating data. It provides powerful tools to both **validate\ndata integrity** and **modify your files into the exact format you need**.\nThis library works in conjunction with file templates created on through the [Data Monkey web application](https://app.data-monkey.com) -- once you've created a file template, it's simple to start processing files locally.\n\nData Monkey currently supports the following file formats:\n1. CSV\n2. Excel\n3. Fixed-width text files (AKA flat files)\n4. JSON (with a flat, non-nested data structure)\n\nCheck out the feature documentation [documentation](https://docs.data-monkey.com/) for more details.\n\n## Main Features\n\n - Spend less time writing routine code to ingest or transfrom data loaded from a file or external API. Data Monkey lets you reliably manipulate data into exactly what you need; whether you're converting a CSV file into JSON or generating an Excel report from an API response, Data Monkey has you covered.\n - A no code, configuration-based approach -- use our [web application](https://app.data-monkey.com) to quickly update your file templates as often as necessary. It's simple to change the file type that's generated, filter out unnecessary results, combine and transform data, and much more.\n - It's frustrating to find unexpected 'surprises' in your data that don't line up with your expectations -- which is why data validation is a core feature of Data Monkey. Configure any number of data validations that will alert you if something isn't quite right. \n - Data Monkey provides play-by-play progress tracking when transforming large files to let you know exactly where you're at in the process.\n\n## Where to get it\n\nThe source code for the community edition is currently hosted on [GitHub](https://github.com/datamonkey-hq/datamonkey).\n\nBinary installers for the latest released version are available at the [Python\npackage index](https://pypi.org/project/datamonkey).\n\n```python\n# PyPI\npip install datamonkey\n```\n\nThis will automatically install the required dependencies listed [below](#dependencies), if you don't already have them available. Additionally, please take a look at the optional dependencies; you may need them if you're working with Excel files, or writing/reading files to S3.\n\n## Usage & Examples\n\nThe first step to processing any file is creating a file template on the [Data Monkey web application](https://app.data-monkey.com).\nFile templates are a configuration-based approach to defining exactly how your files will be transformed -- such as the desired file types, outgoing data fields, and any custom validators or modifications to the data itself.\nYou can view a guide on creating file templates [here](https://documentation.data-monkey.com).\n\nOnce your file template is complete, you can use the template key to locally process files with the following commands:\n\n```python\nfrom datamonkey import FileProcessor\n# intialize the file processor using your file template key (generated on data-monkey.com)\nprocessor = FileProcessor(YOUR_TEMPLATE_KEY)\n# process a local file (or point to a location on S3)\nprocessor.process(PATH/TO/FILE)\n```\n\nIf successful, the new file will be generated; otherwise, you'll see an errors and\nwarnings file that will contain details on any issues.\n\n\n## Additional Configuration\n\n### Offline Processing\nBy default, Data Monkey will attempt to pull the most recent version of your file template\non each run using the template key you provide. If you'd prefer to use a local\nversion of the template (downloadable on the summary screen of your file template),\nyou can initialize the `FileProcessor` with the location of the template file:\n\n```python\nfrom datamonkey import FileProcessor\nprocessor = FileProcessor(YOUR_KEY, template_file_path=\"PATH/TO/FILE\")\n```\n\nThis will allow you to process files completely offline or from within a secure environment with limited network access.\n\n### File Locations\nYou can also override the location/name of:\n1. The generated output file\n2. The errors and warnings file \n\n```python\nfrom datamonkey import FileProcessor\nprocessor = FileProcessor(YOUR_KEY)\nprocessor.process(\"PATH/TO/INPUT\", output_file_path=\"PATH/TO/OUTPUT\", error_file_path=\"PATH/TO/ERROR\")\n```\n\n### Files on AWS S3\nData Monkey uses ``s3fs`` to read and write files on Amazon Web Service's S3. To access S3, replace the file path with the S3 bucket location:\n\n```python\nfrom datamonkey import FileProcessor\nprocessor = FileProcessor(YOUR_KEY)\nprocessor.process(\"s3://BUCKET_NAME/INPUT_FILE_NAME\", output_file_path=\"\"s3://BUCKET_NAME/OUTPUT_FILE_NAME\", error_file_path=\"s3://BUCKET_NAME/ERROR_FILE_NAME\")\n```\n\nNote: You must have locally configured AWS credentials with read/write access to the buckets you want to use. See the [AWS documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) for more help.\n\n## Dependencies\n\nThe following libraries are required for Data Monkey to function:\n\n- [numpy](https://www.numpy.org): 1.14.5 or higher\n- [pandas](https://labix.org/python-dateutil): 0.23.1 or higher\n- [requests](https://pythonhosted.org/pytz): 2.19.1 or higher\n- [ujson](https://pythonhosted.org/pytz): 1.35.0 or higher\n\nThese dependencies are required if reading or writing files stored on Amazon Web Services S3:\n- [s3fs](https://www.numpy.org): 0.1.5 or higher\n\nThese dependencies are required if working with Excel files:\n- [xlrd](https://labix.org/python-dateutil): 1.1.0 or higher\n- [XlsxWriter](https://labix.org/python-dateutil): 1.1.1 or higher\n\n\n## License\n[MIT](https://github.com/DataMonkeyHQ/datamonkey/blob/master/LICENSE)\n\n## Additional Documentation\nThe official documentation is hosted here: https://docs.data-monkey.com/. It has several helpful guides, such as how to create a file template, and an appendix with information on the various operations you can perform on your data.\n\n## Background\n`datamonkey` was born out of the frustration of having to reinvent the wheel every\ntime any new file needed to be processed with specific requirements. Work on ``datamonkey`` started in 2018 and development is ongoing. We'd love your feedback on features you'd like to see or file formats that should be supported -- please let us know at feedback@data-monkey.com!\n\n## Getting Help\nFor usage questions, inquiries, or general feedback, please feel free to contact us via email at help@data-monkey.com.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/datamonkeyhq/datamonkey", "keywords": "datamonkey", "license": "MIT license", "maintainer": "", "maintainer_email": "", "name": "datamonkey", "package_url": "https://pypi.org/project/datamonkey/", "platform": "", "project_url": "https://pypi.org/project/datamonkey/", "project_urls": { "Homepage": "https://github.com/datamonkeyhq/datamonkey" }, "release_url": "https://pypi.org/project/datamonkey/1.0.5/", "requires_dist": [ "pandas", "numpy", "requests", "ujson" ], "requires_python": "", "summary": "Data processing and transformation library.", "version": "1.0.5" }, "last_serial": 4974030, "releases": { "1.0.2": [ { "comment_text": "", "digests": { "md5": "533db45a9dbf0e6d519a8d6973cd027d", "sha256": "f88a061119683d82c33e27c0c2c944e361666b85e566166557f37647fd2c6f02" }, "downloads": -1, "filename": "datamonkey-1.0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "533db45a9dbf0e6d519a8d6973cd027d", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 35347, "upload_time": "2019-03-19T00:10:43", "url": "https://files.pythonhosted.org/packages/07/36/2f748c733570f5ab99e1a5534fc853d1ae0b8ac8495e66f92ddf57fb7367/datamonkey-1.0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4e3bcd3c8d95b9a72bede0836e04ff71", "sha256": "3df79d354eddb11edc43ae91d090d87b43a142a698fe99c44751f5445e13f810" }, "downloads": -1, "filename": "datamonkey-1.0.2.tar.gz", "has_sig": false, "md5_digest": "4e3bcd3c8d95b9a72bede0836e04ff71", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 65005, "upload_time": "2019-03-19T00:10:45", "url": "https://files.pythonhosted.org/packages/12/6c/33ef2eb5349d3cabae4fcfc24de51ee2f2e9a7e58a9c303f706f1fba5455/datamonkey-1.0.2.tar.gz" } ], "1.0.4": [ { "comment_text": "", "digests": { "md5": "900041aa2651461b3b2d0122b4afd609", "sha256": "cacd1b4e4f9f5e58a94a170dab341f175d729286439650fe718ae296be7c59ab" }, "downloads": -1, "filename": "datamonkey-1.0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "900041aa2651461b3b2d0122b4afd609", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 35498, "upload_time": "2019-03-19T18:46:44", "url": "https://files.pythonhosted.org/packages/bd/67/3e5c5de05c1ce1c674dcb18db75c8ee1a86b2b1fd23dda647a25e4740a40/datamonkey-1.0.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ef1bc9330c5393707298516342f77af7", "sha256": "886e43f1d4b058e15845aecc7c8eac3c4aa9a3176429eb6da82681755894234f" }, "downloads": -1, "filename": "datamonkey-1.0.4.tar.gz", "has_sig": false, "md5_digest": "ef1bc9330c5393707298516342f77af7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 65470, "upload_time": "2019-03-19T18:46:46", "url": "https://files.pythonhosted.org/packages/27/c7/a74691beab2ab8ce4736313edb8c1df7670ffbdfdcd2b3da7768e68a5f5a/datamonkey-1.0.4.tar.gz" } ], "1.0.5": [ { "comment_text": "", "digests": { "md5": "79e860f8695ff9019b923660cb0c1b5e", "sha256": "a8f39c4664a944b3de3ec6a380a3e4dd6efef4ae7a076398db16518b00a5ad51" }, "downloads": -1, "filename": "datamonkey-1.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "79e860f8695ff9019b923660cb0c1b5e", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 35544, "upload_time": "2019-03-22T19:07:32", "url": "https://files.pythonhosted.org/packages/f2/2f/cac7e2debc95baaa56a1a1eaf60703da43aca7e3d9556db1b4bb371fa5c0/datamonkey-1.0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "35eb93825dae6df3834f60b021f2e684", "sha256": "774a655e598ec3e1bd3c8003d41f57649ebf676eb96a88b2d93a2300787be9aa" }, "downloads": -1, "filename": "datamonkey-1.0.5.tar.gz", "has_sig": false, "md5_digest": "35eb93825dae6df3834f60b021f2e684", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 65674, "upload_time": "2019-03-22T19:07:33", "url": "https://files.pythonhosted.org/packages/e4/e7/1d156d1b7fda6ba6f27438044736bce0ffe85e3d8b40cf6a9e3f379acbad/datamonkey-1.0.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "79e860f8695ff9019b923660cb0c1b5e", "sha256": "a8f39c4664a944b3de3ec6a380a3e4dd6efef4ae7a076398db16518b00a5ad51" }, "downloads": -1, "filename": "datamonkey-1.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "79e860f8695ff9019b923660cb0c1b5e", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 35544, "upload_time": "2019-03-22T19:07:32", "url": "https://files.pythonhosted.org/packages/f2/2f/cac7e2debc95baaa56a1a1eaf60703da43aca7e3d9556db1b4bb371fa5c0/datamonkey-1.0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "35eb93825dae6df3834f60b021f2e684", "sha256": "774a655e598ec3e1bd3c8003d41f57649ebf676eb96a88b2d93a2300787be9aa" }, "downloads": -1, "filename": "datamonkey-1.0.5.tar.gz", "has_sig": false, "md5_digest": "35eb93825dae6df3834f60b021f2e684", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 65674, "upload_time": "2019-03-22T19:07:33", "url": "https://files.pythonhosted.org/packages/e4/e7/1d156d1b7fda6ba6f27438044736bce0ffe85e3d8b40cf6a9e3f379acbad/datamonkey-1.0.5.tar.gz" } ] }