{ "info": { "author": "Michael Conroy", "author_email": "sietekk@gmail.com", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.2", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4" ], "description": ".. image:: https://travis-ci.org/sietekk/csv.toolkit.svg?branch=master\n :target: https://travis-ci.org/sietekk/csv.toolkit\n :alt: Build Status\n\n.. image:: https://coveralls.io/repos/github/sietekk/csv.toolkit/badge.svg?branch=master\n :target: https://coveralls.io/github/sietekk/csv.toolkit?branch=master\n :alt: Coverage\n\n\n********************\nCSV Toolkit Overview\n********************\n\n\n**NOTE: THIS PROJECT HAS SINCE BEEN FORKED TO THE INTERNAL PROMETHEUS\nRESEACH, LLC TOOL PROPS.CSVTOOLKIT**\n\n\nCSV Toolkit is a `Python`_ package that provides validation tooling\nand processing of CSV files. The validation tooling is based on the\nfantastic package `Vladiate`_. The interface and extension mechanisms\nare similarly implemented as the `rex.core`_ extension mechanisms.\n\n.. _`Python`: https://www.python.org\n.. _`Vladiate`: https://github.com/di/vladiate\n.. _`rex.core`: https://bitbucket.org/rexdb/rex.core\n\n.. contents:: Table of Contents\n\n\nExample Usage\n=============\n\nThis packace comes equipped with validation tooling, a CLI, a tooling interface,\na logging mechaism, and a loader mechanism. All are extensible, allow for\nfuture additions of new tools to this package, and the instroduction of custom\ntools depending on this package. This package comes with implementations built\nin as well.\n\n\nValidation Tooling\n******************\n\nThis application comes with a validation tooling mechanism buil-tin. It allows\nfor defining a validation schema to run against a CSV file. This was implemented\ndue to the severe lack of strict validation mechanisms in the Python standard\nlibrary's ``csv`` module. While it does implement the ``csv`` module to some\nextent, it allows for strict validation with an extensible validation mechanism.\nFurthermore, the validation mechanism may be used via the CLI or as a standard,\ninternal validation mechanism for your pacakge.\n\n\nBuilt-In Simple CSV Validator\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nIncluded with this package is a simple CSV file validation mechanism to use to\nvalidate simple CSV structures where fields may contain any values or may be\nempty. This is also a good example of how to implement a CSV validation schema\nas an internal tool available to the CLI.\n\nNew Implementations\n^^^^^^^^^^^^^^^^^^^\n\nSubclass the ``BaseFileValidator`` class to create a new CSV validation tool. The\nrequired fields ``validators``, ``delimeter``, ``default_validator``,\n``check_duplicate_headers``, and ``logger`` attributes must be defined. Creating\na new logger for each CSV validating tool is recommended, but not necessary.\n\nAn example bare-bones implementation would be::\n\n >>> class YourFirstValidatorLogger(Logger):\n >>> pass\n >>>\n >>> class YourFirstValidator(BaseFileValidator):\n >>> validators = {\n >>> \"Field1\": [],\n >>> \"Field2\": [],\n >>> \"Field3\": [],\n >>> }\n >>> delimiter = \",\"\n >>> default_validator = AnyVal\n >>> check_duplicate_headers = True\n >>> logger = YourFirstValidatorLogger\n >>>\n >>> def validate(self):\n >>> ... validation mechanism here...\n >>>\n >>> validator = YourFirstValidator(LocalFileLoader('/path/to/example.csv'))\n >>> print validator.validate()\n True\n >>> result = validator()\n >>> print result.validation\n True\n >>> print result.log\n ... validation log text...\n\nObviously, you may call the ``validate`` property directly without a logger, but\nyou may also call the validator instance, which returns a named tuple ``Result``\nwith ``validation`` and ``log`` attributes.\n\nPlease note, att this time the ``BaseFileValidator`` only supports loggers of the\nbuilt-in type. Pull requests and contributions to change this are more than\nwelcome.\n\nValidator Attribute Definition\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nThe ``validators`` attribute must define the validation schema for your type of\nCSV. It must be a dictionary with string keys defining the available columns and\nlist values specifying the validator (with any initialization parameters the\nvalidator requires).\n\nAn example validation schema would look like::\n\n >>> validators = {\n >>> \"Foo\": [\n >>> UniqueVal(),\n >>> ],\n >>> \"Bar\": [\n >>> RegexVal(r'^baz$'),\n >>> ],\n >>> \"hello world\": [\n >>> IntVal(empty_ok=True),\n >>> ],\n >>> }\n\nThis schema corresponds to a CSV with headers ``Foo``, ``Bar``, and\n``hello world``. The ``Foo`` column must contain unique values, the ``Bar``\ncolumn must contain fields matching the regular expression ``^baz$``, and the\n``hello world`` column must contain integer values, but allows for empty fields\nas well.\n\nBuilt-In Validators\n^^^^^^^^^^^^^^^^^^^\n\nThis package comes with built-in validators. For example:\n\n- IntVal: Integer values (allows empty values)\n- FloatVal: Float values (allows empty values)\n- BoolVal: Boolean values (allows empty values)\n- EnumVal: Enumerated values::\n\n EnumVal(['a', 'list', 'of', 'enumerations',])\n\n- UniqueVal: Unique values only\n- RegexVal: Fields must match supplied regex value (or no fields are matched)\n- EmptyVal: All fields must be empty\n- AnyVal: Any allowed values, but not empty\n\n**NOTE:** Inclusion of a JSON validator has not been made at this time, but\npull requests and contributions of an implementation are welcome.\n\n\nLogging\n*******\n\nThe logging mechanism is simple, and records logs to an internal dictionary per\ninstantiation. This allows for easy storage and retrieval of logs and logging\ninformation pertinent to your CSV tool.\n\nOne may use the global logging instance ``logger_main``, the logging context\nmanager ``logger_context``, or subclass the logging implementation ``Logger``\nto create custom logging instances.\n\n\nLoaders\n*******\n\nThe loader mechanism provides an easy tool to work with files and string objects.\nA simple wrapper around a specified ``loader``, working with file-like objects\nbecomes much simpler when working with CSV data.\n\nA user may work with the ``StringLoader`` or ``LocalFileLoader`` classes by\ninstantiating them with a source string or directory. For example::\n\n >>> mystring = StringLoader(StringIO(\"A test string.\"))\n >>> teststring = mystring.open()\n >>> print teststring\n \"A test string.\"\n\nTo create new loaders, simply subclass the ``Loader`` class, specify a loader\nand any args or kwargs that are necessary for that loader to operate.\n\n\nTooling\n*******\n\nThis package provides a tooling interface to allow automatic discovery of new\ntooling commands for the CLI. Simply subclass the ``Tool`` class to create a\nnew tool, which will be usable via the CLI. Make sure to specify the required\n``name`` attribute. A ``description`` atrribute is very useful, and if your\ntool/command requires it, specify the ``arguments`` attribute.\n\nThe ``implementation`` method must be overriden to tell the application what to\ndo when the command is run or the tool is used internally to an application. The\nfunction must return a ``0`` if successful and a ``1`` or other if not. The\nreturned value is passed to stdout for successes and stderror for failures.\n\n\nArguments\n^^^^^^^^^\n\nThe arguments must be a list of tuples with each touple containing the\nparameters usually passed to the ``argparse.add_argument()`` function. For\nexample, a typical implementation looks like::\n\n >>> self.parser.add_argument(\n >>> \"filename\",\n >>> type=argparse.FileType('r'),\n >>> help=\"A file.\"\n >>> )\n\nwhich, for a tool implementation, should be converted too::\n\n >>> arguments = [\n >>> (\n >>> 'filename',\n >>> {'type': argparse.FileType('r')},\n >>> {'help': 'A file.'},\n >>> ),\n >>> ]\n\nPlease note that the ``scripts.py`` file (the entry point for the CLI) will\nparse known arguments from the command line, and pass the rest to your tooling\nimplementation.\n\n\nThe CLI\n*******\nThe command line interface automatically discovers all tooling implementations\nsubclassed from the interface ``Tool`` super class. The base command line\nargument is ``csvtoolkit`` with a named parameter. The named parameter is any of\nthe available tooling implementations' ``name`` attribute.\n\nFor example::\n\n >>> class MyTool(Tool):\n >>> name = \"my-super-awesome-tool\"\n >>> ... and so on...\n\nThis tooling implementation is available via the CLI with the command::\n\n $ csvtoolkit my-super-awesome-tool\n\nAgain, please note that the ``scripts.py`` file (the entry point for the CLI)\nwill parse known arguments from the command line, and pass the rest to your\ntooling implementation.\n\n\nContributing\n============\n\nContributions and/or fixes to this package are more than welcome. Please submit\nthem by forking this repository and creating a Pull Request that includes your\nchanges. We ask that you please include unit tests and any appropriate\ndocumentation updates along with your code changes. Code must be `PEP 8`_\ncompliant.\n\nThis project will adhere to the `Semantic Versioning`_ methodology as much as\npossible, so when building dependent projects, please use appropriate version\nrestrictions.\n\n.. _`Semantic Versioning`: http://semver.org\n.. _`PEP 8`: https://www.python.org/dev/peps/pep-0008/\n\nA development environment can be set up to work on this package by doing the\nfollowing::\n\n $ virtualenv csvtools\n $ cd csvtools\n $ . ./bin/activate\n $ git clone https://github.com/sietekk/csv.toolkit.git\n $ pip install -e ./csvtools[dev]\n\n\nLicense/Copyright\n=================\n\nThis project is licensed under The MIT License. See the accompanying\n``LICENSE.rst`` file for details.\n\nCopyright (c) 2016, Michael Conroy\n\n\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/sietekk/csv.toolkit", "keywords": "csv tool validation processing", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "csv.toolkit", "package_url": "https://pypi.org/project/csv.toolkit/", "platform": "", "project_url": "https://pypi.org/project/csv.toolkit/", "project_urls": { "Homepage": "https://github.com/sietekk/csv.toolkit" }, "release_url": "https://pypi.org/project/csv.toolkit/0.1.0/", "requires_dist": [ "six (>=1.5,<2)", "pbbt (>=0.1.4,<1); extra == 'dev'", "coverage (>=3.7,<4); extra == 'dev'", "nose (>=1.3,<2); extra == 'dev'", "nosy (>=1.1,<2); extra == 'dev'", "prospector[with_pyroma] (<0.13,>=0.12); extra == 'dev'", "twine (>=1.5,<2); extra == 'dev'", "wheel (<0.25,>=0.24); extra == 'dev'", "Sphinx (>=1.3,<2); extra == 'dev'", "sphinx-autobuild (>=0.5,<0.6); extra == 'dev'", "tox (<3,>=2); extra == 'dev'", "flake8 (>=2.5.0,<3); extra == 'dev'" ], "requires_python": "", "summary": "Tools for working with CSV files", "version": "0.1.0" }, "last_serial": 2276685, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "36e5072d520dd52acb2b1739b772a7c5", "sha256": "33c56ca3b7f55abecd02463401cc9b7824dfd87cf1b8ba3ca1c9b5f47163cb8c" }, "downloads": -1, "filename": "csv.toolkit-0.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "36e5072d520dd52acb2b1739b772a7c5", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 21207, "upload_time": "2016-08-11T19:34:11", "url": "https://files.pythonhosted.org/packages/b6/7c/97a655bd20389a76b6dfa8845659f9c85986e5ede61a6bae630d52d6ed2a/csv.toolkit-0.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3f12089f8f95da809668e9b4768313f7", "sha256": "b4a104a045258d5866f0475b51f48ab87b41ca3f25332e5c068c6ae20520b018" }, "downloads": -1, "filename": "csv.toolkit-0.1.0.tar.gz", "has_sig": false, "md5_digest": "3f12089f8f95da809668e9b4768313f7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16755, "upload_time": "2016-08-11T19:34:13", "url": "https://files.pythonhosted.org/packages/2a/cc/d76ab72039b1bcb0576b313c303daee4e175aa0cb27d4ce5b32a4ae3f4ed/csv.toolkit-0.1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "36e5072d520dd52acb2b1739b772a7c5", "sha256": "33c56ca3b7f55abecd02463401cc9b7824dfd87cf1b8ba3ca1c9b5f47163cb8c" }, "downloads": -1, "filename": "csv.toolkit-0.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "36e5072d520dd52acb2b1739b772a7c5", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 21207, "upload_time": "2016-08-11T19:34:11", "url": "https://files.pythonhosted.org/packages/b6/7c/97a655bd20389a76b6dfa8845659f9c85986e5ede61a6bae630d52d6ed2a/csv.toolkit-0.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3f12089f8f95da809668e9b4768313f7", "sha256": "b4a104a045258d5866f0475b51f48ab87b41ca3f25332e5c068c6ae20520b018" }, "downloads": -1, "filename": "csv.toolkit-0.1.0.tar.gz", "has_sig": false, "md5_digest": "3f12089f8f95da809668e9b4768313f7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16755, "upload_time": "2016-08-11T19:34:13", "url": "https://files.pythonhosted.org/packages/2a/cc/d76ab72039b1bcb0576b313c303daee4e175aa0cb27d4ce5b32a4ae3f4ed/csv.toolkit-0.1.0.tar.gz" } ] }