{ "info": { "author": "David Davis", "author_email": "davisd50@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 2 - Pre-Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7" ], "description": "#Introduction\n\nA generic information to SQL DB caching utility.\n\n##Usage\n\ncache [-h] [--package PACKAGE] [--verbose] [--debug] source db_url\n\nThis application requires you to create one or more custom Python packages\nthat define and register the CSV <-> SQL mapping mechanisms. See the\n*Requirements* section below for an example.\n\nCSV information cache update script. This will import CSV data into a\ndatabase. If the database is already populated, then only new entries and\nupdates will be imported. Please reference Pypi package documentation for\ndetailed usage information.\n\n**positional arguments:**\n\n\n \n \n \n \n \n \n \n \n \n
source \n Valid CSV source. This should be a path to a CSV file, or a directory \n that contains CSV files. CSV data sourcesthat are found to have a \n matching registered sparc.cache.interfaces.ICachedItemMapper interface \n will import updates to SQL cache area identified by db_url\n
db_url \n Valid database url to use for connection (see \n http://docs.sqlalchemy.org/en/rel_0_7/core/engines.html) for detailed \n specifications for url. Short-hand definition is\n dialect+driver://user:password@host/dbname[?key=value..].\n \n
\n\n**optional arguments:**\n\n\n \n \n \n \n \n \n \n \n \n \n
--module MODULE \n Python package.module to import Python-based configuration from. See \n deatailed package documentation (README.md) for how to create and \n registered the required components. This should be the full \n package.module name.\n
--package PACKAGE \n Python package to configure for processing. There should be a valid \n configure.zcml file within the package that configures the required \n caching factories and mapping subscribers. Thisoption can be issued \n multiple times.\n
\n\n**sample invocations**\n\nThe samples assume you've create the *mypackage* Python package containing the\n*mymodule* Python module referenced below.\n\n Cache mycsvfile.csv information into a SQLite DB based on \n mypackage.mymodule schema.\n\n $ ~# cache --module mypackage.mymodule mycsvfile.csv sqlite:////tmp/cache.db\n\n##Capabilities\n\nThis utility allows the caching of CSV file information into a SQL database\nwith appropriate data type assignment (i.e. text vs. date vs integer vs ...).\n\n##Features\n\n - Can import a named CSV file, or a directory of CSV files into a SQL DB\n - Utilizes SQLAlchemy, allowing command line identified DB type selection\n - Cached data can be imported with appropriate named SQL data types\n\n##Requirements\n\nThis application can be used either as an installed Python package (i.e.\npip install ...), or as a platform-specific standalone binary called\n*cache*. But on it's own, the application will not do anything useful. \n*You* must create some implementation specific python code to augment this\napplication to perform the appropriate data caching.\n\nOne key note before we discuss the Python requirement, is the CSV source data\n*must* contain a primary key field (i.e. a field guranteed to be unique among\nall rows).\n\n###Python Requirement: Cachable Item factory\nYou'll need to provide this for the sole purpose of identifying which CSV field\nis considered the primary key. In the following example, the 'id' field is\nthe primary key.\n\n mypackage.mymodule\n >>> from zope.component.factory import Factory\n >>> from sparc.cache.item import cachableItemMixin\n >>> class myItem(cachableItemMixin):\n >>> def __init__(self, attributes=None):\n >>> super(myItem, self).__init__('id', attributes)\n >>> myCachableItemFactory = Factory(myItem, 'myCachableItemFactory')\n\nThe above code should work for most use-cases, where you would simply change\nthe *id* entries to what-ever CSV field contains the primary key.\n\nFor advanced usage, please reference the sparc.cache.interfaces.ICachableItem\ninterface definition and the sparc.cache documentation on how to create a \nfactory for ICachableItem items.\n\n###Python Requirement: Cached Item factory\nThis component will provide the SQL schema information that will be used to \ncreate and store the equivilant CSV entries in the SQL DB. Most use-cases \ncan leverage the pre-baked mixin class cachedItemMixin to minimize code to lines\nthat are specific to the implementation at hand.\n\n mypackage.mymodule\n >>> from sparc.cache.item import cachedItemMixin\n >>> from sparc.db import Base\n >>> from sparc.db.sql import sparcBaseMixin\n >>> class myCachedItem(cachedItemMixin, sparcBaseMixin, Base):\n >>> _key = 'id'\n >>> id = sqlalchemy.Column(sqlalchemy.VARCHAR(), primary_key=True)\n >>> time = sqlalchemy.Column(sqlalchemy.DateTime(), nullable=True)\n >>> user_ip = sqlalchemy.Column(sqlalchemy.VARCHAR(), nullable=True)\n >>> username = sqlalchemy.Column(sqlalchemy.VARCHAR(), nullable=True)\n >>> myCachedItemFactory = Factory(myCachedItem, 'myCachedItemFactory')\n\nThe above code should work for most use cases where the *_key* identifies which\nattribute should be considered the primary key in the DB table and the other\nattributes indicate the DB table column schemas.\n\nFor advacned usage, please reference sparc.cache.interfaces.ICachedItem\ninterface, sparc.cache documentation, and sparc.cache.sql.\n\n###Python Requirement: Mapper for CSV fields to SQL columns\nThis component maps CSV field names to SQL column names. Once again, we'll use\na mixin class to insure our custom code requirements are minimal. For the most\npart, we'll provide a Python dictionary whose keys reference the CSV field\nnames and values reference the associated SQL column names.\n\nOne note, notice the *normalizedDateTime()* reference below. This is a class\nthat implements IManagedCachedItemMapperAttributeKeyWrapper (see \nsparc.cache.interfaces) allowing Python based data magling before SQL\nstorage. Allow this is an advanced topic, it can be usefull in many\ncircumstances where CSV data is not well-strucutred (for example: dates where\na date string can take many different formats). \n\n mypackage.mymodule\n >>> from sparc.cache.sql import SqlObjectMapperMixin\n >>> from sparc.cache.sources.normalized_datetime import normalizedDateTime\n >>> class myItemCacheMapper(SqlObjectMapperMixin):\n >>> mapper = {\n >>> 'id' :'id', \n >>> 'time' : normalizedDateTime('time'),\n >>> 'user_ip' : 'user_ip',\n >>> 'username' : 'username'\n >>> }\n \n###Python Requirement: Component registration\nRegistration is what allows the caching application to lookup and\nleverage the custom components created above. We'll need to register our \n2 factories for ICachableItem and ICachedItem, and also a subscriber for the\nICachedItemMapper implementation.\n\n mypackage.mymodule\n >>> from zope.component import getGlobalSiteManager\n >>> from zope.component.interfaces import IFactory\n >>> from sparc.cache import ICachedItemMapper, ICachableSource\n >>> gsm = getGlobalSiteManager()\n >>> gsm.registerUtility(mypackage.mymodule.myCachableItemFactory, IFactory, u'mypackage.mymodule.myCachableItemFactory')\n >>> gsm.registerUtility(mypackage.mymodule.myCachedItemFactory, IFactory, u'mypackage.mymodule.myCachedItemFactory')\n >>> gsm.registerSubscriptionAdapter(myItemCacheMapper, (ICachableSource, IFactory,), ICachedItemMapper)\n\nComponents may also be registered via ZCML in combination with the --package\nlaunch parameter. In this case, you would need to have a properly formated\nconfigure.zcml file which defines the above components.\nChangelog\n=========\n\n0.0.1\n++++++++++++++++++\n\n* initial release\n\n0.0.4\n++++++++++++++++++\n\n* packaging update", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/davisd50/sparc.apps.cache", "keywords": "zca", "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "sparc.apps.cache", "package_url": "https://pypi.org/project/sparc.apps.cache/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/sparc.apps.cache/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/davisd50/sparc.apps.cache" }, "release_url": "https://pypi.org/project/sparc.apps.cache/0.0.5/", "requires_dist": null, "requires_python": null, "summary": "A generic CSV to SQL DB caching utility.", "version": "0.0.5" }, "last_serial": 1792196, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "242fa617ee7ee97eed3c5f7f4991ae9f", "sha256": "2bf74625b3302d044c35881c4a6e4c83dd161cb943254c4c487e0e2675d7dfa7" }, "downloads": -1, "filename": "sparc.apps.cache-0.0.1.tar.gz", "has_sig": false, "md5_digest": "242fa617ee7ee97eed3c5f7f4991ae9f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10427, "upload_time": "2015-10-28T19:16:49", "url": "https://files.pythonhosted.org/packages/2b/db/ef835e1043d0a10c1d8403acf7c5a37016eb27ed4202c3fd8f5c3c6606f1/sparc.apps.cache-0.0.1.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "55c067d6b71853b4686d07cddebf2d34", "sha256": "85b439c8f350ff075ce58b29ed543e3cef7aee64f8b424664fc18933a1f2f257" }, "downloads": -1, "filename": "sparc.apps.cache-0.0.3.tar.gz", "has_sig": false, "md5_digest": "55c067d6b71853b4686d07cddebf2d34", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10772, "upload_time": "2015-10-28T19:52:58", "url": "https://files.pythonhosted.org/packages/7f/ac/5864d79a973dfd2f4c3d625c11c312b46e7a3dcdc1ba124f2dae1c032529/sparc.apps.cache-0.0.3.tar.gz" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "b97dc96bde2c83b935cd42fd35e5057c", "sha256": "315c9d6d0933106e959907882a14a4039affd7a731708f4ab0d9fb75ac20f932" }, "downloads": -1, "filename": "sparc.apps.cache-0.0.4.tar.gz", "has_sig": false, "md5_digest": "b97dc96bde2c83b935cd42fd35e5057c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11340, "upload_time": "2015-10-28T20:18:55", "url": "https://files.pythonhosted.org/packages/e2/0c/9e6452f77b285abb5697cec4616efc0587886d46f2b7681561df072dd273/sparc.apps.cache-0.0.4.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "2f4293d284dba39540d0ba426733e036", "sha256": "6fc2c01f93f9803774a2549cd180d1b80f6b86f4ab58053d7d283b1904c12697" }, "downloads": -1, "filename": "sparc.apps.cache-0.0.5.tar.gz", "has_sig": false, "md5_digest": "2f4293d284dba39540d0ba426733e036", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11409, "upload_time": "2015-10-29T16:59:27", "url": "https://files.pythonhosted.org/packages/15/5c/fd61335883e864f3528d0c03ef343a96a2bead50159b09842e83e3bff72f/sparc.apps.cache-0.0.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "2f4293d284dba39540d0ba426733e036", "sha256": "6fc2c01f93f9803774a2549cd180d1b80f6b86f4ab58053d7d283b1904c12697" }, "downloads": -1, "filename": "sparc.apps.cache-0.0.5.tar.gz", "has_sig": false, "md5_digest": "2f4293d284dba39540d0ba426733e036", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11409, "upload_time": "2015-10-29T16:59:27", "url": "https://files.pythonhosted.org/packages/15/5c/fd61335883e864f3528d0c03ef343a96a2bead50159b09842e83e3bff72f/sparc.apps.cache-0.0.5.tar.gz" } ] }