{ "info": { "author": "Romain Dorgueil", "author_email": "romain@dorgueil.net", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Information Technology", "Intended Audience :: System Administrators", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python", "Topic :: Database", "Topic :: Text Processing :: Filters", "Topic :: Utilities" ], "description": "=======\nrdc.etl\n=======\n\n``rdc.etl`` is an Extract Transform Load (ETL) toolkit for python 2.7+ (no\npython 3 support yet). It gives you all the tools needed to create complex data\nintegration jobs from simple atomic io-connected transformation blocks.\n\n.. image:: https://api.travis-ci.org/rdcli/rdc.etl.png\n :target: https://travis-ci.org/rdcli/rdc.etl\n :alt: Build Status\n\n\nDocumentation\n-------------\n\n`Full documentation for rdc.etl on ReadTheDocs `_.\n\nThis is a work in progress, the 1.0 API may change until 1.0 is released.\n\n\nExample usage\n-------------\n\n.. code-block:: python\n\n >>> # Sample data extract transformation.\n >>> # Use hardcoded data here for sample purpose.\n >>> from rdc.etl.transform.extract import Extract\n >>> @Extract\n ... def sample_extract():\n ... yield {'first_name': 'John', 'last_name': 'Doe', }\n ... yield {'first_name': 'Jane', 'last_name': 'Dae', }\n\n\n.. code-block:: python\n\n >>> # Sample data transformation.\n >>> from rdc.etl.transform import Transform\n >>> @Transform\n ... def sample_transform(hash, channel):\n ... hash['last_name'] = hash['last_name'].upper()\n ... hash['initials'] = '{0}.{1}.'.format(hash['first_name'][0], hash['last_name'][0]).upper()\n ... yield hash\n\n\n.. code-block:: python\n\n >>> # Sample load. This is only a screen log for sample purpose.\n >>> from rdc.etl.transform.util import Log\n >>> sample_load_to_screen = Log()\n\n\n.. code-block:: python\n\n >>> # Tie everything together, then run!\n >>> from rdc.etl.harness.threaded import ThreadedHarness\n >>> job = ThreadedHarness()\n >>> job.add_chain(sample_extract, sample_transform, sample_load_to_screen)\n >>> job()\n\n\nRunning the Test Suite\n----------------------\n\n.. code-block:: shell\n\n pip install nose\n make test\n\n\nRelease Notes\n-------------\n\n1.0.0a6\n.......\n\n* Database transformations: now present in rdc.etl.extra subpackages, to avoid\n mixing core API and sugar box.\n* Database transformations: more flexibility in what is allowed (insert/update).\n* Better standard compliance (thanks #python)\n* Harness is now called Job, for simplicity sake. Old name will be kept too (BC).\n* XMLMap enhancements.\n* HTTP status interface (early minimalistic version).\n* Changed examples name to avoid import hell.\n* Less hackish http reader, better unicode support (@jmorel).\n* PasteScript can now be used to generate an empty working ETL project.\n* FileProxy based download manager.\n* Minor fixes.\n\n1.0.0a5\n.......\n\n* Status: console now has amazing ansi, detailed io statistics, overall stats\n (memory, time) added, experimental http status, db stats for database load.\n* API stabilization, cleanup and simplification towards 1.0.0.\n* Simple handling of KeyboardInterrupt: CTRL-C will now exit the running job\n instead of making your process stale.\n* Maps simplification.\n* Enhancements to various transform classes: load.database.DatabaseLoad,\n filter.Filter, map.xml.XmlMap, util.Log, join.database.DatabaseJoin\n* New transforms: util.Limit\n* Various bugfixes.\n* Minor enhancements: custom names in transforms, some more tests.\n* Moved repository to github.com/rdcli/etl.\n\n\nContributing\n------------\n\nI'm Romain Dorgueil.\n\n``rdc.etl`` is on `GitHub `_.\n\nGet in touch, via GitHub or otherwise, if you've got something to contribute,\nit'd be most welcome!\n\nIf you feel overwhelmingly grateful, or want to support the project you can tip\nme on `Gittip `_.", "description_content_type": null, "docs_url": null, "download_url": "https://github.com/rdcli/rdc.etl/tarball/1.0.0a6", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://etl.rdc.li/", "keywords": "ETL Data-Integration", "license": "Apache License, Version 2.0", "maintainer": null, "maintainer_email": null, "name": "rdc.etl", "package_url": "https://pypi.org/project/rdc.etl/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/rdc.etl/", "project_urls": { "Download": "https://github.com/rdcli/rdc.etl/tarball/1.0.0a6", "Homepage": "http://etl.rdc.li/" }, "release_url": "https://pypi.org/project/rdc.etl/1.0.0a6/", "requires_dist": null, "requires_python": null, "summary": "Extract Transform Load (ETL) toolkit for python", "version": "1.0.0a6" }, "last_serial": 1021343, "releases": { "1.0.0a1": [ { "comment_text": "", "digests": { "md5": "714ded5d4ad1b96812888e9872d3d297", "sha256": "a441a157d22a5912d67f459b6812a00ee73aa61a55149a15189f3722e207d52a" }, "downloads": -1, "filename": "rdc.etl-1.0.0a1.tar.gz", "has_sig": false, "md5_digest": "714ded5d4ad1b96812888e9872d3d297", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 23041, "upload_time": "2013-11-06T14:22:11", "url": "https://files.pythonhosted.org/packages/f3/e6/8e8519f761ed51f967fbcfe819f8bba38f9b620bee9b6500a068bd5b7fa0/rdc.etl-1.0.0a1.tar.gz" } ], "1.0.0a2": [ { "comment_text": "", "digests": { "md5": "e8e774cfcb7717bb91165a8844984ea9", "sha256": "fb7e79a64b771cea9a003c968ca5b90ea16c2529f1f20efe8a4c8e878bd3183b" }, "downloads": -1, "filename": "rdc.etl-1.0.0a2.tar.gz", "has_sig": false, "md5_digest": "e8e774cfcb7717bb91165a8844984ea9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 25971, "upload_time": "2013-11-11T23:02:12", "url": "https://files.pythonhosted.org/packages/ff/d6/a56ad7b1cf2704152d46e6d3448dda70e6e07c953b76a322c48fd22740e8/rdc.etl-1.0.0a2.tar.gz" } ], "1.0.0a3": [ { "comment_text": "", "digests": { "md5": "831157200e0b5a855ddf9bd2eff5b6e2", "sha256": "a785f64b0ef2c7031b51107f6b3830ecb09eb75aab9a3386497a1bcc4a657b38" }, "downloads": -1, "filename": "rdc.etl-1.0.0a3.tar.gz", "has_sig": false, "md5_digest": "831157200e0b5a855ddf9bd2eff5b6e2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 23293, "upload_time": "2013-11-29T13:26:00", "url": "https://files.pythonhosted.org/packages/13/ee/eb676708379bde551eb4787d3289ff09bc3555a989ded44e3c54e900fabf/rdc.etl-1.0.0a3.tar.gz" } ], "1.0.0a4": [ { "comment_text": "", "digests": { "md5": "4ae0ec803f9cc8139d8118ff9bc9ebcd", "sha256": "c90447fcbfddb93a077cd773bdd8965582960717f443501224307424f1009560" }, "downloads": -1, "filename": "rdc.etl-1.0.0a4.tar.gz", "has_sig": false, "md5_digest": "4ae0ec803f9cc8139d8118ff9bc9ebcd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 25318, "upload_time": "2013-11-30T19:31:06", "url": "https://files.pythonhosted.org/packages/e7/c3/c738f045db7e85ff65983eb272d847efd4a3f240958bfbf426896e638f32/rdc.etl-1.0.0a4.tar.gz" } ], "1.0.0a5": [ { "comment_text": "", "digests": { "md5": "d2a7d07353d8ca6576efe8d4bc2e4e3c", "sha256": "c25f981f9ce25af0172b80bba31a713bfbc9cb7c71cb76ef4968ed98b82aa864" }, "downloads": -1, "filename": "rdc.etl-1.0.0a5.tar.gz", "has_sig": false, "md5_digest": "d2a7d07353d8ca6576efe8d4bc2e4e3c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 31828, "upload_time": "2014-01-07T10:21:54", "url": "https://files.pythonhosted.org/packages/64/95/1fb5ca7ddc562c8d3019009c69364c62b2690aecbe948c35a5efd86ab5cd/rdc.etl-1.0.0a5.tar.gz" } ], "1.0.0a6": [ { "comment_text": "", "digests": { "md5": "1c4bffc2cacb90dbc94d599c8ee234ef", "sha256": "e855c2f3d44928f3b19683485c677c4e6ef8f1a0f2d87f444a4db8ad2e8051d6" }, "downloads": -1, "filename": "rdc.etl-1.0.0a6.tar.gz", "has_sig": false, "md5_digest": "1c4bffc2cacb90dbc94d599c8ee234ef", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 37105, "upload_time": "2014-03-06T17:16:55", "url": "https://files.pythonhosted.org/packages/eb/fc/dcbd5db525cd16d3e4b9c88a3b2c02654f2140596f0c62e2f6978c39527e/rdc.etl-1.0.0a6.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1c4bffc2cacb90dbc94d599c8ee234ef", "sha256": "e855c2f3d44928f3b19683485c677c4e6ef8f1a0f2d87f444a4db8ad2e8051d6" }, "downloads": -1, "filename": "rdc.etl-1.0.0a6.tar.gz", "has_sig": false, "md5_digest": "1c4bffc2cacb90dbc94d599c8ee234ef", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 37105, "upload_time": "2014-03-06T17:16:55", "url": "https://files.pythonhosted.org/packages/eb/fc/dcbd5db525cd16d3e4b9c88a3b2c02654f2140596f0c62e2f6978c39527e/rdc.etl-1.0.0a6.tar.gz" } ] }