{ "info": { "author": "Vagner Pagotti", "author_email": "pagotti@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.7", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "DasLaden is a simple, tiny and ridiculus ETL made with Python\r\n=============================================================\r\n\r\nDasladen is a general purpose Python package to make an automate ETL (Extracting, Transforming and\r\nLoading data) through the configuration of one or more .json files that represents tasks. \r\nIt is based on `petl`. It can do some tasks like:\r\n\r\n- load a .csv file to database table\r\n- run a database query into a .csv file\r\n- run a database query into a database table\r\n- convert a .csv file into another .csv file\r\n- convert a .xls file into a .csv file\r\n- load a .xml file into a database table\r\n- convert a .xls file into a .csv file\r\n\r\nThis tasks can be configured to do some basic transformations offer by `petl` and you can write your own\r\ntransformations in a Python module or class to be called by Dasladen during loading process.\r\n\r\nThere is others types of tasks to do things like:\r\n\r\n- Compact files into .zip file\r\n- Extract files from .zip file \r\n- Upload a file\r\n- Download a file\r\n- Execute a Python script\r\n- Execute a SQL command\r\n\r\nThe tasks are configured in a `.json` file that supports a sequence of tasks that will be executed \r\nin configured order. Details of how to configure tasks will be in Wiki pages. \r\n\r\nThe basic steps to use DasLaden is:\r\n\r\n- Install dasladen package via `pip install dasladen` in your environment or in virtualenv.\r\n- Install database driver package if you want to execute database tasks. Dasladen is prepared to run with the\r\nfollowing drivers: MySQL via `PyMySQL`, MS SQL Server via `pyodbc` and Oracle via `cx_Oracle`. Please see\r\nthe limitations on the driver package that you choose.\r\n- Create a folder for you project.\r\n- Prepare a folder structure in project folder with following names:\r\n - `input` Is the default folder to put input files, like .csv, .xml, .xls and .sql files\r\n - `output` Is the default folder that tasks write target files\r\n - `module` Is the folder for python scripts if you can't put then in project folder\r\n - `capture` Is the default folder to drop task files (.json or .zip)\r\n - `log` Is the folder that Dasladen write task logs\r\n - `tasks` Is the folder that you can put tasks files. It is only a suggestion.\r\n- Create a `.json` file with your tasks in `tasks` folder.\r\n- Start DasLaden from project folder calling `python -m dasladen`. \r\n- If you want to see log in console window, pass a `--verbose` as argument on call.\r\n- Copy the `.json` tasks file from `tasks` to the `capture` folder.\r\n\r\nThe watcher will open the tasks file and process it. To see result you can open `log` folder and search \r\nfor `watcher_DD_TT.log` where DD_TT is the date and time that log was generated. In `log` folder you\r\ncan see individual tasks logs too.\r\n\r\nIt is important that you copy the task file instead move it, because on finish it will be deleted.\r\n\r\nIf you drop a file other than `.zip` in `capture` folder, that file will be move to `input` folder.\r\n\r\nYou can zip the `.json` file with all other dependent files (.csv, .xls, etc.) and copy\r\nthat zip into `capture` folder too. Watcher will unzip then at a temporary folder, copy input\r\nfiles (other than `.json` files) to input folder and execute the `.json` file.\r\n\r\nIn the `.json` file you can configure a scheduler to run the tasks. With it you can delay a execution or \r\nconfigure its recurrence. \r\n\r\nData drivers via PyPi packages: \r\n- MySQL via [PyMySQL](https://pypi.org/project/PyMySQL/) package. v >= 0.7.5\r\n- MS SQL Server via [pyodbc](https://pypi.org/project/pyodbc/) package. v >= 3.0.10\r\n- Oracle via [cx_Oracle](https://pypi.org/project/cx_Oracle/) package. v >= 5.2.1\r\n- PostgreSQL via [psycopg2](https://pypi.org/project/psycopg2/) package. v >= 2.8.3\r\n\r\nThe current version works with Python 2 and 3.\r\n\r\n\r\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/pagotti/dasladen", "keywords": "etl", "license": "MIT License", "maintainer": "", "maintainer_email": "", "name": "dasladen", "package_url": "https://pypi.org/project/dasladen/", "platform": "", "project_url": "https://pypi.org/project/dasladen/", "project_urls": { "Homepage": "https://github.com/pagotti/dasladen" }, "release_url": "https://pypi.org/project/dasladen/0.2.0/", "requires_dist": [ "schedule", "petl", "backports.tempfile", "ftputil", "xlrd", "xlwt-future", "requests" ], "requires_python": "", "summary": "Simple, tiny and ridiculus ETL made with Python", "version": "0.2.0" }, "last_serial": 5900952, "releases": { "0.1.1": [ { "comment_text": "", "digests": { "md5": "da14898adc29acf69a8ead27c84ab601", "sha256": "5531acdbfd458844f14e3f1a5d70dd80c5f4c1bd222656248ffd5363507b8240" }, "downloads": -1, "filename": "dasladen-0.1.1-py2-none-any.whl", "has_sig": false, "md5_digest": "da14898adc29acf69a8ead27c84ab601", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 13220, "upload_time": "2018-08-18T22:44:05", "url": "https://files.pythonhosted.org/packages/f0/6e/c6523043dbb9a94e90fa454894452de8622db33a741c26f71804b150459e/dasladen-0.1.1-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f32641379d37bcde63759712d5bb40e9", "sha256": "5d9c7cdd4b050caf447e2cac4f0d166eaf49e791b1095c94ff11a93d88302b2b" }, "downloads": -1, "filename": "dasladen-0.1.1.tar.gz", "has_sig": false, "md5_digest": "f32641379d37bcde63759712d5bb40e9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11035, "upload_time": "2018-08-18T22:44:07", "url": "https://files.pythonhosted.org/packages/42/81/ba03ced30418a89589f76bd44991906681311d2effb804ca9d6b6ad103d1/dasladen-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "544d650bd0e25858e2e07728b5d72743", "sha256": "8c3a455be1d44f7cf4f9b3bb3b0eb92e1d03eaf622f06db4ed795976d86f5fea" }, "downloads": -1, "filename": "dasladen-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "544d650bd0e25858e2e07728b5d72743", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15053, "upload_time": "2019-07-24T02:11:59", "url": "https://files.pythonhosted.org/packages/72/61/88cdc686734aa52da78138ed02feafb6414caa330f3f5ca60385fe4bdac8/dasladen-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9e2717c6006bf54d6246316f90eeac21", "sha256": "0fa8907afea8f4af9589d5cfeb76326a720467ccd740b40ebf1f00dd39d296c1" }, "downloads": -1, "filename": "dasladen-0.1.2.tar.gz", "has_sig": false, "md5_digest": "9e2717c6006bf54d6246316f90eeac21", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13428, "upload_time": "2019-07-24T02:12:01", "url": "https://files.pythonhosted.org/packages/e0/7b/a43b5f4b28108d0ebe5e5255186aee61c256243de56663d9434efad0395a/dasladen-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "81be2886045d0729ec46b9304a8d83cd", "sha256": "cb0084699523142dc060b8762e2c06c31bb9523591bc06b2e5c6a2aa9fa34fc4" }, "downloads": -1, "filename": "dasladen-0.1.3-py3-none-any.whl", "has_sig": false, "md5_digest": "81be2886045d0729ec46b9304a8d83cd", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15081, "upload_time": "2019-07-24T02:43:50", "url": "https://files.pythonhosted.org/packages/d9/9a/4c0cdae5d74f5c23b425215df069ff74b238916aa0e25c05228451a84aa8/dasladen-0.1.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "5d7e6a87c846200fe8737b6270d6b009", "sha256": "ddd82f061c7f5f26976036ab6ad58bc36d789a0ab3c6c2bc28c71637b1616075" }, "downloads": -1, "filename": "dasladen-0.1.3.tar.gz", "has_sig": false, "md5_digest": "5d7e6a87c846200fe8737b6270d6b009", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13480, "upload_time": "2019-07-24T02:43:52", "url": "https://files.pythonhosted.org/packages/86/12/e31d1b386ecda0abf67f851fadcc763fc2f8db663d03eb1cf1dc98d38f0e/dasladen-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "76abc801328d189c9455ce7362686765", "sha256": "c541aaf2f18c7645e920175d73759018b268be46c7d8b2e13da6c0961cc4c8ef" }, "downloads": -1, "filename": "dasladen-0.1.4-py3-none-any.whl", "has_sig": false, "md5_digest": "76abc801328d189c9455ce7362686765", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 16077, "upload_time": "2019-07-24T02:54:38", "url": "https://files.pythonhosted.org/packages/e9/a0/126b807d0be12f49b38bc683172da4e2cf748f9821ef381ce3f9fb9e2aff/dasladen-0.1.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1f86886c770debec49bdd02d3cff7329", "sha256": "f0b953312e6e36da7066777996560a75bd4c531cea9b5c2512cb42ae0262b730" }, "downloads": -1, "filename": "dasladen-0.1.4.tar.gz", "has_sig": false, "md5_digest": "1f86886c770debec49bdd02d3cff7329", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13814, "upload_time": "2019-07-24T02:54:40", "url": "https://files.pythonhosted.org/packages/81/aa/b59648065971d2f3275dfa936fbf9626b706c58f0b560ff4c0be01d78230/dasladen-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "8ee08b0307405cdf98e847ca0048d82f", "sha256": "f52d443a107ac8cd1e4f13f7a81cd6710ad275d52152a2c882a4791be476a7b9" }, "downloads": -1, "filename": "dasladen-0.1.5-py3-none-any.whl", "has_sig": false, "md5_digest": "8ee08b0307405cdf98e847ca0048d82f", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 16103, "upload_time": "2019-07-24T03:03:11", "url": "https://files.pythonhosted.org/packages/54/7c/d22bb5f7c7158a354f563c86b19c73e965f0476fa0dc6d2bbd186c72ad8a/dasladen-0.1.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "77ba183e9cede81783d121f40922dc40", "sha256": "54d5ea043eee12766e6482c78bb5e5a5aace92ace2a68fec5fa2f899c2d9dacc" }, "downloads": -1, "filename": "dasladen-0.1.5.tar.gz", "has_sig": false, "md5_digest": "77ba183e9cede81783d121f40922dc40", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13782, "upload_time": "2019-07-24T03:03:13", "url": "https://files.pythonhosted.org/packages/11/3c/9cf44ccd418eefa7f0b1758f7e1cc63931c16bdf34a99b74bed06d492a58/dasladen-0.1.5.tar.gz" } ], "0.1.6": [ { "comment_text": "", "digests": { "md5": "95dc39c40bd1c8e799fb6d3afd4fdc88", "sha256": "268d8f01a4044938ec11004d6917ec74a54ca0242b79d1ff5165753b55b069f8" }, "downloads": -1, "filename": "dasladen-0.1.6-py3-none-any.whl", "has_sig": false, "md5_digest": "95dc39c40bd1c8e799fb6d3afd4fdc88", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 16109, "upload_time": "2019-07-24T03:05:39", "url": "https://files.pythonhosted.org/packages/7c/a5/418ddfd95163d7cc926103e4d325d3b8aa0d1b802f20bf6362be65c1c922/dasladen-0.1.6-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "06136f0c522e9ed65a70e710441a6d30", "sha256": "80794bae5d17f7a7a3e6254fdc55b874a899aa9cb34eb7f82657ebf4870c1071" }, "downloads": -1, "filename": "dasladen-0.1.6.tar.gz", "has_sig": false, "md5_digest": "06136f0c522e9ed65a70e710441a6d30", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13794, "upload_time": "2019-07-24T03:05:41", "url": "https://files.pythonhosted.org/packages/f7/18/a48d633c690f7381c4fdb87d779837218e75fc7950122ca9186bae09b728/dasladen-0.1.6.tar.gz" } ], "0.1.7": [ { "comment_text": "", "digests": { "md5": "a435d580961219fd8c227c6ce3cd13da", "sha256": "84ba8d688c82235a4590ce3fc0d835b1b8e20911ec18a8a5508d76d24bd56669" }, "downloads": -1, "filename": "dasladen-0.1.7-py3-none-any.whl", "has_sig": false, "md5_digest": "a435d580961219fd8c227c6ce3cd13da", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 16107, "upload_time": "2019-07-24T03:07:39", "url": "https://files.pythonhosted.org/packages/f1/ff/93223f240349de46b544c5cb54b5dccbd7cd6a627620dab337f15533eae0/dasladen-0.1.7-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "897f0970e4bd66c9faad3a628df7f9a2", "sha256": "ae1fb22a28c183a28bff3e82cb068ff56db8a8dfcd08a7f43feeec3e150bb611" }, "downloads": -1, "filename": "dasladen-0.1.7.tar.gz", "has_sig": false, "md5_digest": "897f0970e4bd66c9faad3a628df7f9a2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13787, "upload_time": "2019-07-24T03:07:41", "url": "https://files.pythonhosted.org/packages/e6/ae/768d802b94beef9349b3130000a6e93a8657c27dea602b68da4576c2be79/dasladen-0.1.7.tar.gz" } ], "0.1.8": [ { "comment_text": "", "digests": { "md5": "25cd6b0b3f268946c4a794ea886fdefd", "sha256": "0d33de6ae8ac21e98ff99fe890a95516b0e60a6966e98ce6a4d550108456bed9" }, "downloads": -1, "filename": "dasladen-0.1.8-py2-none-any.whl", "has_sig": false, "md5_digest": "25cd6b0b3f268946c4a794ea886fdefd", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 16145, "upload_time": "2019-08-04T16:53:02", "url": "https://files.pythonhosted.org/packages/01/dc/670aa7afd7d7b9a329389c820ecb31c9d21d40fece9ec82b898f2eba234b/dasladen-0.1.8-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "cff2278e96bd71d1f8959b725a56cc4d", "sha256": "62f22627c001ba1eaae86560ca4ad1dab774692e5ae0f5a02ad143bc18b4a7b0" }, "downloads": -1, "filename": "dasladen-0.1.8.tar.gz", "has_sig": false, "md5_digest": "cff2278e96bd71d1f8959b725a56cc4d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12216, "upload_time": "2019-08-04T16:53:04", "url": "https://files.pythonhosted.org/packages/91/27/82c4e22ce4e092dda026ebc3089abdbbd69536ff3e204edc5a5e1b7e400e/dasladen-0.1.8.tar.gz" } ], "0.1.9": [ { "comment_text": "", "digests": { "md5": "560604bbc23725dbc27e506a6fa2f13f", "sha256": "4726ef0b4ce6929f83576ac7b8d1e3aa9515d9bfab563f986352763a8bb4c158" }, "downloads": -1, "filename": "dasladen-0.1.9-py2-none-any.whl", "has_sig": false, "md5_digest": "560604bbc23725dbc27e506a6fa2f13f", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 16445, "upload_time": "2019-09-15T02:14:26", "url": "https://files.pythonhosted.org/packages/47/42/2b69a1de5d2343f179688dfbb8ad02cbe040d0656938d6e6dfd52acf42bf/dasladen-0.1.9-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "561a6c31323a254c83a208327c8bef0b", "sha256": "7b57c3be0c3c9279738224c21beefcb83c9d21a30357090f16e6a0fc297de389" }, "downloads": -1, "filename": "dasladen-0.1.9-py3-none-any.whl", "has_sig": false, "md5_digest": "561a6c31323a254c83a208327c8bef0b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 16422, "upload_time": "2019-09-15T02:14:28", "url": "https://files.pythonhosted.org/packages/48/80/ffed065a71880ac34070aa69e8ae0e5b7a5b2f4de05bc65afdbeedbd372c/dasladen-0.1.9-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1c9f26c5180034ad95f48d8ef748dce7", "sha256": "b189ffbdc705d1be9131d9b91e0ee0f341e36909630def76092fb265b185eed2" }, "downloads": -1, "filename": "dasladen-0.1.9.tar.gz", "has_sig": false, "md5_digest": "1c9f26c5180034ad95f48d8ef748dce7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14143, "upload_time": "2019-09-15T02:14:29", "url": "https://files.pythonhosted.org/packages/fc/20/ff90c039072333a330cd15b9a529adcb941c78cf9ef5a93065bcb1ce9bb5/dasladen-0.1.9.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "1277634a879b4250f7da3b6c2a9b125d", "sha256": "12672e34f634ad63e8e70364c37e20f2dd91f39730b6b3fa9c0723fc08776721" }, "downloads": -1, "filename": "dasladen-0.2.0-py2-none-any.whl", "has_sig": false, "md5_digest": "1277634a879b4250f7da3b6c2a9b125d", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 16589, "upload_time": "2019-09-28T23:11:50", "url": "https://files.pythonhosted.org/packages/de/64/138b472e835637ae6fd2ef5631de18159e65052b431b0cfe518809c803fa/dasladen-0.2.0-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c4a5085f9ac98a95f0f54a1871532ef9", "sha256": "64c7cc602e2a86cda6992af2b398b32ae6b982cfd198f1f3e996c45778acfe2f" }, "downloads": -1, "filename": "dasladen-0.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "c4a5085f9ac98a95f0f54a1871532ef9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 16566, "upload_time": "2019-09-28T23:11:52", "url": "https://files.pythonhosted.org/packages/aa/14/1f3f8db89e8b6b1759b0bb64976846069ec2a21c45bcad3ab0dabef5a700/dasladen-0.2.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "337369b66a137b0ffd60627fe082d76a", "sha256": "89a05ca1b89189ac8efff4e09f3399022152fabbe9b4ef0ecb6557e2d685076e" }, "downloads": -1, "filename": "dasladen-0.2.0.tar.gz", "has_sig": false, "md5_digest": "337369b66a137b0ffd60627fe082d76a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14311, "upload_time": "2019-09-28T23:11:54", "url": "https://files.pythonhosted.org/packages/2d/bc/fd9fae8cb6904ba0d431b060e33362e6743d1e80edfedc223ebd6912fe05/dasladen-0.2.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1277634a879b4250f7da3b6c2a9b125d", "sha256": "12672e34f634ad63e8e70364c37e20f2dd91f39730b6b3fa9c0723fc08776721" }, "downloads": -1, "filename": "dasladen-0.2.0-py2-none-any.whl", "has_sig": false, "md5_digest": "1277634a879b4250f7da3b6c2a9b125d", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 16589, "upload_time": "2019-09-28T23:11:50", "url": "https://files.pythonhosted.org/packages/de/64/138b472e835637ae6fd2ef5631de18159e65052b431b0cfe518809c803fa/dasladen-0.2.0-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c4a5085f9ac98a95f0f54a1871532ef9", "sha256": "64c7cc602e2a86cda6992af2b398b32ae6b982cfd198f1f3e996c45778acfe2f" }, "downloads": -1, "filename": "dasladen-0.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "c4a5085f9ac98a95f0f54a1871532ef9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 16566, "upload_time": "2019-09-28T23:11:52", "url": "https://files.pythonhosted.org/packages/aa/14/1f3f8db89e8b6b1759b0bb64976846069ec2a21c45bcad3ab0dabef5a700/dasladen-0.2.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "337369b66a137b0ffd60627fe082d76a", "sha256": "89a05ca1b89189ac8efff4e09f3399022152fabbe9b4ef0ecb6557e2d685076e" }, "downloads": -1, "filename": "dasladen-0.2.0.tar.gz", "has_sig": false, "md5_digest": "337369b66a137b0ffd60627fe082d76a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14311, "upload_time": "2019-09-28T23:11:54", "url": "https://files.pythonhosted.org/packages/2d/bc/fd9fae8cb6904ba0d431b060e33362e6743d1e80edfedc223ebd6912fe05/dasladen-0.2.0.tar.gz" } ] }