{ "info": { "author": "Mike Alfare", "author_email": "alfare@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "License :: OSI Approved :: Apache Software License", "Operating System :: Microsoft :: Windows", "Programming Language :: Python :: 3.7", "Topic :: Database" ], "description": "# Datarade\n\n**This library provides tools to allow users to describe data pipelines in yaml format.**\n\n---\n\n# Overview\n\nThis library separates the 'how' from the 'what' when sourcing datasets and producing data pipelines.\nThe definition of a dataset is stored in a git repository and referenced by name in the client application.\nThis allows the definition to be source controlled independently from the client application.\n\n# Requirements\n\n- Python 3.7+\n- sqlalchemy\n- marshmallow\n- pyyaml\n- pyodbc\n- pymssql\n- bcp\n- requests\n- requests_ntlm\n\n# Installation\n\nThis package is hosted on PyPI:\n\n```shell script\npip install datarade\n```\n\n# Examples\n\nUse the dataset catalog services to obtain datasets:\n```python\nfrom datarade.services.dataset_catalog import api\n\ndataset_repository_url = 'https://raw.githubusercontent.com/mikealfare/dataset_catalog_test/master'\ndataset = api.get_dataset(dataset_name='my_dataset',\n dataset_repository_url=dataset_repository_url,\n dataset_catalog='catalog')\nprint(dataset.definition)\n```\n\nUse the dataset subscription services to move datasets to a database:\n```python\nfrom datarade.services.dataset_subscription import api\n\ndataset_name = 'test_dataset'\ndataset_repository_url = 'https://raw.githubusercontent.com/mikealfare/dataset_catalog_test/master'\napi.register_dataset_container(dataset_container_id='test',\n dataset_repository_url=dataset_repository_url,\n dataset_catalog='catalog',\n driver='mssql',\n database_name='my_db',\n host='my_host',\n port=54321,\n schema_name='my_schema'\n )\napi.add_dataset(dataset_container_id='test',\n dataset_name=dataset_name,\n dataset_username='user',\n dataset_password='password1234')\napi.refresh_dataset(dataset_container_id='test', dataset_name=dataset_name)\n```\n\n# Full Documentation\n\nFor the full documentation, please visit: https://datarade.readthedocs.io/en/latest/\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/mikealfare/datarade", "keywords": "datarade mssql database", "license": "", "maintainer": "", "maintainer_email": "", "name": "datarade", "package_url": "https://pypi.org/project/datarade/", "platform": "", "project_url": "https://pypi.org/project/datarade/", "project_urls": { "Homepage": "https://github.com/mikealfare/datarade" }, "release_url": "https://pypi.org/project/datarade/0.1.2/", "requires_dist": [ "sqlalchemy >=1.3.8,<2.0", "marshmallow >=3.2.0,<4.0", "pyyaml >=5.1.2,<6.0", "pyodbc >=4.0.27,<5.0", "pymssql >=2.1.4,<3.0", "bcp >=0.2.1,<0.3", "requests >=2.22.0,<3.0", "requests_ntlm >=1.1.0,<2.0", "sphinx >=2.2.0,<3.0; extra == \"doc\"", "sphinx-autodoc-typehints >=1.8.0,<2.0; extra == \"doc\"", "sphinx_rtd_theme >=0.4.3,<0.5; extra == \"doc\"", "pytest >=5.1.3,<6.0; extra == \"test\"", "pytest-cov >=2.7.1,<3.0; extra == \"test\"" ], "requires_python": ">=3.7,<4.0", "summary": "This library provides tools to allow users to describe data pipelines in yaml format.", "version": "0.1.2" }, "last_serial": 5992654, "releases": { "0.1.2": [ { "comment_text": "", "digests": { "md5": "410e36c7a6fa5473e2c263488bb20493", "sha256": "2fa74c93666a7f17183fe210d56c6d0cf3b56293748f728cd4db51f3539bf422" }, "downloads": -1, "filename": "datarade-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "410e36c7a6fa5473e2c263488bb20493", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.7,<4.0", "size": 64902, "upload_time": "2019-10-17T22:20:08", "url": "https://files.pythonhosted.org/packages/2c/bf/5b9bc61f04c1e9c1779d060fce679c805f7e24e2730dfc1bcaf932d0ed5c/datarade-0.1.2-py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "410e36c7a6fa5473e2c263488bb20493", "sha256": "2fa74c93666a7f17183fe210d56c6d0cf3b56293748f728cd4db51f3539bf422" }, "downloads": -1, "filename": "datarade-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "410e36c7a6fa5473e2c263488bb20493", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.7,<4.0", "size": 64902, "upload_time": "2019-10-17T22:20:08", "url": "https://files.pythonhosted.org/packages/2c/bf/5b9bc61f04c1e9c1779d060fce679c805f7e24e2730dfc1bcaf932d0ed5c/datarade-0.1.2-py3-none-any.whl" } ] }