{ "info": { "author": "Nick Vogt", "author_email": "nick.vogt@dynatrace.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8" ], "description": "Alyeska /al-ee-EHS-kah/ n. A Data Engineering Toolkit\n=====================================================\n\n.. image:: https://img.shields.io/pypi/pyversions/alyeska.svg\n :alt: Supported Versions\n :target: https://pypi.org/project/alyeska/\n\n.. image:: https://img.shields.io/badge/Licence-Apache%202.0-blue.svg\n :alt: License\n :target: ./LICENSE\n\n.. image:: https://readthedocs.org/projects/alyeska/badge/?version=latest\n :alt: Documentation Status\n :target: https://alyeska.readthedocs.io/en/latest/?badge=latest\n\n.. image:: https://img.shields.io/pypi/v/alyeska.svg?color=blue\n :alt: Version\n :target: https://pypi.org/project/alyeska/\n\n.. image:: https://img.shields.io/pypi/dw/alyeska.svg\n :alt: PyPI - Downloads\n :target: https://pypi.org/project/alyeska/\n\n.. image:: https://img.shields.io/github/last-commit/Dynatrace/alyeska.svg\n :alt: Last Commit\n :target: https://github.com/Dynatrace/alyeska\n\n.. image:: https://img.shields.io/github/issues/Dynatrace/alyeska.svg\n :alt: Count Open Issues\n :target: https://pypi.org/project/alyeska/\n\nAlyeska is a data engineering toolkit to simplify the nuts & bolts of data engineering tasks.\n\nMore concretely, alyeska bridges the gap between common Python modules and common data engineering technologies. i.e. pandas, psycopg2, AWS Redshift, AWS secretsmanager, and more. Alyeska offers simple functions and/or syntactic sugar to common tasks:\n\n* Safely executing many SQL statements against a database (``sqlagent``)\n* Loading a pandas dataframe into a database (``redpandas``)\n* Assuming an AWS IAM user with multi-factor authorization (``locksmith.authmfa``)\n* Creating psycopg2 connections to Redshift (``locksmith``)\n* Generate shell scripts that respect workflow dependencies (``compose.compose_sh``)\n\nWhile Alyeska mimics some functionalities, it is not a replacement for Airflow, AWS Glue, or other purpose-built data engineering technologies. That is, metaphorically, Alyeska is your parents' toolbox. While terrific for fixing a leaky faucet, but it is no replacement for a plumber.\n\nSample Usage\n------------\n\n**Assume an AWS IAM user with multi-factor authorization**\n\nAlyeska's ``authmfa`` command line utility is useful for quickly assuming an AWS IAM user with MFA.\n\n.. code-block:: sh\n\n $ authmfa MyAwsUser\n export AWS_ACCESS_KEY_ID=ABCDEFGHIJKLMNOPQRSTUVWXYZ\n export AWS_SECRET_ACCESS_KEY=abcdefg1234567!@#$%^&\n export AWS_SESSION_TOKEN=notarealsessiontoken///////5AVHwuGc*hYLp%$vr51*XTEHJjRD2JxavaD8wlJqi!aCZVhvp7nzt!U5elvoPZ@GlG%a9sT^HBrgKzQ8xZrpAADp65RYQzqvawF\n $ eval `authmfa MyAwsUser` # export to environment\n\nLearn more about how to config this utility with ``authmfa -h``.\n\n**Load a pandas dataframe into AWS Redshift**\n\nLarge tables can be frustrating to load into Redshift. Alyeska reduces the process to a short one-line statement:\n\n.. code-block:: python\n\n >>> aly.redpandas.insert_pandas_into(cnxn, target_table, payload_df)\n\nIn practice, it may function as\n\n.. code-block:: python\n\n >>> import alyeska as aly\n >>> import alyeska.locksmith.redshift as rs\n >>> import pandas as pd\n\n >>> cnxn = rs.connect_with_environment(\"my-user\")\n >>> target_table = \"db.natural_numbers\"\n\n >>> sql = f\"CREATE TABLE {target_table}(n INT NOT NULL)\"\n >>> aly.sqlagent.execute_sql(cnxn, sql) # create table\n\n >>> natural_numbers_df = pd.DataFrame({\"n\": range(1, 1_000_001)})\n >>> aly.redpandas.insert_pandas_into(cnxn, target_table, natural_numbers_df)\n\nComponents\n----------\n\nTools are broken out into modules with niche purposes:\n\n1. ``compose`` is a workflow dependency management tool\n2. ``locksmith`` helps authorize AWS sessions and Redshift connections\n3. ``logging`` is another thin module that standardizes logging practices\n4. ``redpandas`` supports less verbose pandas/redshift functionality\n5. ``sqlagent`` supports SQL executation and runtime configuration\n\nLicense\n-------\n\nThis project is licensed under the Apache v2.0 License - see the LICENSE_ file for details.\n\nContribute\n----------\n\nBegin by reading our `Code of Conduct`_.\n\nThere are some devtools required to contribute to the repo. Create a development environment and install pre-commit to run hooks on your code.\n\n.. code-block:: sh\n\n $ python3 -m venv .venv\n $ source .venv/bin/activate\n $ pip install -r requirements.txt\n $ pip install -r requirements.dev.txt\n $ pre-commit install\n $ pre-commit autoupdate\n\nNamesake\n--------\n\nThe Alyeska Pipeline Service company maintains the Alaska pipeline; a 1200 km long pipeline connecting the oil-rich, subterranean earth in Alaska to port on the north pacific ocean.\n\n.. _LICENSE: https://github.com/Dynatrace/alyeska/blob/master/LICENSE\n.. _Code of Conduct: https://github.com/Dynatrace/alyeska/blob/master/code-of-conduct.rst\n\n\n", "description_content_type": "text/x-rst", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/Dynatrace/alyeska", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "alyeska", "package_url": "https://pypi.org/project/alyeska/", "platform": "", "project_url": "https://pypi.org/project/alyeska/", "project_urls": { "Homepage": "https://github.com/Dynatrace/alyeska" }, "release_url": "https://pypi.org/project/alyeska/0.3.0a1/", "requires_dist": [ "boto3 (>=1.9.197)", "packaging (>=19.2)", "pandas (>=0.25.0)", "psycopg2-binary (>=2.8.3)", "pyotp (>=2.3.0)", "pytz (>=2019.1)", "pyyaml (>=5.1)" ], "requires_python": "", "summary": "Alyeska /al-ee-EHS-kah/ n. A Data Pipeline Toolkit", "version": "0.3.0a1" }, "last_serial": 5951600, "releases": { "0.2.0a0": [ { "comment_text": "", "digests": { "md5": "7494d3f0cdfee5e1280dbee288e403c9", "sha256": "5fc75891f3595db28ccad8342ebd6bc4590a82774e35840e5e5de366b98e1f67" }, "downloads": -1, "filename": "alyeska-0.2.0a0-py3-none-any.whl", "has_sig": false, "md5_digest": "7494d3f0cdfee5e1280dbee288e403c9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 32586, "upload_time": "2019-09-11T13:54:25", "url": "https://files.pythonhosted.org/packages/cd/c8/623780d1da6dd33b47c04f94beb5d0a92f91507554aea879e86e7c5b8c18/alyeska-0.2.0a0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9db3705f85bd31f1f3f822b8abc4fe72", "sha256": "146f97546a73807f4c1f7780aa3fc9192d5f132389dcdb2a9230681863b943a0" }, "downloads": -1, "filename": "alyeska-0.2.0a0.tar.gz", "has_sig": false, "md5_digest": "9db3705f85bd31f1f3f822b8abc4fe72", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 21819, "upload_time": "2019-09-11T13:54:28", "url": "https://files.pythonhosted.org/packages/d5/34/0fdb26f2b2c5d7bbb0f70f46753474a297a12cacb8bb03090a55520a8c0e/alyeska-0.2.0a0.tar.gz" } ], "0.3.0a0": [ { "comment_text": "", "digests": { "md5": "d254666250cc05b44f40397254efb8b4", "sha256": "8318160f23da083568cf5bca398eff9dff7c73d0b91b2388eecbf67e12a19f26" }, "downloads": -1, "filename": "alyeska-0.3.0a0-py3-none-any.whl", "has_sig": false, "md5_digest": "d254666250cc05b44f40397254efb8b4", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 32769, "upload_time": "2019-10-07T17:03:12", "url": "https://files.pythonhosted.org/packages/f7/1e/39bdc8f84b2323628c3ff7adb76c6d3293ed82484df8a97aa2e7a53f5311/alyeska-0.3.0a0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "917e0938d8ca70cc44cfa3d1f9bc9335", "sha256": "017af34dadc203bda91831bf60b76e43eab2121b4b8627be538858a75123b9eb" }, "downloads": -1, "filename": "alyeska-0.3.0a0.tar.gz", "has_sig": false, "md5_digest": "917e0938d8ca70cc44cfa3d1f9bc9335", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22020, "upload_time": "2019-10-07T17:03:14", "url": "https://files.pythonhosted.org/packages/0b/c8/425ae5944c47a91a3ab7549d361060ca3bf0bce4d50d1d54f670f7d882d6/alyeska-0.3.0a0.tar.gz" } ], "0.3.0a1": [ { "comment_text": "", "digests": { "md5": "bfd4249e744bcc71b387537c62ab259a", "sha256": "e5d6daec43d5b3ec3fd1bc393fccc085a6203a4a5fce466038aa14a42c9528ab" }, "downloads": -1, "filename": "alyeska-0.3.0a1-py3-none-any.whl", "has_sig": false, "md5_digest": "bfd4249e744bcc71b387537c62ab259a", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 32771, "upload_time": "2019-10-09T19:30:30", "url": "https://files.pythonhosted.org/packages/e1/36/97ef015678ae558f8b5de8d8f37d03b6563609c84879f7ab81e279668369/alyeska-0.3.0a1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9262189af6607085914c3ab582aa6cc0", "sha256": "524f5e24ee88071247c1349eecd37d7657c4e043db2879d6f8649192fb515a00" }, "downloads": -1, "filename": "alyeska-0.3.0a1.tar.gz", "has_sig": false, "md5_digest": "9262189af6607085914c3ab582aa6cc0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22004, "upload_time": "2019-10-09T19:30:32", "url": "https://files.pythonhosted.org/packages/de/98/df67accdc8318702fd837b158fb05aed28d61296ca292dfcfb4497661b22/alyeska-0.3.0a1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "bfd4249e744bcc71b387537c62ab259a", "sha256": "e5d6daec43d5b3ec3fd1bc393fccc085a6203a4a5fce466038aa14a42c9528ab" }, "downloads": -1, "filename": "alyeska-0.3.0a1-py3-none-any.whl", "has_sig": false, "md5_digest": "bfd4249e744bcc71b387537c62ab259a", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 32771, "upload_time": "2019-10-09T19:30:30", "url": "https://files.pythonhosted.org/packages/e1/36/97ef015678ae558f8b5de8d8f37d03b6563609c84879f7ab81e279668369/alyeska-0.3.0a1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9262189af6607085914c3ab582aa6cc0", "sha256": "524f5e24ee88071247c1349eecd37d7657c4e043db2879d6f8649192fb515a00" }, "downloads": -1, "filename": "alyeska-0.3.0a1.tar.gz", "has_sig": false, "md5_digest": "9262189af6607085914c3ab582aa6cc0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22004, "upload_time": "2019-10-09T19:30:32", "url": "https://files.pythonhosted.org/packages/de/98/df67accdc8318702fd837b158fb05aed28d61296ca292dfcfb4497661b22/alyeska-0.3.0a1.tar.gz" } ] }