{ "info": { "author": "laughingman7743", "author_email": "laughingman7743@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Java", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Database :: Front-Ends" ], "description": ".. image:: https://img.shields.io/pypi/pyversions/PyAthenaJDBC.svg\n :target: https://pypi.python.org/pypi/PyAthenaJDBC/\n\n.. image:: https://travis-ci.org/laughingman7743/PyAthenaJDBC.svg?branch=master\n :target: https://travis-ci.org/laughingman7743/PyAthenaJDBC\n\n.. image:: https://codecov.io/gh/laughingman7743/PyAthenaJDBC/branch/master/graph/badge.svg\n :target: https://codecov.io/gh/laughingman7743/PyAthenaJDBC\n\n.. image:: https://img.shields.io/pypi/l/PyAthenaJDBC.svg\n :target: https://github.com/laughingman7743/PyAthenaJDBC/blob/master/LICENSE\n\n\nPyAthenaJDBC\n============\n\nPyAthenaJDBC is a Python `DB API 2.0 (PEP 249)`_ compliant wrapper for `Amazon Athena JDBC driver`_.\n\n.. _`DB API 2.0 (PEP 249)`: https://www.python.org/dev/peps/pep-0249/\n.. _`Amazon Athena JDBC driver`: http://docs.aws.amazon.com/athena/latest/ug/connect-with-jdbc.html\n\nRequirements\n------------\n\n* Python\n\n - CPython 2,7, 3,4, 3.5, 3.6\n\n* Java\n\n - Java >= 8\n\nInstallation\n------------\n\n.. code:: bash\n\n $ pip install PyAthenaJDBC\n\nExtra packages:\n\n+---------------+------------------------------------------+----------+\n| Package | Install command | Version |\n+===============+==========================================+==========+\n| Pandas | ``pip install PyAthenaJDBC[Pandas]`` | >=0.19.0 |\n+---------------+------------------------------------------+----------+\n| SQLAlchemy | ``pip install PyAthenaJDBC[SQLAlchemy]`` | >=1.0.0 |\n+---------------+------------------------------------------+----------+\n\nUsage\n-----\n\nBasic usage\n~~~~~~~~~~~\n\n.. code:: python\n\n from pyathenajdbc import connect\n\n conn = connect(s3_staging_dir='s3://YOUR_S3_BUCKET/path/to/',\n region_name='us-west-2')\n try:\n with conn.cursor() as cursor:\n cursor.execute(\"\"\"\n SELECT * FROM one_row\n \"\"\")\n print(cursor.description)\n print(cursor.fetchall())\n finally:\n conn.close()\n\nCursor iteration\n~~~~~~~~~~~~~~~~\n\n.. code:: python\n\n from pyathenajdbc import connect\n\n conn = connect(s3_staging_dir='s3://YOUR_S3_BUCKET/path/to/',\n region_name='us-west-2')\n try:\n with conn.cursor() as cursor:\n cursor.execute(\"\"\"\n SELECT * FROM many_rows LIMIT 10\n \"\"\")\n for row in cursor:\n print(row)\n finally:\n conn.close()\n\nQuery with parameter\n~~~~~~~~~~~~~~~~~~~~\n\nSupported `DB API paramstyle`_ is only ``PyFormat``.\n``PyFormat`` only supports `named placeholders`_ with old ``%`` operator style and parameters specify dictionary format.\n\n.. code:: python\n\n from pyathenajdbc import connect\n\n conn = connect(s3_staging_dir='s3://YOUR_S3_BUCKET/path/to/',\n region_name='us-west-2')\n try:\n with conn.cursor() as cursor:\n cursor.execute(\"\"\"\n SELECT col_string FROM one_row_complex\n WHERE col_string = %(param)s\n \"\"\", {'param': 'a string'})\n print(cursor.fetchall())\n finally:\n conn.close()\n\nif ``%`` character is contained in your query, it must be escaped with ``%%`` like the following:\n\n.. code:: sql\n\n SELECT col_string FROM one_row_complex\n WHERE col_string = %(param)s OR col_string LIKE 'a%%'\n\n.. _`DB API paramstyle`: https://www.python.org/dev/peps/pep-0249/#paramstyle\n.. _`named placeholders`: https://pyformat.info/#named_placeholders\n\nJVM Options\n~~~~~~~~~~~\n\nIn the connect method or connection object, you can specify JVM options with a string array.\n\nYou can increase the JVM heap size like the following:\n\n.. code:: python\n\n from pyathenajdbc import connect\n\n conn = connect(s3_staging_dir='s3://YOUR_S3_BUCKET/path/to/',\n region_name='us-west-2',\n jvm_options=['-Xms1024m', '-Xmx4096m'])\n try:\n with conn.cursor() as cursor:\n cursor.execute(\"\"\"\n SELECT * FROM many_rows\n \"\"\")\n print(cursor.fetchall())\n finally:\n conn.close()\n\nSQLAlchemy\n~~~~~~~~~~\n\nInstall SQLAlchemy with ``pip install SQLAlchemy>=1.0.0`` or ``pip install PyAthenaJDBC[SQLAlchemy]``.\nSupported SQLAlchemy is 1.0.0 or higher.\n\n.. code:: python\n\n import contextlib\n from urllib.parse import quote_plus # PY2: from urllib import quote_plus\n from sqlalchemy.engine import create_engine\n from sqlalchemy.sql.expression import select\n from sqlalchemy.sql.functions import func\n from sqlalchemy.sql.schema import Table, MetaData\n\n conn_str = 'awsathena+jdbc://{access_key}:{secret_key}@athena.{region_name}.amazonaws.com:443/'\\\n '{schema_name}?s3_staging_dir={s3_staging_dir}'\n engine = create_engine(conn_str.format(\n access_key=quote_plus('YOUR_ACCESS_KEY'),\n secret_key=quote_plus('YOUR_SECRET_ACCESS_KEY'),\n region_name='us-west-2',\n schema_name='default',\n s3_staging_dir=quote_plus('s3://YOUR_S3_BUCKET/path/to/')))\n try:\n with contextlib.closing(engine.connect()) as conn:\n many_rows = Table('many_rows', MetaData(bind=engine), autoload=True)\n print(select([func.count('*')], from_obj=many_rows).scalar())\n finally:\n engine.dispose()\n\nThe connection string has the following format:\n\n.. code:: python\n\n awsathena+jdbc://{access_key}:{secret_key}@athena.{region_name}.amazonaws.com:443/{schema_name}?s3_staging_dir={s3_staging_dir}&driver_path={driver_path}&...\n\nNOTE: ``s3_staging_dir`` requires quote. If ``access_key``, ``secret_key`` and other parameter contain special characters, quote is also required.\n\nPandas\n~~~~~~\n\nMinimal example for Pandas DataFrame:\n\n.. code:: python\n\n from pyathenajdbc import connect\n import pandas as pd\n\n conn = connect(access_key='YOUR_ACCESS_KEY_ID',\n secret_key='YOUR_SECRET_ACCESS_KEY',\n s3_staging_dir='s3://YOUR_S3_BUCKET/path/to/',\n region_name='us-west-2',\n jvm_path='/path/to/jvm') # optional, as used by JPype\n df = pd.read_sql(\"SELECT * FROM many_rows LIMIT 10\", conn)\n\nAs Pandas DataFrame:\n\n.. code:: python\n\n import contextlib\n from pyathenajdbc import connect\n from pyathenajdbc.util import as_pandas\n\n with contextlib.closing(\n connect(s3_staging_dir='s3://YOUR_S3_BUCKET/path/to/'\n region_name='us-west-2'))) as conn:\n with conn.cursor() as cursor:\n cursor.execute(\"\"\"\n SELECT * FROM many_rows\n \"\"\")\n df = as_pandas(cursor)\n print(df.describe())\n\nExamples\n--------\n\nRedash_ query runner example\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nSee `examples/redash/athena.py`_\n\n.. _Redash: https://github.com/getredash/redash\n.. _`examples/redash/athena.py`: examples/redash/athena.py\n\nCredential\n----------\n\nSupport `AWS CLI credentials`_, `Properties file credentials`_ and `AWS credentials provider chain`_.\n\n.. _`AWS CLI credentials`: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html\n.. _`Properties file credentials`: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/PropertiesFileCredentialsProvider.html\n.. _`AWS credentials provider chain`: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/DefaultAWSCredentialsProviderChain.html\n\nCredential Files\n~~~~~~~~~~~~~~~~\n\n~/.aws/credentials\n\n.. code:: cfg\n\n [default]\n aws_access_key_id=YOUR_ACCESS_KEY_ID\n aws_secret_access_key=YOUR_SECRET_ACCESS_KEY\n\n~/.aws/config\n\n.. code:: cfg\n\n [default]\n region=us-west-2\n output=json\n\nEnvironment variables\n~~~~~~~~~~~~~~~~~~~~~\n\n.. code:: bash\n\n $ export AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID\n $ export AWS_SECRET_ACCESS_KEY=YOUR_SECRET_ACCESS_KEY\n $ export AWS_DEFAULT_REGION=us-west-2\n\nAdditional environment variable:\n\n.. code:: bash\n\n $ export AWS_ATHENA_S3_STAGING_DIR=s3://YOUR_S3_BUCKET/path/to/\n\nProperties file credentials\n~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nCreate a property file of the following format.\n\n/path/to/AWSCredentials.properties\n\n.. code:: properties\n\n accessKeyId:YOUR_ACCESS_KEY_ID\n secretKey:YOUR_SECRET_ACCESS_KEY\n\nSpecify the property file path with ``credential_file`` of the connect method or connection object.\n\n.. code:: python\n\n from pyathenajdbc import connect\n\n conn = connect(credential_file='/path/to/AWSCredentials.properties',\n s3_staging_dir='s3://YOUR_S3_BUCKET/path/to/',\n region_name='us-west-2')\n\nPyAthenaJDBC uses the property file to authenticate Amazon Athena.\n\nAWS credentials provider chain\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nSee `AWS credentials provider chain`_\n\n AWS credentials provider chain that looks for credentials in this order:\n\n * Environment Variables - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (RECOMMENDED since they are recognized by all the AWS SDKs and CLI except for .NET), or AWS_ACCESS_KEY and AWS_SECRET_KEY (only recognized by Java SDK)\n * Java System Properties - aws.accessKeyId and aws.secretKey\n * Credential profiles file at the default location (~/.aws/credentials) shared by all AWS SDKs and the AWS CLI\n * Credentials delivered through the Amazon EC2 container service if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI\" environment variable is set and security manager has permission to access the variable,\n * Instance profile credentials delivered through the Amazon EC2 metadata service\n\nIn the connect method or connection object, you can connect by specifying at least ``s3_staging_dir`` and ``region_name``.\nIt is not necessary to specify ``access_key`` and ``secret_key``.\n\n.. code:: python\n\n from pyathenajdbc import connect\n\n conn = connect(s3_staging_dir='s3://YOUR_S3_BUCKET/path/to/',\n region_name='us-west-2')\n\nTerraform_ Instance profile example:\n\nSee `examples/terraform/`_\n\n.. _Terraform: https://github.com/hashicorp/terraform\n.. _`examples/terraform/`: examples/terraform/\n\n\nTesting\n-------\n\nDepends on the following environment variables:\n\n.. code:: bash\n\n $ export AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID\n $ export AWS_SECRET_ACCESS_KEY=YOUR_SECRET_ACCESS_KEY\n $ export AWS_DEFAULT_REGION=us-west-2\n $ export AWS_ATHENA_S3_STAGING_DIR=s3://YOUR_S3_BUCKET/path/to/\n\nRun test\n~~~~~~~~\n\n.. code:: bash\n\n $ pip install pipenv\n $ pipenv install --dev\n $ pipenv run scripts/test_data/upload_test_data.sh\n $ pipenv run pytest\n $ pipenv run scripts/test_data/delete_test_data.sh\n\nRun test multiple Python versions\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n.. code:: bash\n\n $ pip install pipenv\n $ pipenv install --dev\n $ pipenv run scripts/test_data/upload_test_data.sh\n $ pyenv local 3.6.5 3.5.5 3.4.8 2.7.14\n $ pipenv run tox\n $ pipenv run scripts/test_data/delete_test_data.sh\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/tanaya1234/PyAthenaJDBC-1", "keywords": "", "license": "MIT License", "maintainer": "", "maintainer_email": "", "name": "PyAthenaJDBC-qubole", "package_url": "https://pypi.org/project/PyAthenaJDBC-qubole/", "platform": "", "project_url": "https://pypi.org/project/PyAthenaJDBC-qubole/", "project_urls": { "Homepage": "https://github.com/tanaya1234/PyAthenaJDBC-1" }, "release_url": "https://pypi.org/project/PyAthenaJDBC-qubole/1.3.4/", "requires_dist": [ "future", "jpype1 (>=0.6.0)", "botocore (>=1.0.0)", "pandas (>=0.19.0); extra == 'Pandas'", "SQLAlchemy (>=1.0.0); extra == 'SQLAlchemy'" ], "requires_python": "", "summary": "Python DB API 2.0 (PEP 249) compliant wrapper for Amazon Athena JDBC driver", "version": "1.3.4" }, "last_serial": 4208591, "releases": { "1.3.4": [ { "comment_text": "", "digests": { "md5": "f56411e45dbf3568ce3b74825e44588f", "sha256": "383cccdffe8b54677a2000323ff5ccbfa8fcd0a2138e2eb58e6dea655f688073" }, "downloads": -1, "filename": "PyAthenaJDBC_qubole-1.3.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f56411e45dbf3568ce3b74825e44588f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 24769, "upload_time": "2018-08-26T16:05:50", "url": "https://files.pythonhosted.org/packages/6e/32/66d70e6609ff86c9ba6e8dfd335c88e82a947f3d58ce80ace45ac6d84c65/PyAthenaJDBC_qubole-1.3.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a10be159c9767fb93bfe150b3f95a258", "sha256": "972a36244425aa1929299c4e1438473141b81cc09e7a0af3bd6d3a48115a093b" }, "downloads": -1, "filename": "PyAthenaJDBC_qubole-1.3.4.tar.gz", "has_sig": false, "md5_digest": "a10be159c9767fb93bfe150b3f95a258", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22665, "upload_time": "2018-08-26T16:05:52", "url": "https://files.pythonhosted.org/packages/dc/fa/25a6344e917801c7cafb52962b13feff01b62517f847d9f330e9e53c3eef/PyAthenaJDBC_qubole-1.3.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "f56411e45dbf3568ce3b74825e44588f", "sha256": "383cccdffe8b54677a2000323ff5ccbfa8fcd0a2138e2eb58e6dea655f688073" }, "downloads": -1, "filename": "PyAthenaJDBC_qubole-1.3.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f56411e45dbf3568ce3b74825e44588f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 24769, "upload_time": "2018-08-26T16:05:50", "url": "https://files.pythonhosted.org/packages/6e/32/66d70e6609ff86c9ba6e8dfd335c88e82a947f3d58ce80ace45ac6d84c65/PyAthenaJDBC_qubole-1.3.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a10be159c9767fb93bfe150b3f95a258", "sha256": "972a36244425aa1929299c4e1438473141b81cc09e7a0af3bd6d3a48115a093b" }, "downloads": -1, "filename": "PyAthenaJDBC_qubole-1.3.4.tar.gz", "has_sig": false, "md5_digest": "a10be159c9767fb93bfe150b3f95a258", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22665, "upload_time": "2018-08-26T16:05:52", "url": "https://files.pythonhosted.org/packages/dc/fa/25a6344e917801c7cafb52962b13feff01b62517f847d9f330e9e53c3eef/PyAthenaJDBC_qubole-1.3.4.tar.gz" } ] }