{ "info": { "author": "Vincent Feng", "author_email": "ivincentfeng@live.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Framework :: Jupyter", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python :: 3.6" ], "description": "\n# Azure Data Service Notebook (Alpha)\nAzure Data Service Notebook is a set of extentions for working with Azure Data Service (e.g. Azure Data Lake, HDIsight, CosmosDB, Azure SQL and Azure Data Warehouse etc.) using [Jupyter Notebook](http://jupyter.org/).\n\n**WARNING**: This SDK/CLI is currently in very early stage of development. It can and will change in backwards incompatible ways. \n\nLatest Version: `0.0.1a0` \n# Feature\nAzure Data Service Notebook currently provides a set of [Jupyter Magic Functions](https://ipython.readthedocs.io/en/stable/interactive/tutorial.html?#magic-functions) for users to access Azure Data Lake. Available magics are captured in the table below. Please click on the command name to see the syntax reference.\n\n| Command | Function |\n|-----|-----|\n|[%adl login](#adllogin) |Line magic[*](#myfootnote1) to log in to Azure Data Lake.|\n|[%adl listaccounts](#adllistaccounts)|Line magic to list the Azure Data Lake Analytic accounts for current user.|\n|[%adl listjobs](#adllistjobs)|Line magic to list the Azure Data Lake jobs for a given account.|\n|[%%adl submitjob](#adlsubmitjob)|Cell magic[*](#myfootnote1) to submit a USQL job to Azure Data Lake cluster.|\n|[%adl viewjob](#adlviewjob)|Line magic to view detailed job info.|\n|[%adl liststoreaccounts](#adlliststoreaccounts)|Line magic to list the Azure Data Lake Store accounts.|\n|[%adl liststorefolders](#adlliststorefolders)|Line magic to list the folders under a given directory.|\n|[%adl liststorefiles](#adlliststorefiles)|Line magic to list the files under a given directory.|\n|[%adl sample](#adlsample)|Line magic to sample a given file, return results as Pandas DataFrame.|\n|[%adl logout](#adllogout)|Line magic to log out.|\n\n*Please check [Magic Functions ](https://ipython.readthedocs.io/en/stable/interactive/tutorial.html?#magic-functions) for detailed definiton of `Line magic` and `Cell magics`.\n\n# Installation\n- Download and Install [python 3.6+](https://www.python.org/downloads/)\n- Install jupyter: \n\t\t```pip install jupyter``` \n- Install adlmagic extention : \n\t\t```pip install --no-cache-dir adlmagics```\n# Examples\n- [adlmagics_demo.ipynb](/adlmagics/00_adlmagics_demo.ipynb), demo file of `adlmgics` functions for Azure Data Lake job control and data exploration.\n- [usql_samples.ipynb](/adlmagics/01_usql_samples.ipynb), samples code of common U-SQL scenarios, e.g. query a TSV file, create a database, populate table, query table and create rowset in script.\n\n# Feedback\n- You can submit [bug report](https://github.com/Azure/Azure-Data-Service-Notebook/issues/new?template=bug_report.md) or [feature request](https://github.com/Azure/Azure-Data-Service-Notebook/issues/new?template=feature_request.md) directly in this repo. Our team will triage issues actively.\n\n# Reference\n\n\n\n## %adl login\n\nLine magic to login to Azure Data Lake service.\n\n```\n%adl login --tenant \n```\n\n#### Input Parameters\n\n|Name|Type |Example|Description|\n|----|----|----|----|\n|tenant `required`|string|microsoft.onmicrosoft.com| The value of this argument can either be an `.onmicrosoft.com` domain or the Azure object ID for the tenant. |\n\n\n\n## %adl listaccounts\n\nLine magic to enumerate the Azure Data Lake Analytic accounts for current user. The account list will be returned as Pandas DataFrame, you can call Pandas funtions directly afterward.\n\n```\n%adl listaccounts\t--page_index\n\t\t\t--page_account_number\n```\n\n#### Input Parameters\n\n|Name|Type |Example|Description|\n|----|----|----|----|\n|page_index `required`|int|0|The result page number. This must be greater than 0. Default value is 0.|\n|page_account_number `required`|int|10|The number of results per page.|\n\n\n\n## %adl listjobs\n\nLine magic to enumerate the Azure Data Lake jobs for a given account. The job list will be returned as Pandas DataFrame, you can call Pandas funtions directly afterward.\n\n```\n%adl listjobs\t--account \n\t\t--page_index\n\t\t--page_account_number\n```\n\n#### Input Parameters\n\n|Name|Type |Example|Description|\n|----|----|----|----|\n|account `required`|string|telemetryadla|The Azure Data Lake Analytics account to list the job from.|\n|page_index `required`|int|0|The result page number. This must be greater than 0. Default value is 0.|\n|page_account_number `required`|int|10|The number of results per page.|\n\n\n\n## %%adl submitjob\n\nCell magic to submit a U-SQL job to Azure Data Lake cluster.\n\n```\n%%adl submitjob\t--account \n\t\t--name \n\t\t--parallelism\n\t\t--priority\n\t\t--runtime\n```\n#### Input Parameters\n\n|Name|Type |Example|Description|\n|----|----|----|----|\n|account `required`|string|telemetryadla|the Azure Data Lake Analytics account to execute job operations on.|\n|name `required`|string|myscript|the friendly name of the job to submit.|\n|parallelism |int|5|the degree of parallelism used for this job. This must be greater than 0, if set to less than 0 it will default to 1.|\n|priority|int|1000|the priority value for the current job. Lower numbers have a higher priority. By default, a job has a priority of 1000. This must be greater than 0.|\n|runtime|string|default|the runtime version of the Data Lake Analytics engine to use for the specific type of job being run.|\n\n\n\n## %adl viewjob\n\nLine magic to view detailed job info.\n```\n%adl view job\t--account \n\t\t--job_id \n```\n#### Input Parameters\n\n|Name|Type |Example|Description|\n|----|----|----|----|\n|account `required`|string|telemetryadla|the Azure Data Lake Analytics account to execute job operations on.|\n|job_id `required`|GUID|36a62f78-1881-1935-8a6a-9e37b497582d|job identifier. uniquely identifies the job across all jobs submitted to the service.|\n\n\n\n## %adl liststoreacconts\t\n\nLine magic to list the Azure Data Lake Store accounts.\n```\n%adl liststoreaccounts\n```\n\n\n\n## %adl liststorefolders\nLine magic to list the folders under a given directory.\n```\n%adl liststorefolders\t--account \n\t\t\t--folder_path \n```\n#### Input Parameters\n\n|Name|Type |Example|Description|\n|----|----|----|----|\n|account `required`|string|telemetryadls|the name of the Data Lake Store account.|\n|folder_path `required`|string|root/data|the directory path under the Data Lake Store account.|\n\n\n\n## %adl liststorefiles\nLine magic to list the files under a given directory.\n```\n%adl liststorefiles\t--account \n\t\t\t--folder_path\n```\n#### Input Parameters\n\n|Name|Type |Example|Description|\n|----|----|----|----|\n|account `required`|string|telemetryadls|the name of the Data Lake Store account.|\n|folder_path `required`|string|root/data|the directory path under the Data Lake Store account.|\n\n\n\n## %adl sample\n\nLine magic to sample a given file, return results as Pandas DataFrame.\n\n```\n%adl sample\t--account \n\t\t--file_path \n\t\t--file_type \n\t\t--encoding \n\t\t--row_number \n```\n#### Input Parameters\n\n|Name|Type |Example|Description|\n|----|----|----|----|\n|account `required`|string|telemetryadls|the name of the Data Lake Store account.|\n|file_path `required`|string|root/data/sample.tsv|the file path to sample data from.|\n|file_type |string|tsv|the type of the file to sample from. |\n|encoding |string|UTF-8|encoding type of the file.|\n|row_number|int|10|number of rows to read from the file.|\n\n\n\n## %adl logout\n\nLine magic to log out.\n\n```\n%adl logout\n```\n\n\n# Contributing\n\nThis project welcomes contributions and suggestions. Most contributions require you to agree to a\nContributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us\nthe rights to use your contribution. For details, visit https://cla.microsoft.com.\n\nWhen you submit a pull request, a CLA-bot will automatically determine whether you need to provide\na CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions\nprovided by the bot. You will only need to do this once across all repos using our CLA.\n\nThis project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).\nFor more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or\ncontact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "https://github.com/Azure/Azure-Data-Service-Notebook", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/Azure/Azure-Data-Service-Notebook", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "adlmagics", "package_url": "https://pypi.org/project/adlmagics/", "platform": "", "project_url": "https://pypi.org/project/adlmagics/", "project_urls": { "Download": "https://github.com/Azure/Azure-Data-Service-Notebook", "Homepage": "https://github.com/Azure/Azure-Data-Service-Notebook" }, "release_url": "https://pypi.org/project/adlmagics/0.0.1a2/", "requires_dist": [ "ipython (>=5.3.0)", "notebook (>=5.0.0)", "ipykernel (>=4.6.1)", "pandas (>=0.20.1)", "numpy", "adal", "azure-mgmt-resource", "azure-mgmt-datalake-analytics (>=0.4.0)", "azure-mgmt-datalake-store", "azure-datalake-store" ], "requires_python": "", "summary": "Azure Data Lake management magics for Jupyter Notebook", "version": "0.0.1a2" }, "last_serial": 3985009, "releases": { "0.0.1a2": [ { "comment_text": "", "digests": { "md5": "12193ddda9e48613296d174765d6b769", "sha256": "eb23dce01e37c42322ac1d8e0557df008c33eac5e5df4dac17956f9d7c7c8db7" }, "downloads": -1, "filename": "adlmagics-0.0.1a2-py3-none-any.whl", "has_sig": false, "md5_digest": "12193ddda9e48613296d174765d6b769", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 23491, "upload_time": "2018-06-21T09:02:02", "url": "https://files.pythonhosted.org/packages/58/69/69f0e784f9e2e48a505ab974d032d784beb98cc6aeddb2c72c97c6845331/adlmagics-0.0.1a2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0823c620381681310548b9e6c0624793", "sha256": "5ce794a7acda0c795411c9237609dd77378528dea75b63c85a98ebf155f1bdea" }, "downloads": -1, "filename": "adlmagics-0.0.1a2.tar.gz", "has_sig": false, "md5_digest": "0823c620381681310548b9e6c0624793", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12682, "upload_time": "2018-06-21T09:02:03", "url": "https://files.pythonhosted.org/packages/7a/2a/80a22b988f103079bd047ffe2eb156a2356a85e94032bb847d89cd7244ef/adlmagics-0.0.1a2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "12193ddda9e48613296d174765d6b769", "sha256": "eb23dce01e37c42322ac1d8e0557df008c33eac5e5df4dac17956f9d7c7c8db7" }, "downloads": -1, "filename": "adlmagics-0.0.1a2-py3-none-any.whl", "has_sig": false, "md5_digest": "12193ddda9e48613296d174765d6b769", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 23491, "upload_time": "2018-06-21T09:02:02", "url": "https://files.pythonhosted.org/packages/58/69/69f0e784f9e2e48a505ab974d032d784beb98cc6aeddb2c72c97c6845331/adlmagics-0.0.1a2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0823c620381681310548b9e6c0624793", "sha256": "5ce794a7acda0c795411c9237609dd77378528dea75b63c85a98ebf155f1bdea" }, "downloads": -1, "filename": "adlmagics-0.0.1a2.tar.gz", "has_sig": false, "md5_digest": "0823c620381681310548b9e6c0624793", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12682, "upload_time": "2018-06-21T09:02:03", "url": "https://files.pythonhosted.org/packages/7a/2a/80a22b988f103079bd047ffe2eb156a2356a85e94032bb847d89cd7244ef/adlmagics-0.0.1a2.tar.gz" } ] }