{ "info": { "author": "Antsa Randriamihaja", "author_email": "antsa.randriamihaja@moov.ai", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3.6" ], "description": "## MOOVAI TOOLBOX\n\nThis repository contains reusable code to expedite development. \nTo use repository, ensure you are using Python 3.6. \n\n### Installation:\nTo use this package, install using pip:\n\n```python\npip install moovai\n```\n\n### Google Cloud:\nThis folder contains code to handle GCP resources. \nPrior to using the methods here, you need to be authenticated to GCP.\nIf you are using a service account, and have the key JSON file:\n- On Linux or MacOS\n```python\nexport GOOGLE_APPLICATION_CREDENTIALS=\"[PATH]\"\n```\n- On Windows:\n\n```python\nset GOOGLE_APPLICATION_CREDENTIALS=[PATH]\n```\n\n#### cloud-storage\nBefore using the code here, first make sure that you have a bucket on GCS. If you don't, create one.\n\n##### upload_to_gcs: \nUploads a local file to a folder on google cloud storage. The file is then DELETED locally.\n###### Parameters:\n- file: REQUIRED: [STRING] Local file path to upload to google cloud storage.\n- bucket_name: REQUIRED: [STRING] Name of your bucket.\n- folder: REQUIRED: [STRING] Folder path to where you want your file to be uploaded to on GCS.\n\nreturns: GCS URI where the file was uploaded\n###### Sample usage:\n\n```python\nfrom moovai.google_cloud import cloud_storage\n\nfile = \"/path/to/my/file.txt\"\nbucket_name = \"my_bucket\"\nfolder = \"Folder/Subfolder/\"\n\ncloud_storage.upload_to_gcs(file, bucket_name, folder)\n# returns \"gs://my_bucket/Folder/Subfolder/file.txt\"\n\n```\n\n##### download_file_gcs:\nDownloads file from google cloud storage to local disk. \n###### Parameters:\n- gcs_uri: REQUIRED. [STRING] URI of file located on google cloud storage to be downloaded\n\nreturns: None. GCS file is downloaded locally.\n###### Sample usage:\n\n```python\nfrom moovai.google_cloud import cloud_storage\n\ngcs_uri = \"gs://my_bucket/Folder/Subfolder/my_file.txt\"\n\ncloud_storage.download_file_gcs(gcs_uri)\n# file \"my_file.txt\" downloaded locally\n\n```\n\n#### bigquery\n##### get_schema_from_json:\nTakes a schema.json file and converts it into a schema file compatible with BigQuery.\n###### Parameters:\n- schema_path: REQUIRED. [STRING] Path to your_schema.json file.\n\nreturns: schema to plug into BigQuery upload job.\n###### Sample usage:\n\n```python\nfrom moovai.google_cloud import bigquery\n\nschema_file = \"/path/to/my/schema.json\"\nbigquery.get_schema_from_json(schema_file)\n\n```\n\n##### get_schema_from_csv:\nTakes a csv file containing your data and extracts the schema from your file.\nIt is recommended to simply use BigQuery's auto-detect schema feature. Use this as a last resort.\n###### Parameters:\n- csv_file_path: REQUIRED. [STRING] Path to csv file.\n\nreturns: schema to plug into BigQuery upload job.\n###### Sample usage:\n\n```python\nfrom moovai.google_cloud import bigquery\n\ncsv_file = \"/path/to/my/file.csv\"\nbigquery.get_schema_from_csv(csv_file)\n\n```\n\n##### upload_local_file_to_bq:\nUploads local file to BigQuery. schema_path and schema are optional arguments. They are exclusive of one another, provide only one if you want to.\n###### Parameters:\n- file: REQUIRED. [STRING] Path to local CSV file to upload\n- dataset_id: REQUIRED. [STRING] Name of your BigQuery dataset.\n- table_id: REQUIRED. [STRING] Name of your BigQuery table to query.\n- schema_path: OPTIONAL. [STRING] Path to schema.json file. \n- schema: OPTIONAL. [STRING] schema compatible with BigQuery. \n- overwrite: OPTIONAL. [BOOL] Defaults to False. if set to True, BigQuery will overwrite table, else, it will append new data to table.\n\nreturns: None.\n###### Sample usage:\n```python\nfrom moovai.google_cloud import bigquery\n\nfile = \"/path/to/my_file.csv\"\ndataset_id = \"my_dataset_id\"\ntable_id = \"my_table_id\"\nbigquery.upload_local_file_to_bq(file, dataset_id, table_id)\n\n```\n\n##### upload_gcs_file_to_bq:\nUploads file from cloud storage to BigQuery. schema_path and schema are optional arguments. They are exclusive of one another, provide only one if you want to.\n###### Parameters:\n- gcs_uri: REQUIRED. [STRING] Path to CSV file located on cloud storage to upload to BigQuery.\n- dataset_id: REQUIRED. [STRING] Name of your BigQuery dataset.\n- table_id: REQUIRED. [STRING] Name of your BigQuery table to query.\n- schema_path: OPTIONAL. [STRING] Path to schema.json file. \n- schema: OPTIONAL. [STRING] schema compatible with BigQuery. \n- overwrite: OPTIONAL. [BOOL] Defaults to False. if set to True, BigQuery will overwrite table, else, it will append new data to table.\n\nreturns: None.\n###### Sample usage:\n```python\nfrom moovai.google_cloud import bigquery\n\ngcs_uri = \"gs://my_bucket/Path/To/my_file.csv\"\ndataset_id = \"my_dataset_id\"\ntable_id = \"my_table_id\"\nbigquery.upload_local_file_to_bq(gcs_uri, dataset_id, table_id)\n\n```\n\n##### generate_sql_query:\nGenerates a SQL query string to use to query a BigQuery table.\n###### Parameters:\n- project: REQUIRED. [STRING] Project ID\n- dataset_id: REQUIRED. [STRING] Name of your BigQuery dataset.\n- table_id: REQUIRED. [STRING] Name of your BigQuery table to query.\n- columns: OPTIONAL. [ARRAY] List of column names you want to select.\n- conditions: OPTIONAL. [ARRAY] List of conditions to satisfy.\n\nreturns: STRING. Standard SQL query string.\n###### Sample usage:\n\n```python\nfrom moovai.google_cloud import bigquery\n\nproject = \"my_project_id\"\ndataset_id = \"my_dataset_id\"\ntable_id = \"my_table_id\"\nbigquery.generate_sql_query(project, dataset_id, table_id)\n#returns \"SELECT * FROM `project.dataset_id.table_id`\" (return everything from table)\n\ncolumns = [\"col1\", \"col2\"]\nconditions = [\"Date >= TIMESTAMP('2019-05-01')\", \"col3 < 3\"]\n\nbigquery.generate_sql_query(project, dataset_id, table_id, columns=columns, conditions=conditions)\n#returns \"SELECT col1, col2 FROM `project.dataset_id.table_id` WHERE Date >= TIMESTAMP('2019-05-01') AND col3 < 3\" (returns col1 and col2 that meet the specified conditions)\n\n```\n\n##### get_data_from_bq:\nTakes a SQL query string (Standard SQL) and returns pandas dataframe\n###### Parameters:\n- sql_query: REQUIRED. [STRING] string representing the query you want to make.\n\nreturns: pandas Dataframe containg the result of your query.\n###### Sample usage:\n\n```python\nfrom moovai.google_cloud import bigquery\n\nsql_query = \"SELECT * FROM `my_project.my_dataset.my_table`\"\nbigquery.get_data_from_bq(sql_query)\n\n```\n\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/moovai/Toolbox/", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "moovai", "package_url": "https://pypi.org/project/moovai/", "platform": "", "project_url": "https://pypi.org/project/moovai/", "project_urls": { "Documentation": "https://github.com/moovai/Toolbox/blob/master/README.md", "Homepage": "https://github.com/moovai/Toolbox/", "Source Code": "https://github.com/moovai/Toolbox/tree/master/MoovAI" }, "release_url": "https://pypi.org/project/moovai/0.0.9/", "requires_dist": [ "google-cloud-error-reporting (==0.31.0)", "google-cloud-bigquery (==1.15.0)", "google-cloud-storage (==1.16.1)", "joblib", "pandas", "scikit-learn", "pysnooper", "tensorflow", "keras", "matplotlib" ], "requires_python": "", "summary": "A package for working with GCS and BigQuery", "version": "0.0.9" }, "last_serial": 5775881, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "c916723a4c06876b3aed2f214976e907", "sha256": "19d44f87844dfc06bb69e55ffba4864d496d586fd3637d4a7cc075e166ccf2bd" }, "downloads": -1, "filename": "moovai-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "c916723a4c06876b3aed2f214976e907", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 4060, "upload_time": "2019-07-23T17:37:20", "url": "https://files.pythonhosted.org/packages/b7/37/03257be98356afad27821dd91489526b41fcff83274b8cc804c371d618bb/moovai-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1bff95b970d824a380f9ff7a4ef25570", "sha256": "a72d041b83f5b02317ccd055d41fa1ce2e890340f37d2ce97ebf81f6d103c9ae" }, "downloads": -1, "filename": "moovai-0.0.1.tar.gz", "has_sig": false, "md5_digest": "1bff95b970d824a380f9ff7a4ef25570", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1844, "upload_time": "2019-07-23T17:37:22", "url": "https://files.pythonhosted.org/packages/5e/a5/0b049f280e29ef9b0e4cf35d7f9a642ef04c6bb44d3f78de0dc5cf5989da/moovai-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "b29d3152c85e7d7fb50449263f59e0b6", "sha256": "2f5dac855889c1a78514639addcc2418c400c311c4551c879b82335224e0dcc7" }, "downloads": -1, "filename": "moovai-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "b29d3152c85e7d7fb50449263f59e0b6", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 10376, "upload_time": "2019-07-23T21:07:04", "url": "https://files.pythonhosted.org/packages/00/46/df8950479b1d2883bd55c4cea352592cce05d89efa684292b29a5a735637/moovai-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3c01624f092a77167edf6db3b31a5402", "sha256": "2763cfa87af5d835490fd167534631ecbec62de7217732e50724c858572630a2" }, "downloads": -1, "filename": "moovai-0.0.2.tar.gz", "has_sig": false, "md5_digest": "3c01624f092a77167edf6db3b31a5402", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5833, "upload_time": "2019-07-23T21:07:06", "url": "https://files.pythonhosted.org/packages/92/c0/dff2592aec413e849d8f576ff7f6244e46f9ff47427fb1c52e941fe12f73/moovai-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "74d726b39219acd45a872b5b79014161", "sha256": "8c390dda2a3df7b9e8e78e44fbb7a6a467d496f040ff31f4ea37ce75bd1dabbe" }, "downloads": -1, "filename": "moovai-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "74d726b39219acd45a872b5b79014161", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 17993, "upload_time": "2019-07-31T18:57:49", "url": "https://files.pythonhosted.org/packages/61/8b/861ce14004ed1c69428d3e108f0e99a68c035f8e67edefd4e3e59b2ccf8a/moovai-0.0.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "44b42b2e3606181f738a122743dc8997", "sha256": "d897c6d4905bfe3bcfdbc0273b637a39853d5917310a2e1b989833093d0e9cc4" }, "downloads": -1, "filename": "moovai-0.0.3.tar.gz", "has_sig": false, "md5_digest": "44b42b2e3606181f738a122743dc8997", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10505, "upload_time": "2019-07-31T18:57:51", "url": "https://files.pythonhosted.org/packages/bf/fe/d00a51a9b6e8db325cd8520b7f38791dc7a7875ba52003693225170d2cfa/moovai-0.0.3.tar.gz" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "547f133232a096513c0ba74922026d50", "sha256": "183409deda6288fb6c63f20ed92593fa6087d63ee60c6de609d44c3241c09681" }, "downloads": -1, "filename": "moovai-0.0.4-py3-none-any.whl", "has_sig": false, "md5_digest": "547f133232a096513c0ba74922026d50", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 17994, "upload_time": "2019-07-31T19:27:30", "url": "https://files.pythonhosted.org/packages/32/2d/b8e6fae2a63f47439ce4d97d728dc289fd3f27754b8ed287afb7a53f9920/moovai-0.0.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f18a533cee8ec553ec81804a402ce8da", "sha256": "3bd43745a7e0fdd6b1bccf22d888849d93b8429d7995f544fa4665743a361d94" }, "downloads": -1, "filename": "moovai-0.0.4.tar.gz", "has_sig": false, "md5_digest": "f18a533cee8ec553ec81804a402ce8da", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10506, "upload_time": "2019-07-31T19:27:33", "url": "https://files.pythonhosted.org/packages/1c/5a/070d1f3c581ba576a1989e6b4243bf307fa317bbc2f1373a21136fb7b6c2/moovai-0.0.4.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "7f8422ee0287e393edd286c1f74d4b03", "sha256": "0480d4b08c8f53e07fee52039f631fdc30ff7237a35e7d0eee5ebdae6b078aa6" }, "downloads": -1, "filename": "moovai-0.0.5-py3-none-any.whl", "has_sig": false, "md5_digest": "7f8422ee0287e393edd286c1f74d4b03", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 23428, "upload_time": "2019-07-31T19:32:19", "url": "https://files.pythonhosted.org/packages/70/08/88037c77dddc1a242a44a423e3267be0d94a30bdb039c35155df56bfcbb8/moovai-0.0.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "eb44f6301c1ac7ede72e1c2cdfac571e", "sha256": "d5ebbb6dbdf51e0491ef45d61cf71eac17963e08d40c6f7ca230efb1e83882be" }, "downloads": -1, "filename": "moovai-0.0.5.tar.gz", "has_sig": false, "md5_digest": "eb44f6301c1ac7ede72e1c2cdfac571e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10474, "upload_time": "2019-07-31T19:32:23", "url": "https://files.pythonhosted.org/packages/05/a9/52d2617fe3db5e000b2f93484864159d1ce40f17d584df4a3ef8245a5c74/moovai-0.0.5.tar.gz" } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "696feca9cd5bdfc9962dedafe3a8fbcb", "sha256": "2623d967a5fb3c7f6b25f98bd4b4c6a30c085c04a9ced8117bfd32f3bc51d590" }, "downloads": -1, "filename": "moovai-0.0.6-py3-none-any.whl", "has_sig": false, "md5_digest": "696feca9cd5bdfc9962dedafe3a8fbcb", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 23429, "upload_time": "2019-07-31T19:33:49", "url": "https://files.pythonhosted.org/packages/53/29/eb94a75e32d03ba864aec34481cb057627e4b1aede262788c937c1ee6a23/moovai-0.0.6-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1162e1be81336d926734d62843029c8f", "sha256": "bc8f4624ea32aa5ec7a5b500d5de8e14467693a9623e54a6f0f53d46df6649b6" }, "downloads": -1, "filename": "moovai-0.0.6.tar.gz", "has_sig": false, "md5_digest": "1162e1be81336d926734d62843029c8f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10471, "upload_time": "2019-07-31T19:33:51", "url": "https://files.pythonhosted.org/packages/bd/74/cad9e9f5fdab565d74fc6ccda421b2137d534e5f630d9794261098e988e5/moovai-0.0.6.tar.gz" } ], "0.0.7": [ { "comment_text": "", "digests": { "md5": "e36dd80bb5c7ddc78302281751bfa86c", "sha256": "36ab8d9d06b4222a4463d61728e9b90357642c11eccdeebb25d2677948650b40" }, "downloads": -1, "filename": "moovai-0.0.7-py3-none-any.whl", "has_sig": false, "md5_digest": "e36dd80bb5c7ddc78302281751bfa86c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 23494, "upload_time": "2019-08-20T17:21:45", "url": "https://files.pythonhosted.org/packages/d9/38/b06bdfbebea7842b72a5f24cb592fe0bc1f3cfc227223375d690bf291ae2/moovai-0.0.7-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e0f3ecaa28d8a09905e8dbb37755dd05", "sha256": "6d49775d489b6ae40abd17f1b789b0d71ecb0ba716701451efd469a202d1de0c" }, "downloads": -1, "filename": "moovai-0.0.7.tar.gz", "has_sig": false, "md5_digest": "e0f3ecaa28d8a09905e8dbb37755dd05", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10544, "upload_time": "2019-08-20T17:21:47", "url": "https://files.pythonhosted.org/packages/c8/e5/2bdf93f909a2c8180e11c16f4c5cc07d7a8fcbc74bb9ae8c41a4ae9afa77/moovai-0.0.7.tar.gz" } ], "0.0.8": [ { "comment_text": "", "digests": { "md5": "73139f63165cd606d4a189aa639297e6", "sha256": "48066c8e7d5399cd41c9c1512b6e80a1e64c15b8cb193bd4daa4d324de7c0497" }, "downloads": -1, "filename": "moovai-0.0.8-py3-none-any.whl", "has_sig": false, "md5_digest": "73139f63165cd606d4a189aa639297e6", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 23558, "upload_time": "2019-09-03T12:49:53", "url": "https://files.pythonhosted.org/packages/61/14/e80d816e02f46bd43caa980ba77578908d007e6d6789b83f745db30c8574/moovai-0.0.8-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "66fea7cffa6c9fcf0bcc394f0cb4780b", "sha256": "e82ff112fb6f6505ba1623cbe058ab7029daab8415b5fb9a3d9db174fb8043dd" }, "downloads": -1, "filename": "moovai-0.0.8.tar.gz", "has_sig": false, "md5_digest": "66fea7cffa6c9fcf0bcc394f0cb4780b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10584, "upload_time": "2019-09-03T12:49:54", "url": "https://files.pythonhosted.org/packages/cb/72/b5a8349767c19362814e37edc39c72602d90db9a45349ab47e475daee53a/moovai-0.0.8.tar.gz" } ], "0.0.9": [ { "comment_text": "", "digests": { "md5": "71c0164b837e92994136321d66bd7f15", "sha256": "17bcd52431597cd468cb85fadc90b4144f5e418372455f4264322f35d6e28f89" }, "downloads": -1, "filename": "moovai-0.0.9-py3-none-any.whl", "has_sig": false, "md5_digest": "71c0164b837e92994136321d66bd7f15", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 23578, "upload_time": "2019-09-03T13:12:25", "url": "https://files.pythonhosted.org/packages/28/18/70787fcc932f1da45183693470e56b07bdc0bc72396f22528ba085d3ba92/moovai-0.0.9-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "590a6433cbd0b466bdc94d363f0f2c5a", "sha256": "efb75910d3f30c46817bfd5dad48dd092354c757207a79d17ed444df5fc1c43b" }, "downloads": -1, "filename": "moovai-0.0.9.tar.gz", "has_sig": false, "md5_digest": "590a6433cbd0b466bdc94d363f0f2c5a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10594, "upload_time": "2019-09-03T13:12:27", "url": "https://files.pythonhosted.org/packages/3d/60/597c66c91f9cbcee74927b2964656c458bbc3da176846da6e56e77cce768/moovai-0.0.9.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "71c0164b837e92994136321d66bd7f15", "sha256": "17bcd52431597cd468cb85fadc90b4144f5e418372455f4264322f35d6e28f89" }, "downloads": -1, "filename": "moovai-0.0.9-py3-none-any.whl", "has_sig": false, "md5_digest": "71c0164b837e92994136321d66bd7f15", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 23578, "upload_time": "2019-09-03T13:12:25", "url": "https://files.pythonhosted.org/packages/28/18/70787fcc932f1da45183693470e56b07bdc0bc72396f22528ba085d3ba92/moovai-0.0.9-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "590a6433cbd0b466bdc94d363f0f2c5a", "sha256": "efb75910d3f30c46817bfd5dad48dd092354c757207a79d17ed444df5fc1c43b" }, "downloads": -1, "filename": "moovai-0.0.9.tar.gz", "has_sig": false, "md5_digest": "590a6433cbd0b466bdc94d363f0f2c5a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10594, "upload_time": "2019-09-03T13:12:27", "url": "https://files.pythonhosted.org/packages/3d/60/597c66c91f9cbcee74927b2964656c458bbc3da176846da6e56e77cce768/moovai-0.0.9.tar.gz" } ] }