{ "info": { "author": "datamill", "author_email": "", "bugtrack_url": null, "classifiers": [ "Programming Language :: Python :: 3 :: Only" ], "description": "# Target Snowflake\n\n[![CircleCI](https://circleci.com/gh/datamill-co/target-snowflake.svg?style=svg)](https://circleci.com/gh/datamill-co/target-snowflake)\n\n[![PyPI version](https://badge.fury.io/py/target-snowflake.svg)](https://pypi.org/project/target-snowflake/)\n\n[![](https://img.shields.io/librariesio/github/datamill-co/target-snowflake.svg)](https://libraries.io/github/datamill-co/target-snowflake)\n\nA [Singer](https://singer.io/) Snowflake target, for use with Singer streams generated by Singer taps.\n\n## Snowflake Connector\n\n[Docs](https://docs.snowflake.net/manuals/user-guide/python-connector.html)\n\n## Install\n\n```sh\npip install target-snowflake\n```\n\n## Usage\n\n1. Follow the\n [Singer.io Best Practices](https://github.com/singer-io/getting-started/blob/master/docs/RUNNING_AND_DEVELOPING.md#running-a-singer-tap-with-a-singer-target)\n for setting up separate `tap` and `target` virtualenvs to avoid version\n conflicts.\n\n1. Create a [config file](#configjson) at\n `~/singer.io/target_snowflake_config.json` with Snowflake connection\n information and target Snowflake schema and warehouse.\n\n ```json\n {\n \"snowflake_account\": \"https://XXXXX.snowflakecomputing.com\",\n \"snowflake_username\": \"myuser\",\n \"snowflake_role\": \"myrole\",\n \"snowflake_password\": \"1234\",\n \"snowflake_database\": \"my_analytics\",\n \"snowflake_schema\": \"mytapname\",\n \"snowflake_warehouse\": \"dw\"\n }\n ```\n\n````\n\n1. Run `target-snowfkajke` against a [Singer](https://singer.io) tap.\n\n ```bash\n ~/.virtualenvs/tap-something/bin/tap-something \\\n | ~/.virtualenvs/target-snowflake/bin/target-snowflake \\\n --config ~/singer.io/target_snowflake_config.json >> state.json\n````\n\nIf you are running windows, the following is equivalent:\n\n```\nvenvs\\tap-exchangeratesapi\\Scripts\\tap-exchangeratesapi.exe | ^\nvenvs\\target-snowflake\\Scripts\\target-snowlfake.exe ^\n--config target_snowflake_config.json\n```\n\n### Config.json\n\nThe fields available to be specified in the config file are specified\nhere.\n\n| Field | Type | Default | Details |\n| --------------------------- | --------------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `snowflake_account` | `[\"string\"]` | `N/A` | `ACCOUNT` might require the `region` and `cloud` platform where your account is located, in the form of: `..` (e.g. `xy12345.east-us-2.azure`) [Refer to Snowflake's documentation about Account](https://docs.snowflake.net/manuals/user-guide/connecting.html#your-snowflake-account-name-and-url) |\n| `snowflake_username` | `[\"string\"]` | `N/A` | |\n| `snowflake_password` | `[\"string\", \"null\"]` | `null` | |\n| `snowflake_role` | `[\"string\"]` | `null` | If not specified, Snowflake will use the user's default role. |\n| `snowflake_database` | `[\"string\"]` | `N/A` | |\n| `snowflake_authenticator` | `[\"string\"]` | `\"snowflake\"` | Speifies the authentication provider for snowflake to use. Valud options are the internal one (\"snowflake\"), a browser session (\"externalbrowser\"), or Okta (\"https://.okta.com\"). See the snowflake docs for more details.\n| `snowflake_schema` | `[\"string\", \"null\"]` | `\"PUBLIC\"` | |\n| `snowflake_warehouse` | `[\"string\"]` | `N/A` | |\n| `invalid_records_detect` | `[\"boolean\", \"null\"]` | `true` | Include `false` in your config to disable crashing on invalid records |\n| `invalid_records_threshold` | `[\"integer\", \"null\"]` | `0` | Include a positive value `n` in your config to allow at most `n` invalid records per stream before giving up. |\n| `disable_collection` | `[\"string\", \"null\"]` | `false` | Include `true` in your config to disable [Singer Usage Logging](#usage-logging). |\n| `logging_level` | `[\"string\", \"null\"]` | `\"INFO\"` | The level for logging. Set to `DEBUG` to get things like queries executed, timing of those queries, etc. See [Python's Logger Levels](https://docs.python.org/3/library/logging.html#levels) for information about valid values. |\n| `persist_empty_tables` | `[\"boolean\", \"null\"]` | `False` | Whether the Target should create tables which have no records present in Remote. |\n| `state_support` | `[\"boolean\", \"null\"]` | `True` | Whether the Target should emit `STATE` messages to stdout for further consumption. In this mode, which is on by default, STATE messages are buffered in memory until all the records that occurred before them are flushed according to the batch flushing schedule the target is configured with. |\n| `target_s3` | `[\"object\", \"null\"]` | `N/A` | When included, use `S3` to stage files. See `S3` below |\n\n#### S3 Config.json\n\n| Field | Type | Default | Details |\n| ----------------------- | -------------------- | ------- | ---------------------------------------------------------------------------- |\n| `aws_access_key_id` | `[\"string\"]` | `N/A` | |\n| `aws_secret_access_key` | `[\"string\"]` | `N/A` | |\n| `bucket` | `[\"string\"]` | `N/A` | Bucket where staging files should be uploaded to. |\n| `key_prefix` | `[\"string\", \"null\"]` | `\"\"` | Prefix for staging file uploads to allow for better delineation of tmp files |\n\n## Limitations\n\n- [Snowflake SQL Identifiers](https://docs.snowflake.net/manuals/sql-reference/identifiers-syntax.html):\n - Although Snowflake supports quoted identifiers to have non-alphanumeric values, `target-snowflake` limits\n identifiers to uppercase alphanumerics, and underscores\n - This is done to make querability/useability in Snowflake simpler, so as to not require users to _have_ to use\n sometimes cumbersome quotes to query their data\n- Requires a [JSON Schema](https://json-schema.org/) for every stream.\n- Only string, string with date-time format, integer, number, boolean,\n object, and array types with or without null are supported. Arrays can\n have any of the other types listed, including objects as types within\n items.\n - Example of JSON Schema types that work\n - `['number']`\n - `['string']`\n - `['string', 'null']`\n - `['string', 'integer']`\n - `['integer', 'number']`\n - Exmaple of JSON Schema types that **DO NOT** work\n - `['any']`\n - `['null']`\n- JSON Schema combinations such as `anyOf` and `allOf` are not supported.\n- JSON Schema \\$ref is partially supported:\n - **_NOTE:_** The following limitations are known to **NOT** fail gracefully\n - Presently you cannot have any circular or recursive `$ref`s\n - `$ref`s must be present within the schema:\n - URI's do not work\n - if the `$ref` is broken, the behaviour is considered unexpected\n- Any values which are the `string` `\\\\N` will be streamed to Snowflake as the literal `null`\n\n## Sponsorship\n\nTarget Snowflake is sponsored by Data Mill (Data Mill Services, LLC) [datamill.co](https://datamill.co/).\n\nData Mill helps organizations utilize modern data infrastructure and data science to power analytics, products, and services.\n\n---\n\nCopyright Data Mill Services, LLC 2018\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/datamill-co/target-snowflake", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "target-snowflake", "package_url": "https://pypi.org/project/target-snowflake/", "platform": "", "project_url": "https://pypi.org/project/target-snowflake/", "project_urls": { "Homepage": "https://github.com/datamill-co/target-snowflake" }, "release_url": "https://pypi.org/project/target-snowflake/0.2.4/", "requires_dist": [ "singer-python (==5.9.0)", "singer-target-postgres (==0.2.4)", "target-redshift (==0.2.4)", "botocore (<1.13.0,>=1.12.253)", "snowflake-connector-python (==2.2.5)", "chance (==0.110) ; extra == 'tests'", "Faker (==4.0.3) ; extra == 'tests'", "pytest (==5.4.1) ; extra == 'tests'" ], "requires_python": "", "summary": "Singer.io target for loading data into Snowflake", "version": "0.2.4", "yanked": false, "yanked_reason": null }, "last_serial": 7197431, "releases": { "0.0.0": [ { "comment_text": "", "digests": { "md5": "0cb3e688cf13ff06efbb01b995693c25", "sha256": "4d20e6c8af6ed90772d34ba645827469f634b8a96cac482a9eaea0e67181c44b" }, "downloads": -1, "filename": "target_snowflake-0.0.0-py3-none-any.whl", "has_sig": false, "md5_digest": "0cb3e688cf13ff06efbb01b995693c25", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 16163, "upload_time": "2019-08-18T18:10:46", "upload_time_iso_8601": "2019-08-18T18:10:46.780698Z", "url": "https://files.pythonhosted.org/packages/4f/bf/8a19130c17fa3f65c44b0748d7f809f169063c9306b6b4588b630a5a99a4/target_snowflake-0.0.0-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "19c644b60b6055b30ed97e1eb3128f3f", "sha256": "7db25e67d9b93ad553457d185621b443835462a6ce93f61fabd90c3fc134e4b1" }, "downloads": -1, "filename": "target-snowflake-0.0.0.tar.gz", "has_sig": false, "md5_digest": "19c644b60b6055b30ed97e1eb3128f3f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3483, "upload_time": "2019-08-18T18:10:49", "upload_time_iso_8601": "2019-08-18T18:10:49.120180Z", "url": "https://files.pythonhosted.org/packages/94/ca/30f4f3c12a0b94fba8d6a5f2318d08c4e3b6866aaafbf797664ba23e1413/target-snowflake-0.0.0.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.1": [ { "comment_text": "", "digests": { "md5": "a6947c83fdf79d7c020ec5b9f8c6d3c2", "sha256": "54fee4e9b60cc1e622e3b249c09281c2c13efb264ea11c82d94146e972550469" }, "downloads": -1, "filename": "target_snowflake-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "a6947c83fdf79d7c020ec5b9f8c6d3c2", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 24570, "upload_time": "2019-08-27T20:23:27", "upload_time_iso_8601": "2019-08-27T20:23:27.234175Z", "url": "https://files.pythonhosted.org/packages/0c/ed/99a941a910599ab6dd6eccc4e655c3e09e8a35b76480a9f274bd0797d77a/target_snowflake-0.0.1-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "0a8056e0b53f9dd2f9a0d44a044193dc", "sha256": "fd08f404b7fac8b0bb0ce65e82e99c637b9f462e7a5ea59fba7f81e0e53f3b85" }, "downloads": -1, "filename": "target-snowflake-0.0.1.tar.gz", "has_sig": false, "md5_digest": "0a8056e0b53f9dd2f9a0d44a044193dc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14217, "upload_time": "2019-08-27T20:23:28", "upload_time_iso_8601": "2019-08-27T20:23:28.841921Z", "url": "https://files.pythonhosted.org/packages/cf/63/f6fac240a38d7e619ed5e84fbd370837fcde71be2451c843b88c1baa6912/target-snowflake-0.0.1.tar.gz", "yanked": false, "yanked_reason": null } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "76c0bff2957250b81eda139ce41e51c2", "sha256": "a81c72578f239294841b64d2912407c0c067477ac99e40b4eee82dc0899581f5" }, "downloads": -1, "filename": "target_snowflake-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "76c0bff2957250b81eda139ce41e51c2", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 25075, "upload_time": "2019-10-24T14:43:00", "upload_time_iso_8601": "2019-10-24T14:43:00.767640Z", "url": "https://files.pythonhosted.org/packages/2f/75/2f72f91ed30333a3dfb87232dfd2d0b6a54e86d5d3cb03ac7ab9e8e26cb2/target_snowflake-0.0.2-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "51ded5de9978eecb5b964604d2fcb070", "sha256": "2d1db544c6878c16cc22816598637314e35b733a2d37b4e3e6022790d181092b" }, "downloads": -1, "filename": "target-snowflake-0.0.2.tar.gz", "has_sig": false, "md5_digest": "51ded5de9978eecb5b964604d2fcb070", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14690, "upload_time": "2019-10-24T14:43:02", "upload_time_iso_8601": "2019-10-24T14:43:02.518484Z", "url": "https://files.pythonhosted.org/packages/91/6b/08d12d74be2647551bb279affe3315926544d7cbfa38d879d9dfb93236a9/target-snowflake-0.0.2.tar.gz", "yanked": false, "yanked_reason": null } ], "0.2.4": [ { "comment_text": "", "digests": { "md5": "670644cecbf0469d503af00757dfd7cc", "sha256": "a4872044616c74da2279ee94728cdc7159fac7a5c80479fec161b7aaf0861b76" }, "downloads": -1, "filename": "target_snowflake-0.2.4-py3-none-any.whl", "has_sig": false, "md5_digest": "670644cecbf0469d503af00757dfd7cc", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 25324, "upload_time": "2020-05-08T14:14:23", "upload_time_iso_8601": "2020-05-08T14:14:23.418136Z", "url": "https://files.pythonhosted.org/packages/6c/92/52b365921de6b242f00ef8a3b19d7123d3a42598a44cbd39182e41036f36/target_snowflake-0.2.4-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "f61025ac501328097016bf248c2d93cc", "sha256": "09670035c023606646ceeba498fb978549150de272d0783ce763231214cda6ec" }, "downloads": -1, "filename": "target-snowflake-0.2.4.tar.gz", "has_sig": false, "md5_digest": "f61025ac501328097016bf248c2d93cc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15083, "upload_time": "2020-05-08T14:14:24", "upload_time_iso_8601": "2020-05-08T14:14:24.574887Z", "url": "https://files.pythonhosted.org/packages/23/a2/5b538a1bf49a06d32d33e70022a12e7623dc3375452fe5b9e4396f2c3a7e/target-snowflake-0.2.4.tar.gz", "yanked": false, "yanked_reason": null } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "670644cecbf0469d503af00757dfd7cc", "sha256": "a4872044616c74da2279ee94728cdc7159fac7a5c80479fec161b7aaf0861b76" }, "downloads": -1, "filename": "target_snowflake-0.2.4-py3-none-any.whl", "has_sig": false, "md5_digest": "670644cecbf0469d503af00757dfd7cc", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 25324, "upload_time": "2020-05-08T14:14:23", "upload_time_iso_8601": "2020-05-08T14:14:23.418136Z", "url": "https://files.pythonhosted.org/packages/6c/92/52b365921de6b242f00ef8a3b19d7123d3a42598a44cbd39182e41036f36/target_snowflake-0.2.4-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "f61025ac501328097016bf248c2d93cc", "sha256": "09670035c023606646ceeba498fb978549150de272d0783ce763231214cda6ec" }, "downloads": -1, "filename": "target-snowflake-0.2.4.tar.gz", "has_sig": false, "md5_digest": "f61025ac501328097016bf248c2d93cc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15083, "upload_time": "2020-05-08T14:14:24", "upload_time_iso_8601": "2020-05-08T14:14:24.574887Z", "url": "https://files.pythonhosted.org/packages/23/a2/5b538a1bf49a06d32d33e70022a12e7623dc3375452fe5b9e4396f2c3a7e/target-snowflake-0.2.4.tar.gz", "yanked": false, "yanked_reason": null } ], "vulnerabilities": [] }