{ "info": { "author": "Matthew Turland", "author_email": "me@matthewturland.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Environment :: Console", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3 :: Only", "Topic :: Database", "Topic :: Utilities" ], "description": "# pycopyql\n\nExports a subset of data from a relational database.\n\nInspired by [copyql](https://github.com/dolfelt/copyql). \u2764\ufe0f\n\n## Installation\n\nUse [pip](https://pip.pypa.io/en/stable/installing/).\n\n```sh\npip install pycopyql\n```\n\n## Usage\n\n```sh\npycopyql [-h] [-c CONFIG] [-f FORMAT] connection output query [query ...]\n```\n\n* `-h`: Shows usage information\n* `-c`: Path to a configuration file, defaults to `./pycopyql.py`\n* `-f`: Output format, `sql` and `json` are supported, defaults to `sql`\n* `connection`: Name of the connection to use, defined in the configuration file\n* `output`: Path to output file\n* `query`: One or more query strings (delimited by spaces) of the format `TABLE.COLUMN:VALUE` (e.g. `accounts.id:1`)\n\nExample:\n\n```sh\npycopyql -c ./path/to/pycopyql.py -f json my_connection path/to/output.sql accounts.id:1 users.id:2\n```\n\n## Configuration\n\n`pycopyql` uses a Python file for configuration. It looks like this.\n\n```python\n# pycopyql.py\n\nconfig = {\n 'connections': {\n 'my_connection': {\n 'drivername': 'mysql+mysqlconnector',\n 'username': 'myuser',\n 'password': 'mypassword',\n 'host': 'localhost',\n 'port': 3306,\n 'database': 'mydatabase',\n }\n }\n}\n```\n\nThe file must define a `config` variable that is assigned a [dictionary](https://docs.python.org/3/tutorial/datastructures.html#dictionaries). That dictionary must have a `'connections'` key that is likewise assigned a dictionary.\n\nWithin `'connections'`, each key-value pair represents the configuration for a single database connection. The key is a name used to reference the connection when invoking `pycopyql`.\n\nUnder the hood, `pycopyql` uses the popular [SQLAlchemy library](http://www.sqlalchemy.org). Most settings for each connection correspond to the settings used for [SQLAlchemy URLs](http://docs.sqlalchemy.org/en/latest/core/engines.html#sqlalchemy.engine.url.URL).\n\n### Resolvers\n\nA **resolver** in `pycopyql` is a function that receives the names of a table and a column contained in that table and returns a list of table-column pairs related to the specified table-column pair that `pycopyql` will then search for data to include in its output.\n\nLet's take a look at an example of a resolver that's bundled with `pycopyql`.\n\n```\n\u276f python3\nPython 3.7.0 (default, Jun 29 2018, 20:14:27)\n[Clang 9.0.0 (clang-900.0.39.2)] on darwin\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n>>> from pycopyql.resolver.inflector import dependents_inflector\n>>> help(dependents_inflector)\n\nHelp on function dependents_inflector in module pycopyql.resolver.inflector:\n\ndependents_inflector(meta, table, column)\n Uses inflection of column names to return foreign keys from tables that reference the specified column if it is a primary key.\n\n Parameters:\n meta (sqlalchemy.schema.MetaData): Metadata for the database structure\n table (string): Name of the table containing the column being examined\n column (string): Name of the column being examined\n\n Returns:\n list: List of dictionaries each containing the table and column of foreign key columns that reference the specified column\n```\n\nLet's assume that we're using this resolver with a database that looks like this.\n\n```\n+-----------+ +------------+ +---------+\n| libraries <--+ | books | +--> authors |\n+-----------+ | +------------+ | +---------+\n| id | | | id | | | id |\n| name | +--+ library_id | | | name |\n| owner | | title | | +---------+\n+-----------+ | author_id +--+\n +------------+\n```\n\nGiven a table name of `'libraries'` and a column name of `'id'`, `dependents_inflector` would return this.\n\n```python\n[\n { 'table': 'books', 'column': 'library_id' },\n]\n```\n\n`pycopyql.resolver.inflector` also includes another resolver called `dependencies_inflector`, which works in the opposite direction of `dependents_inflector`. That is, where `dependents_inflector` returns foreign keys that reference a primary key, `dependencies_inflector` returns the primary key corresponding to a foreign key.\n\nUsing the above database as an example again, given a table name of `'books'` and a column name of `'library_id'`, `dependencies_inflector` would return this.\n\n```python\n[\n { 'table': 'libraries', 'column': 'id' },\n]\n```\n\nA custom resolver that handles both of these cases using the bundled resolvers shown above might look like this.\n\n```python\n# pycopyql.py\n\nfrom pycopyql.resolver.inflector import dependents_inflector, dependencies_inflector\n\ndef my_resolver(meta, table, column):\n dependents = dependents_inflector(meta, table, column)\n dependencies = dependencies_inflector(meta, table, column)\n return dependents + dependencies\n\nconfig = {\n 'connections': {\n 'my_connection': {\n # SQLAlchemy connection parameters go here\n\n 'resolver': my_resolver,\n },\n },\n}\n```\n\nIn fact, `pycopyql.resolver.inflector` contains yet another resolver called `inflector` that functions exactly this way.\n\nCustom resolvers can operate in many ways. For example, one might use foreign keys to determine relationships. The `dependents_foreign_key` and `dependencies_foreign_key` resolvers contained in `pycopyql.resolver.foreign_key` are equivalent to the `dependents_inflector` and `dependencies_inflector` resolvers described earlier. `pycopyql.resolver.foreign_key` also includes the `foreign_key` resolver, which is likewise equivalent to the `inflector` resolver described earlier.\n\nFor some cases where database column naming conventions are inconsistent, or where one table requires multiple references to another table (e.g. identifiers of the last users who created and updated a record), a resolver can use a static list of keys specific to that database. Here's an example using the database schema shown earlier.\n\n```python\ndef my_resolver(meta, table, column):\n keys = {\n 'libraries': {\n 'id': [ { 'table': 'books', 'column': 'library_id' } ],\n },\n 'books': {\n 'library_id': [ { 'table': 'libraries', 'column': 'id' } ],\n 'author_id': [ { 'table': 'authors', 'column': 'id' } ]\n },\n 'authors': {\n 'id': [ { 'table': 'books', 'column': 'author_id' } ],\n },\n }\n if table in keys and column in keys[table]:\n return keys[table][column]\n return []\n```\n\n`pycopyql.resolver.static` contains a convenience function, `static_map`, that takes in a dictionary like the one assigned to `keys` in the example above and returns a resolver that will use that static map in the same way.\n\n```python\nfrom pycopyql.resolver.static import static_map\n\nkeys = {\n 'table_1': {\n 'column_1': [\n { 'table': 'other_table_1', 'column': 'other_column_1' },\n { 'table': 'other_table_2', 'column': 'other_column_2' },\n # ...\n ]\n },\n 'table_2': {\n # ...\n },\n # ...\n}\n\nconfig = {\n 'connections': {\n 'my_connection': {\n # ...\n\n 'resolver': static_map(keys),\n },\n },\n}\n```\n\nCustom resolvers can use several of the resolvers described here together simultaneously. For example, if you wish to use inflection, but have some cases for which you must provide a static mapping, you can do both using a custom resolver.\n\n```python\nfrom pycopyql.resolver.inflector import inflector\nfrom pycopyql.resolver.static import static_map\n\nstatic_keys = {\n # ...\n}\n\nstatic_resolver = static_map(static_keys)\n\ndef my_resolver(meta, table, column):\n static = static_resolver(meta, table, column)\n inflected = inflector(meta, table, column)\n return static + inflected\n```\n\n## Output Formats\n\nBy default, `pycopyql` supports `sql` and `json` output formats. You can add additional formats or replace the bundled export functions for supported formats with your own.\n\nIn your configuration file, on the same level as the `'connections'` key, add an `'exporters'` key. The corresponding value should be a dictionary keyed by a format name that will be passed to `pycopyql` using its `-f` flag. The value should be a function that outputs given data in that format.\n\nLet's take a look at one of the bundled export functions.\n\n```\n\u276f python3\nPython 3.7.0 (default, Jun 29 2018, 20:14:27)\n[Clang 9.0.0 (clang-900.0.39.2)] on darwin\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n>>> from pycopyql.export import export_sql\n>>> help(export_sql)\n\nHelp on function export_sql in module pycopyql.export:\n\nexport_sql(meta, data, output)\n Outputs data as SQL INSERT statements.\n\n Parameters:\n meta (sqlalchemy.schema.MetaData): Metadata for the database structure\n data (dict): Dictionary keyed by table name of dictionaries corresponding to table rows\n output (string): Path for output file\n\n Returns:\n None\n```\n\nIf you wanted to override the bundled SQL exporter, here's what it might look like.\n\n```python\ndef my_sql_exporter(meta, data, output):\n # ...\n\nconfig = {\n 'connections': {\n # ...\n },\n 'exporters': {\n 'sql': my_sql_exporter, \n },\n}\n```\n\n## License\n\n[MIT License](https://en.wikipedia.org/wiki/MIT_License)\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/elazar/pycopyql", "keywords": "database relational data export tool utility", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "pycopyql", "package_url": "https://pypi.org/project/pycopyql/", "platform": "", "project_url": "https://pypi.org/project/pycopyql/", "project_urls": { "Homepage": "https://github.com/elazar/pycopyql" }, "release_url": "https://pypi.org/project/pycopyql/0.4.0/", "requires_dist": [ "inflect (~=1.0.0)", "SQLAlchemy (~=1.2.10)" ], "requires_python": ">=3.5", "summary": "Exports a subset of data from a relational database", "version": "0.4.0" }, "last_serial": 4250865, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "693de5efeb1fe68d498c286caa97243a", "sha256": "a7a26583ff46f6c9e5ae3512d22b7ed65917243ef495e51c0a95009282bd7951" }, "downloads": -1, "filename": "pycopyql-0.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "693de5efeb1fe68d498c286caa97243a", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.5", "size": 9391, "upload_time": "2018-08-02T13:03:27", "url": "https://files.pythonhosted.org/packages/12/67/724d93ab6a94fad62fac094572396099f1f58da0402211e03a402c95369e/pycopyql-0.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "d2b2c02e195e906555ad9b664258324b", "sha256": "1e6c806fa5d2234a65a2cd70db3ab8c30ec5a723ea8b120dd5fa17e05acdb1d3" }, "downloads": -1, "filename": "pycopyql-0.1.0.tar.gz", "has_sig": false, "md5_digest": "d2b2c02e195e906555ad9b664258324b", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 8892, "upload_time": "2018-08-02T13:03:28", "url": "https://files.pythonhosted.org/packages/0c/04/0c3f3743bdb3b6dd8f491c8c9f5acec589667f419e2d50a306f5295bec68/pycopyql-0.1.0.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "f78e4034500cd02ed8720cf799068883", "sha256": "b615d4a28b85bc771f00ba68d10fc2a7fd6016d1727284ae2fc275245b632f83" }, "downloads": -1, "filename": "pycopyql-0.2.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "f78e4034500cd02ed8720cf799068883", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.5", "size": 9062, "upload_time": "2018-08-11T14:51:13", "url": "https://files.pythonhosted.org/packages/df/7b/e7f3ddc6391e6172681bd331b7970103fd423ce48856810b4a49eb71aada/pycopyql-0.2.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ca8ba2a8d58011332b44ff897b499474", "sha256": "f6a7e003e0769f2cfa284bc58ce0a136ddca1745a20642338bf922d91e1b7fdd" }, "downloads": -1, "filename": "pycopyql-0.2.0.tar.gz", "has_sig": false, "md5_digest": "ca8ba2a8d58011332b44ff897b499474", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 8953, "upload_time": "2018-08-11T14:51:15", "url": "https://files.pythonhosted.org/packages/b4/b9/f2a1bff1b130c30ceb1d2c520ea42add0181050a5332d8e0a177eb37a566/pycopyql-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "c67b58971eaa48d2cc6503b74952e528", "sha256": "cd116b0ae243407185423eb2c8664c5f618b7ea0799b206a03413923977541dd" }, "downloads": -1, "filename": "pycopyql-0.2.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "c67b58971eaa48d2cc6503b74952e528", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.5", "size": 11368, "upload_time": "2018-08-13T16:22:46", "url": "https://files.pythonhosted.org/packages/89/fe/e4613e4b7bb544b29016a79a9fd4558f61535db1937584571dccf10ab1ff/pycopyql-0.2.1-py2.py3-none-any.whl" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "ee9e5d708392e3a73e9fb999649e34f8", "sha256": "51c47feee1fd411698865277315e500031adcb04bfc4baf62db97afcd2e8a4cd" }, "downloads": -1, "filename": "pycopyql-0.3.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "ee9e5d708392e3a73e9fb999649e34f8", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.5", "size": 11505, "upload_time": "2018-09-01T01:36:41", "url": "https://files.pythonhosted.org/packages/41/b8/63da3066922618f805adf2fd66ee17ef3dc6cebef9009bb5b012ac50b057/pycopyql-0.3.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "7f5c149376d869b1859ac5b6603158be", "sha256": "b4a5b17c6db297abae4f2db3f95f5335f1823b1b05bbfac834d7abb2f949dbe0" }, "downloads": -1, "filename": "pycopyql-0.3.0.tar.gz", "has_sig": false, "md5_digest": "7f5c149376d869b1859ac5b6603158be", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 11547, "upload_time": "2018-09-01T01:36:42", "url": "https://files.pythonhosted.org/packages/61/6b/62230e31a1cf6c0efc027f4d85b51a21e741bd45d3d5dc1cfddb2736847d/pycopyql-0.3.0.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "7aeed300c56097062bc57be5b8aa3f6a", "sha256": "0eca8b4a5c2747d39859e59429d6c770fda5ee813be3a54f811cc2c43c6d3c7c" }, "downloads": -1, "filename": "pycopyql-0.4.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "7aeed300c56097062bc57be5b8aa3f6a", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.5", "size": 11623, "upload_time": "2018-09-08T03:30:52", "url": "https://files.pythonhosted.org/packages/4e/18/61843543d219c84ebe6e52c79cfce75dae8b573971aad927e2ea7ab939c6/pycopyql-0.4.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b87568bfbfb7420e61168663d5488556", "sha256": "6674c9c3e403c2e6fb92893d803b542cc35e4a310ba63f4b425fb926934fce98" }, "downloads": -1, "filename": "pycopyql-0.4.0.tar.gz", "has_sig": false, "md5_digest": "b87568bfbfb7420e61168663d5488556", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 11681, "upload_time": "2018-09-08T03:30:53", "url": "https://files.pythonhosted.org/packages/cf/8b/6f7e19d9794ed3e8bb8314c09e92e44d7592bc9c45b713977099f0dd15e7/pycopyql-0.4.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "7aeed300c56097062bc57be5b8aa3f6a", "sha256": "0eca8b4a5c2747d39859e59429d6c770fda5ee813be3a54f811cc2c43c6d3c7c" }, "downloads": -1, "filename": "pycopyql-0.4.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "7aeed300c56097062bc57be5b8aa3f6a", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.5", "size": 11623, "upload_time": "2018-09-08T03:30:52", "url": "https://files.pythonhosted.org/packages/4e/18/61843543d219c84ebe6e52c79cfce75dae8b573971aad927e2ea7ab939c6/pycopyql-0.4.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b87568bfbfb7420e61168663d5488556", "sha256": "6674c9c3e403c2e6fb92893d803b542cc35e4a310ba63f4b425fb926934fce98" }, "downloads": -1, "filename": "pycopyql-0.4.0.tar.gz", "has_sig": false, "md5_digest": "b87568bfbfb7420e61168663d5488556", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 11681, "upload_time": "2018-09-08T03:30:53", "url": "https://files.pythonhosted.org/packages/cf/8b/6f7e19d9794ed3e8bb8314c09e92e44d7592bc9c45b713977099f0dd15e7/pycopyql-0.4.0.tar.gz" } ] }