{ "info": { "author": "Eirik Krogstad", "author_email": "eirikkr@gmail.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7" ], "description": "migrado\n=======\n\n[![PyPI package](https://badge.fury.io/py/migrado.svg)](https://pypi.org/project/migrado/)\n[![Build status](https://travis-ci.org/protojour/migrado.svg?branch=master)](https://travis-ci.org/protojour/migrado)\n\n\ud83e\udd51 ArangoDB migrations and batch processing manager.\n\nmigrado is a command-line client that can help build and run schema or data migrations against your ArangoDB instance. \n\nmigrado utilizes ArangoDB Transactions when running data migrations to ensure failed scripts are rolled back automatically. [Docker](https://docs.docker.com/install/) or arangosh from the [ArangoDB Client Tools](https://www.arangodb.com/download-major/) is required to run schema migrations, however no transaction safety is available at this point.\n\n**migrado should be considered alpha software.** Make sure you test well before using in a production setting.\n\nIf you have trouble, open an issue. Contributions are welcome.\n\nInstallation\n------------\n\nmigrado requires Python 3.6 or higher.\n\n```bash\n$ pip install --user migrado\n```\n\nUsage\n-----\n\nmigrado can create a migrations directory and generate an initial set of collections from the given schema file:\n\n```bash\n$ migrado init --schema schema.yml\n```\n\nSee [YAML schemas](#yaml-schemas) for details. If no schema is specified, migrado will create an empty initial migration.\n\nTo make a new template migration script:\n\n```bash\n$ migrado make --name rewrite_names\n```\n\nThis will create a new file, `migrations/0002_rewrite_names.js` (`--name` is optional), which you can edit as you see fit. See [Migration scripts](#migration-scripts) for details.\n\nWhen you are ready, run all migrations not currently ran against the database: \n\n```bash\n$ migrado run\n```\n\nmigrado stores migration state in a configurable collection, see `--help` or [Environment vars](#environment-vars) for details.\n\nIf you wrote a `reverse()` migration, you can revert to an earlier point by specifying a target migration id. To revert to the initial migration:\n\n```bash\n$ migrado run --target 0001\n```\n\nUse the `--help` option for help on any command when using the client.\n\nEnvironment vars\n----------------\n\nThe following environment variables are employed by migrado:\n\n- `MIGRADO_PATH`: Specifies the path to the migrations directory, replaces `-p`, `--path` (default: `migrations`).\n- `MIGRADO_DB`: Specifies the ArangoDB database name for generated migrations to interact with, replaces `-d`, `--db` (no default, but required for the `run` command).\n- `MIGRADO_COLL`: Specifies ArangoDb collection name to store migration state in, replaces `-c`, `--state-coll` (default: `migrado`).\n- `MIGRADO_TLS`: Use TLS for connection when running migrations, replaces `-T`, `--tls` (default: `False`).\n- `MIGRADO_HOST`: Specifies the database host for running migrations, replaces `-H`, `--host` (default: `localhost`).\n- `MIGRADO_PORT`: Specifies the database port for running migrations, replaces `-P`, `--port` (default: `8529`).\n- `MIGRADO_USER`: Specifies the database username for running migrations, replaces `-U`, `--username` (no default).\n- `MIGRADO_PASS`: Specifies the database password for running migrations, replaces `-W`, `--password` (no default).\n- `MIGRADO_DOCKER_IMAGE`: Specifies the Docker image (and optionally tag) for the container running migrations, replaces `-I`, `--docker-image` (default: `arangodb`).\n- `MIGRADO_DOCKER_NETWORK`: Specifies a Docker network mode or name for running migrations, replaces `-N`, `--docker-network`. Valid values are `host` (default, use the host network), or any network name.\n- `MIGRADO_DOCKER_SERVICE`: Specifies a Docker service to connect to when running migrations, replaces `-S`, `--docker-service` (default: same value as `MIGRADO_HOST`).\n\nIf your ArangoDB instance is itself running on Docker, `MIGRADO_DOCKER_NETWORK` and `MIGRADO_DOCKER_SERVICE` may be configured so the created container can connect easily to the correct network and service.\n\nFor connection to the Docker daemon, additional environment variables are accepted; `DOCKER_HOST`, `DOCKER_TLS_VERIFY` and `DOCKER_CERT_PATH`. They are same as those used by the Docker command-line client.\n\nYAML schemas\n------------\n\nArangoDB may be schemaless, but in a larger project it still makes sense to keep a schema spec up to date, both for an overview of collections and their data structures, and as a basis for validation.\n\nmigrado uses a schema model based on JSON Schema, in YAML, and can use this to generate an initial migration for the collections available in your database.\n\nExample schema:\n\n```yaml\n---\nall: &all\n _id:\n type: string\n readOnly: true\n _key:\n type: string\n readOnly: true\n _rev:\n type: string\n readOnly: true\n\nedges: &edges\n _from:\n type: string\n _to:\n type: string\n\ncollections:\n\n books:\n type: object\n properties:\n <<: *all\n title:\n type: string\n isbn:\n type: string\n required:\n - title\n - isbn\n\n authors: \n # Note, you do not actually need to specify the object schema,\n # but they can be used in API specs (e.g. OpenAPI) and/or validation,\n # and may be handled by migrado in the future.\n\nedge_collections:\n\n # authors --> books \n author_of:\n type: object\n properties:\n <<: *all\n <<: *edges\n required:\n - _from\n - _to\n``` \n\nMigration scripts\n-----------------\n\nMigration scripts are structured so they may be parsed and run easily by both migrado and ArangoDB. In addition, they are structured so they may be run manually against ArangoDB using `arangosh`.\n\nThere are two types of script, **data** and **schema** migration scripts.\n\n### Data migrations\n\nYou need to declare all collections subject to write operations using the syntax `// write collection_name`, because ArangoDB needs this information for locking during transactions. We've made the declaration explicit to reduce errors. _Attempting to write to collections not declared in this way will cause the migration to fail._\n\nIn general, a reverse migration should do the logical opposite of a forward migration. `forward()` and `reverse()` functions can contain anything that the ArangoDB V8 engine understands, but must be fully self-contained. _Anything outside these functions is ignored and unavailable when running migrations._\n\nHere's an example migration script for adding `new_field` in collection `things`:\n\n```javascript\n// write things\n\nfunction forward() {\n var db = require(\"@arangodb\").db\n db._query(`\n FOR thing IN things\n UPDATE thing WITH { new_field: \"some value\" } IN things\n `)\n}\n\nfunction reverse() {\n var db = require(\"@arangodb\").db\n db._query(`\n FOR thing IN things\n REPLACE thing WITH UNSET(thing, \"new_field\") IN things\n `)\n}\n```\n\nPlease make sure you read [limitations when running transactions](https://www.arangodb.com/docs/stable/transactions-limitations.html) in the ArangoDB documentation. In particular, _creation and deletion of databases, collections, and indexes_ is not allowed in transactions.\n\nIf a migration contains such operations, you will be asked if you want to run the migration through Docker.\n\n### Schema migrations\n\nSchema migrations are stuctured in the same way as data migrations, but are run against `arangosh` in a container from an [official arangodb Docker image](https://hub.docker.com/_/arangodb). There is no transaction safety when running schema migrations.\n\nSchema migrations are structured the same way as data migrations, but `// write` declarations are not required. All operations are allowed.\n\nHere's an example migration script generated from the YAML schema above:\n\n```javascript\nfunction forward() {\n var db = require(\"@arangodb\").db\n db._createDocumentCollection(\"books\")\n db._createDocumentCollection(\"authors\")\n db._createEdgeCollection(\"author_of\")\n}\n\nfunction reverse() {\n var db = require(\"@arangodb\").db\n db._drop(\"books\")\n db._drop(\"authors\")\n db._drop(\"author_of\")\n}\n```\n\nPlease be careful when running schema migrations in reverse. As you can see, the `reverse()` function above would drop your collections if you were to reverse beyond this point. Currently, you will not be able to do so for an initial migration.\n\nTODO\n----\n- [ ] Transaction-like safe runs for schema migrations\n- [ ] Automatic diffing of schema migrations\n- [ ] Using migrado to add indexes\n- [ ] Using migrado to add recurring tasks\n\nLicense\n-------\n\nmigrado is copyright \u00a9 2019 Protojour AS, and is licensed under MIT. See [LICENSE.txt](https://github.com/protojour/migrado/blob/master/LICENSE.txt) for details.\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/protojour/migrado", "keywords": "", "license": "MIT", "maintainer": "Eirik Krogstad", "maintainer_email": "eirikkr@gmail.com", "name": "migrado", "package_url": "https://pypi.org/project/migrado/", "platform": "", "project_url": "https://pypi.org/project/migrado/", "project_urls": { "Homepage": "https://github.com/protojour/migrado", "Repository": "https://github.com/protojour/migrado" }, "release_url": "https://pypi.org/project/migrado/0.3.0/", "requires_dist": [ "click (>=7.0,<8.0)", "pyyaml (>=5.1,<6.0)", "python-arango (>=5.2,<6.0)", "docker[tls] (>=4.0,<5.0)" ], "requires_python": ">=3.6,<4.0", "summary": "ArangoDB migrations and batch processing manager", "version": "0.3.0" }, "last_serial": 5977653, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "3123edfdf9355e0ea02107d9cd3a3723", "sha256": "230bea37ab22062aa690165a15ef899458bb0404d66628c0bc4eb247971f88a2" }, "downloads": -1, "filename": "migrado-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "3123edfdf9355e0ea02107d9cd3a3723", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 9968, "upload_time": "2019-07-11T13:40:06", "url": "https://files.pythonhosted.org/packages/07/fd/f8ad301fcb34ade119ad9596c5be5fcb571e03304165422217707f58c275/migrado-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f06ec325a5a86a668da74f487436193e", "sha256": "f3d800a2a4dec8400006384f284567a81abcebfa040b62246ce6277fb5816cd3" }, "downloads": -1, "filename": "migrado-0.1.0.tar.gz", "has_sig": false, "md5_digest": "f06ec325a5a86a668da74f487436193e", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 11380, "upload_time": "2019-07-11T13:40:08", "url": "https://files.pythonhosted.org/packages/70/af/fe694442b2a7bd291303998ed80271201e8784a57f53116336a1cd2cad2e/migrado-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "68669b856f54e65e180a1c00b01cc5f2", "sha256": "5239e297b8d13247ebceb796f39e82ae8b8299c8a44ed72a2ca8fbcd86e5f69d" }, "downloads": -1, "filename": "migrado-0.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "68669b856f54e65e180a1c00b01cc5f2", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 9993, "upload_time": "2019-07-12T13:00:44", "url": "https://files.pythonhosted.org/packages/c4/2a/8d2a9c42d85a2e570e40e606fbf5ddb9f84c5a17cb3f6ee1d4de6df80614/migrado-0.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "63be1fc8fff788c05ed307d138023832", "sha256": "fa2dda5b35397546c4fb969c347c09bf25a68847a1fff9daefe2aaac73e0dee7" }, "downloads": -1, "filename": "migrado-0.1.1.tar.gz", "has_sig": false, "md5_digest": "63be1fc8fff788c05ed307d138023832", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 11417, "upload_time": "2019-07-12T13:00:46", "url": "https://files.pythonhosted.org/packages/76/6c/e6e6323d53111684954cfbee4ca5b6697cb544b47c7c47a635029cfdb2d5/migrado-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "8fedd15b127b8fcb61e82563bf20b908", "sha256": "ec7d0bfc402efb55b1d69b3a568964b06a28e4c4e7d0cc3aef2aa3e89163a6d6" }, "downloads": -1, "filename": "migrado-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "8fedd15b127b8fcb61e82563bf20b908", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5,<4.0", "size": 10141, "upload_time": "2019-07-30T09:28:10", "url": "https://files.pythonhosted.org/packages/cb/54/78cd33dca6b64dd50a38475c84dccc82b1485d276db7f1cb4924701dc48b/migrado-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "02e9c7d3a5733adc542c494e50c971d0", "sha256": "48de0c962ab6c6fe96066557a31abfdb387ce5458bcf5bac63badeee708293ac" }, "downloads": -1, "filename": "migrado-0.1.2.tar.gz", "has_sig": false, "md5_digest": "02e9c7d3a5733adc542c494e50c971d0", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5,<4.0", "size": 11622, "upload_time": "2019-07-30T09:28:11", "url": "https://files.pythonhosted.org/packages/0d/5c/20cea39d1263b3428a2db3668309d02915372f9fbf00fdf94ac257553407/migrado-0.1.2.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "6027e20b4f1ed98eb7904fdfdbe58188", "sha256": "4a5a05c630dc0283504cc912f33ce1b1eb6601ab6df5c9f749e3669751e66bc9" }, "downloads": -1, "filename": "migrado-0.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "6027e20b4f1ed98eb7904fdfdbe58188", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5,<4.0", "size": 10511, "upload_time": "2019-07-31T13:06:48", "url": "https://files.pythonhosted.org/packages/95/14/563b41854d215950224957bfd406f98b83f78371f0efe8a877248e0ca00c/migrado-0.2.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4fb5e33165479479023220f9f07e975f", "sha256": "f8594d7a8ca6e60e4e0983a7d6d8aabe7fcd46db4617d4796a4f6faa5f3d7587" }, "downloads": -1, "filename": "migrado-0.2.0.tar.gz", "has_sig": false, "md5_digest": "4fb5e33165479479023220f9f07e975f", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5,<4.0", "size": 12025, "upload_time": "2019-07-31T13:06:50", "url": "https://files.pythonhosted.org/packages/96/a4/704edec64bf4b84094bd27b92403f75515ead058379a0c6ce5818be8dbd8/migrado-0.2.0.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "3e92262c3468e96133ee8c8396347885", "sha256": "469f2d39728fd192cc51d4ca1e40d650d458ab8cbe2025824e4a9452996a315d" }, "downloads": -1, "filename": "migrado-0.3.0-py3-none-any.whl", "has_sig": false, "md5_digest": "3e92262c3468e96133ee8c8396347885", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 10654, "upload_time": "2019-10-15T14:25:24", "url": "https://files.pythonhosted.org/packages/c4/65/644a0f9f459e1e82b9dc6fc5d104ca79d29f8010eba5f6efb40db06da581/migrado-0.3.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "db55a8d9b98054108e7ef1e6b37bbc63", "sha256": "56d65e70f4878fa9c16b225a99244982b7619a3625767d462269734c1ad2316a" }, "downloads": -1, "filename": "migrado-0.3.0.tar.gz", "has_sig": false, "md5_digest": "db55a8d9b98054108e7ef1e6b37bbc63", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 12204, "upload_time": "2019-10-15T14:25:26", "url": "https://files.pythonhosted.org/packages/60/2a/38afd8a885b64ff761fe464815f2d1e957cc38d0f6031cec36e7d73419c0/migrado-0.3.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "3e92262c3468e96133ee8c8396347885", "sha256": "469f2d39728fd192cc51d4ca1e40d650d458ab8cbe2025824e4a9452996a315d" }, "downloads": -1, "filename": "migrado-0.3.0-py3-none-any.whl", "has_sig": false, "md5_digest": "3e92262c3468e96133ee8c8396347885", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 10654, "upload_time": "2019-10-15T14:25:24", "url": "https://files.pythonhosted.org/packages/c4/65/644a0f9f459e1e82b9dc6fc5d104ca79d29f8010eba5f6efb40db06da581/migrado-0.3.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "db55a8d9b98054108e7ef1e6b37bbc63", "sha256": "56d65e70f4878fa9c16b225a99244982b7619a3625767d462269734c1ad2316a" }, "downloads": -1, "filename": "migrado-0.3.0.tar.gz", "has_sig": false, "md5_digest": "db55a8d9b98054108e7ef1e6b37bbc63", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 12204, "upload_time": "2019-10-15T14:25:26", "url": "https://files.pythonhosted.org/packages/60/2a/38afd8a885b64ff761fe464815f2d1e957cc38d0f6031cec36e7d73419c0/migrado-0.3.0.tar.gz" } ] }