{ "info": { "author": "Shalom Bhooshi", "author_email": "s.bhooshi@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Information Technology", "Intended Audience :: System Administrators", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Operating System :: POSIX :: Linux", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Software Development :: Build Tools", "Topic :: Software Development :: Version Control", "Topic :: Software Development :: Version Control :: Git", "Topic :: System :: Software Distribution", "Topic :: System :: Systems Administration" ], "description": "[![pipeline status](https://gitlab.com/s.bhooshi/gitlab-job-guard/badges/master/pipeline.svg)](https://gitlab.com/s.bhooshi/gitlab-job-guard/commits/master)\n\n# gitlab-job-guard\n\nGuard Gitlab CI pipeline jobs from multiple parallel executions.\n\n```bash\n$ PRIVATE_TOKEN=\"$GITLAB_API_TOKEN\" gitlab-job-guard\n$ my-unguarded-deployment-task --to=production\n```\n\n`gitlab-job-guard` will block if it detects other pipelines running for\nthe current project to avoid multiple pipelines from clobbering up a\ndeployment/environment. This is especially a problem when jobs are distributed\nand picked up by multiple gitlab runners.\n\nWhile gitlab will _auto-cancel redundant, pending pipelines_ for the same branch\nby default - this is not the case for multiple pipelines from _different\nbranches_ targeting a particular deployment/environment. Gitlab has no way to\ndetect or control these user-defined branch-to-environment mappings and this\nmeans environments can easily be left in an unsafe/broken state. (e.g.\n`terraform apply` or `ansible`, etc from different pipelines running at the same\ntime).\n\n`gitlab-job-guard` uses the Gitlab API to determine if existing pipelines are\nscheduled and to backoff-and-retry until it is safe to proceed even if jobs are\ndistributed across multiple runners. Conflicts are detected by user-defined\nmatches on pipeline ref names (branch, tag, etc) and/or pipeline status.\n\n## Usage\n\nThe simplest use-case would likely be placing `gitlab-job-guard` in a\n`before_script` section in your `gitlab-ci.yml` to protect all jobs (though this\ncan slow things down and be unfriendly to your local gitlab-ops).\n\n> `$GITLAB_API_TOKEN` is a [Personal Access\nToken](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#creating-a-personal-access-token)\nwith at minimum `api` scope.\n\n```yaml\nbefore_script:\n - PRIVATE_TOKEN=\"$GITLAB_API_TOKEN\" gitlab-job-guard\n```\n\nThough often, most pipelines only need to guard the critical jobs that share\ncommon state/data (i.e. a provisioning/deployment job, an artifact build/release\njob, etc).\n\n```yaml\ndeploy-production:\n stage: deploy\n script:\n - PRIVATE_TOKEN=\"$GITLAB_API_TOKEN\" gitlab-job-guard\n - my-unguarded-deployment-task --to=production\n```\n\ne.g To guard something critical like a `terraform` job running for a `tag`.\n\n```yaml\nprovision-infrastructure:\n stage: provision\n script:\n - export PRIVATE_TOKEN=\"$GITLAB_API_TOKEN\"\n - gitlab-job-guard --guard-ref-regex='^v[0-9\\.]+' # Regex matches tags\n - terraform plan ...\n - terraform apply ...\n only:\n - tags\n```\n\n### Other usages\n\n`--guard-ref-regex` takes a regular expression to detect conflicts on other\npipelines whose ref name (branch, tag) is a (partial) match. This allows for\nguarding of jobs in the complex use-cases.\n\nTo hold jobs for a collisions on pattern matches on the branch/tag names.\n\n```bash\ngitlab-job-guard -c=^master$ # Match branch names matching 'master' exactly\n\ngitlab-job-guard -c=^(master|dev(elop)?)$ # Match any of the mainline branches\n\ngitlab-job-guard -c=^(feature|release|hotfix)/ # Match any gitflow transient branch prefixes\n\ngitlab-job-guard -c=^[0-9]\\- # Match branch names beginning with a number\n # and dash ignoring all other text.\n # e.g. a gitlab branch made from an issue\n\ngitlab-job-guard -c=^v?[\\d.]+$ # Match (semver) tags like v1.0.9, 2.0\n\ngitlab-job-guard -c=^environment/ # Match any environment deployments?\n\ngitlab-job-guard -c=^environment/dc1.+ # Match environment deployments to DC1?\n\ngitlab-job-guard -c=\"$CI_BUILD_REF_NAME\" # Match current branch name (partially).\n # i.e. 'master' matches 'feature/master-document'\n\ngitlab-job-guard -c=\"^$CI_BUILD_REF_NAME$\" # Match current branch name (exactly).\n # i.e. 'master' does not match 'master-deployment'\n\ngitlab-job-guard -c='.+' -s='running|pending' # Match any pipeline in running or pending state\n```\n\nTo hold a job for a collision on part of the ref name (e.g. on branch prefix\nsuch as `feature/` or `hotfix/` or `release/`, etc _a la_ `gitflow`).\n\n```bash\n# Assuming current pipeline is for the 'feature/foo' branch and that\n# CI_BUILD_REF_NAME=feature/foo\n\n# Extract the branch prefix\nCI_BUILD_REF_PREFIX=$(echo \"$CI_BUILD_REF_NAME\" | sed -r 's@(.+/)(.+)@\\1@')\n\n# CI_BUILD_REF_PREFIX now contains 'feature/'\n# This will now block if there are any other pipelines running or pending\n# for feature/ branches.\ngitlab-job-guard -c=\"^$CI_BUILD_REF_PREFIX\" -s='running|pending'\n```\n\n## Other arguments\n\nTypically `gitlab-job-guard` reads in the following environment variables\nto compose its arguments. NOTE: environment variables take precedence\nover arguments passed at the CLI and so only the `user-specified`\narguments/envvars need to be set. Setting these as [protected environment\nvariables](https://docs.gitlab.com/ee/ci/variables/#protected-variables) avoids\nleaking sensitive information and needing to duplicate them for every call.\n\n```\nenvvar | argument | source\n------------------- --------------------- ----------------\nGUARD_REF_REGEX | --guard-ref-regex | user specified\nGUARD_STATUS_REGEX | --guard-status-regex | user specified\nPRIVATE_TOKEN | --private-token | user specified\nGUARD_TIMEOUT | --guard-timeout | user specified\nCI_PROJECT_ID | --ci-project-id | runner environment\nCI_BUILD_REF_NAME | --ci-build-ref-name | runner environment\nCI_API_V4_URL | --ci-api-v4-url | runner environment\nCI_PIPELINE_ID | --ci-pipeline-id | runner environment\n```\n\nSee `gitlab-job-guard -h` for the full argument set and current/default values.\n\n## Behaviour\n\n`gitlab-job-guard` will backoff-and-retry until it is safe to proceed or a\ntimeout is reached. An exit code of `0` indicates no other pipelines conflict\nand that the job can continue. On reaching the timeout (`1 hour` by default to\nalign with the default timeout for gitlab jobs), it will return an error to fail\nthe job. The timeout in seconds can be set via `-w/--guard-timeout`.\n\nIf a fail-fast approach is needed, the `-x/--no-wait` flag will cause\n`gitlab-job-guard` to exit immediately on conflicts. (Though, this means the\npipeline run will have to be restarted some other way or abandoned).\n\n# TODO\n\nFor long-running pipelines, this solution can have subtle consequences with\ngrowing queues and increased contention and unpredictability as to which\npipeline is the first-past-the-post. An older pipeline taking precedence over\nnewer commits if often not desired and newer pipelines always winning is\nprobably desired.\n\n* Handle existing conflicting pipelines - cancel them or give-way.\n* Narrow down conflicts to jobs (`CI_JOB_NAME`) or stages (`CI_JOB_STAGE`)\n so that other parts of the pipelines that do not share state are allowed to\n run freely.\n\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://gitlab.com/s.bhooshi/gitlab-job-guard", "keywords": "gitlab-ci gitlab-job gitlab-job-guard pipeline job guard", "license": "Apache License 2.0", "maintainer": "", "maintainer_email": "", "name": "gitlab-job-guard", "package_url": "https://pypi.org/project/gitlab-job-guard/", "platform": "Linux", "project_url": "https://pypi.org/project/gitlab-job-guard/", "project_urls": { "Homepage": "https://gitlab.com/s.bhooshi/gitlab-job-guard" }, "release_url": "https://pypi.org/project/gitlab-job-guard/0.0.4.3/", "requires_dist": [ "future", "requests", "six" ], "requires_python": ">=2.7, >=2.7.1, !=3.0, !=3.0.*, !=3.1, !=3.1.*, !=3.2, !=3.2.*, !=3.3, !=3.3.*, !=3.4, !=3.4.*", "summary": "Guard gitlab jobs from multiple simultaneous executions", "version": "0.0.4.3" }, "last_serial": 5157487, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "270af9dd40fb093c7d640114ae034497", "sha256": "fcfb1fb6c9050f217062c56906607e1ee89e9097b291032abe10c63d3057a797" }, "downloads": -1, "filename": "gitlab_job_guard-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "270af9dd40fb093c7d640114ae034497", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=2.7, >=3.5, !=3.0, !=3.0.*, !=3.1, !=3.1.*, !=3.2, !=3.2.*, !=3.3, !=3.3.*, !=3.4, !=3.4.*", "size": 13157, "upload_time": "2019-04-17T23:06:21", "url": "https://files.pythonhosted.org/packages/1a/50/337b8fbbb978e13568f4ad93d7e7c6be5f67dabbe50250a1c26fdceb9bf0/gitlab_job_guard-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a85b81ebc0ca77b37934a0cdb157ff6a", "sha256": "76e9a5f8c001fbe27e7fbed27d8628c270f35f81664a7a6f94d2b33b15a5f0f1" }, "downloads": -1, "filename": "gitlab-job-guard-0.0.1.tar.gz", "has_sig": false, "md5_digest": "a85b81ebc0ca77b37934a0cdb157ff6a", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, >=3.5, !=3.0, !=3.0.*, !=3.1, !=3.1.*, !=3.2, !=3.2.*, !=3.3, !=3.3.*, !=3.4, !=3.4.*", "size": 4747, "upload_time": "2019-04-17T23:06:22", "url": "https://files.pythonhosted.org/packages/01/eb/02ef2fd23af8cc66842db3e5cefd8f4e3240d9ce1c8a94d067a09f93569f/gitlab-job-guard-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "45a114937f387fa3a4d9c5a9a4578392", "sha256": "9a9e1058d41ad6597a0f7ffabdbec986be16c63adabb9dc6811768be61eb56a4" }, "downloads": -1, "filename": "gitlab_job_guard-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "45a114937f387fa3a4d9c5a9a4578392", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=2.7, >=2.7.1, !=3.0, !=3.0.*, !=3.1, !=3.1.*, !=3.2, !=3.2.*, !=3.3, !=3.3.*, !=3.4, !=3.4.*", "size": 19640, "upload_time": "2019-04-17T23:05:40", "url": "https://files.pythonhosted.org/packages/0b/52/98fb7d15379b44bde82e5ad86ead4553b35614e52ff1b06e77ca631e18bc/gitlab_job_guard-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2fe917a3f94c02bfcc602e4960058ed0", "sha256": "505d53a49cc30414cee1c67c4c60981c3d5587e8f9cad9996bc58ca03787b085" }, "downloads": -1, "filename": "gitlab-job-guard-0.0.2.tar.gz", "has_sig": false, "md5_digest": "2fe917a3f94c02bfcc602e4960058ed0", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, >=2.7.1, !=3.0, !=3.0.*, !=3.1, !=3.1.*, !=3.2, !=3.2.*, !=3.3, !=3.3.*, !=3.4, !=3.4.*", "size": 7892, "upload_time": "2019-04-17T23:05:41", "url": "https://files.pythonhosted.org/packages/96/a7/cf3527be51d2770a09b3f75d54adec7718e9d43ec061f6eda4aa2564578a/gitlab-job-guard-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "6124b72fc7a3c892f91e170ba7ee59ad", "sha256": "f68c686b9dd2914f73ed21eba769fb2173d54f3a2bafd4790fef989ea0815736" }, "downloads": -1, "filename": "gitlab_job_guard-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "6124b72fc7a3c892f91e170ba7ee59ad", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=2.7, >=2.7.1, !=3.0, !=3.0.*, !=3.1, !=3.1.*, !=3.2, !=3.2.*, !=3.3, !=3.3.*, !=3.4, !=3.4.*", "size": 19657, "upload_time": "2019-04-16T22:08:31", "url": "https://files.pythonhosted.org/packages/04/6e/199634b824cb0c43fa93aae4f64c78d04f3921974613b77c2cd2e6185a83/gitlab_job_guard-0.0.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8c5785c35c4a148a6253eb668ce207b4", "sha256": "39eae87151edc9b31aa9c5691895abf0b0e07d51bfc070bdcd6fa3c6fc262113" }, "downloads": -1, "filename": "gitlab-job-guard-0.0.3.tar.gz", "has_sig": false, "md5_digest": "8c5785c35c4a148a6253eb668ce207b4", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, >=2.7.1, !=3.0, !=3.0.*, !=3.1, !=3.1.*, !=3.2, !=3.2.*, !=3.3, !=3.3.*, !=3.4, !=3.4.*", "size": 7910, "upload_time": "2019-04-16T22:08:34", "url": "https://files.pythonhosted.org/packages/7c/70/b87aa8d2fba83b9c9d1375c42cf0cc2b98666595e0690787670a6ace46df/gitlab-job-guard-0.0.3.tar.gz" } ], "0.0.4.3": [ { "comment_text": "", "digests": { "md5": "ad9953a966af6721cfae26019f18fe98", "sha256": "3c69058801863147a1f26481b8c62a3067f889afe842039bbe12fcde9a06b434" }, "downloads": -1, "filename": "gitlab_job_guard-0.0.4.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "ad9953a966af6721cfae26019f18fe98", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, >=2.7.1, !=3.0, !=3.0.*, !=3.1, !=3.1.*, !=3.2, !=3.2.*, !=3.3, !=3.3.*, !=3.4, !=3.4.*", "size": 20997, "upload_time": "2019-04-17T00:58:39", "url": "https://files.pythonhosted.org/packages/37/7b/5208d0c0420be8ed4b3b35bcec9d0b05e3d9b8bf9ceafa368474f248c0b4/gitlab_job_guard-0.0.4.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9c3cacafec317ea1c76aa538e48fa8ac", "sha256": "d716da73b6edabfb385263508c25741cdcff11ddbe24a2d9a2fdc418534e9e98" }, "downloads": -1, "filename": "gitlab-job-guard-0.0.4.3.tar.gz", "has_sig": false, "md5_digest": "9c3cacafec317ea1c76aa538e48fa8ac", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, >=2.7.1, !=3.0, !=3.0.*, !=3.1, !=3.1.*, !=3.2, !=3.2.*, !=3.3, !=3.3.*, !=3.4, !=3.4.*", "size": 11751, "upload_time": "2019-04-17T00:58:40", "url": "https://files.pythonhosted.org/packages/00/ca/23909dfce1a4c2f9d378975aa90073b412356c609fda28a7688025007cdc/gitlab-job-guard-0.0.4.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ad9953a966af6721cfae26019f18fe98", "sha256": "3c69058801863147a1f26481b8c62a3067f889afe842039bbe12fcde9a06b434" }, "downloads": -1, "filename": "gitlab_job_guard-0.0.4.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "ad9953a966af6721cfae26019f18fe98", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, >=2.7.1, !=3.0, !=3.0.*, !=3.1, !=3.1.*, !=3.2, !=3.2.*, !=3.3, !=3.3.*, !=3.4, !=3.4.*", "size": 20997, "upload_time": "2019-04-17T00:58:39", "url": "https://files.pythonhosted.org/packages/37/7b/5208d0c0420be8ed4b3b35bcec9d0b05e3d9b8bf9ceafa368474f248c0b4/gitlab_job_guard-0.0.4.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9c3cacafec317ea1c76aa538e48fa8ac", "sha256": "d716da73b6edabfb385263508c25741cdcff11ddbe24a2d9a2fdc418534e9e98" }, "downloads": -1, "filename": "gitlab-job-guard-0.0.4.3.tar.gz", "has_sig": false, "md5_digest": "9c3cacafec317ea1c76aa538e48fa8ac", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, >=2.7.1, !=3.0, !=3.0.*, !=3.1, !=3.1.*, !=3.2, !=3.2.*, !=3.3, !=3.3.*, !=3.4, !=3.4.*", "size": 11751, "upload_time": "2019-04-17T00:58:40", "url": "https://files.pythonhosted.org/packages/00/ca/23909dfce1a4c2f9d378975aa90073b412356c609fda28a7688025007cdc/gitlab-job-guard-0.0.4.3.tar.gz" } ] }