{ "info": { "author": "Jan Graichen", "author_email": "jgraichen@altimos.de", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Environment :: Plugins", "Intended Audience :: System Administrators", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Topic :: System :: Systems Administration" ], "description": "

\"Salt


\n\n# Salt Tower - A Flexible External Pillar Module\n\n[![Build Status](https://travis-ci.org/jgraichen/salt-tower.svg?branch=master)](https://travis-ci.org/jgraichen/salt-tower)\n\nSalt Tower is an advanced and flexible `ext_pillar` that gives access to pillar values while processing and merging them, can render all usual salt file formats and include private and binary files for a minion.\n\nSalt Tower is inspired by [pillarstack](https://github.com/bbinet/pillarstack) for merging pillar files and giving access to them. It also has a [top file](#top-file) like salt itself and utilizes salt renderers to supports all formats such as YAML, Jinja, Python and any combination. Supercharged [renderers for plain text and YAML](#yamlet-renderer) are included too.\n\nEach tower data file is passed the current processed pillars. They can therefore access previously defined values. Data files can include other files that are all merged together.\n\nSalt Tower is designed to completely replace the usual pillar repository or can be utilized beside salts original pillar that e.g. can bootstrap a salt master with Salt Tower.\n\n## Installation\n\nRecommended installation is using `pip` / `pip3` on the salt master:\n\n```\n$ pip install salt-tower\n```\n\n#### Manual installation\n\nInstall the extension files from the `salt_tower/{pillar,renderers}` directories into the `extension_modules` directory configured in salt.\n\n## Configuration\n\nSalt Tower is configured as an `ext_pillar`:\n\n```yaml\next_pillar:\n - tower: /path/to/tower.sls\n```\n\n### Top File\n\nThe tower file is similar to the usual `top.sls` with some important differences.\n\n##### Ordered matchers\n\nPillar top items are ordered and processed in order of appearance. You can therefore define identical matchers multiple times.\n\n```yaml\nbase:\n - '*':\n - first\n\n - '*':\n - second\n```\n\n##### Common includes\n\nYou do not need to define a matcher at all, the files will be included for all minions. You also can use globs to match multiple files, e.g. include all files from `common/`.\n\n```yaml\nbase:\n - common/*\n```\n\n##### Grains\n\nThe top file itself is rendered using the default renderer (`yaml|jinja`). Therefore you can use e.g. `grains` to include specific files.\n\n```yaml\nbase:\n - common/*\n - dist/{{ grains['oscodename'] }}\n```\n\n##### Embedded data\n\nYou can directly include pillar data into the top file simply be defining a `dict` item.\n\n```yaml\nbase:\n - '*.a.example.org':\n - site:\n id: a\n name: A Site\n```\n\n##### Iterative pillar processing\n\nAll matchers are compound matchers by default. As items are processes in order of appearance, later items can patch on previously defined pillar values. The above example includes `application.sls` for any minion matching `*.a.example.org` simply because it defines a `site` pillar value.\n\n```yaml\nbase:\n - '*.a.example.org':\n - site: {id: a, name: A Site}\n\n - 'I@site:*':\n - applications\n```\n\n##### Late-bound variable replacement\n\nFile includes are pre-processed by a string formatter to late-bind pillar values.\n\n```yaml\nbase:\n - '*.a.example.org':\n - site: {id: a, env: production}\n\n - '*.a-staging.example.org':\n - site: {id: a, env: staging}\n\n - 'I@site:*':\n - site/default\n - site/{site.id}\n - site/{site.id}/{site.env}/*\n```\n\nIn the above example a minion `node0.a-staging.example.org` will include the following files:\n\n```\nsite/default\nsite/a\nsite/a/staging/*\n```\n\n##### File lookup\n\nFile names will be matches to files and directories, e.g. when including `path/to/file` the first existing match will be used:\n\n```\npath/to/file\npath/to/file.sls\npath/to/file/init.sls\n```\n\n### Tower Data File\n\nA data file is processed like a usual pillar file. Rendering uses salts template engines therefore all usual features should be available.\n\nThe injected `pillar` objects can be used to access previously defined values. The additional `.get` method allows to traverse the pillar tree.\n\n```yaml\napplication:\n title: Site of {{ pillar.get('tenant:name') }}\n```\n\n**Note:** Using `salt['pillar.get']()` will *not* work.\n\nTower data files can be [any supported template format](https://docs.saltstack.com/en/latest/ref/renderers/) including python files:\n\n```py\n#!py\n\ndef run():\n ret = {'databases': []}\n\n for app in __pillar__['application']:\n ret['databases'].append({\n 'name': '{0}-{1}'.format(app['name'], app['env'])\n })\n\n return ret\n```\n\n##### Includes\n\nPillar data files can include other pillar files similar to how states can be included:\n\n```yaml\ninclude:\n - another/pillar\n\ndata: more\n```\n\nIncluded files cannot be used in the pillar data file template itself but are merge in the pillar before the new pillar data. Includes can be relative to the current file by prefixing a dot:\n\n```yaml\ninclude:\n - file/from/pillar/root.sls\n - ./adjacent_file.sls\n - ../parent_file.sls\n```\n\n### Yamlet renderer\n\nThe Yamlet renderer is an improved YAML renderer that supports loading other files and rendering templates:\n\n```yaml\nssh_private_key: !read id_rsa\nssh_public_key: !read id_rsa.pub\n```\n\nThis reads a file from the pillar directory in plain text or binary and embeds it into the pillar. This eases shipping private files to minions.\n\nUsing the `!include` tag files can be pushed through salts rendering pipeline on the server:\n\n```yaml\nnginx:\n sites:\n my-app: !include ../files/site.conf\n```\n\n```\n#!jinja | text strip\nserver {\n listen {{ pillar.get('my-app:ip') }}:80;\n root /var/www/my-app;\n}\n```\n\nThe pillar will return the following:\n\n```yaml\nnginx:\n sites:\n my-app: |\n server {\n listen 127.0.0.1:80;\n root /var/www/my-app;\n }\n```\n\nThis can greatly simplify states as they only need to drop pillar values into config files and restart services:\n\n```sls\nnginx:\n pkg.installed: []\n service.running: []\n\n{% for name, site in pillar.get('nginx:sites').items() %}\n/etc/nginx/sites-enabled/{{ name }}:\n file.managed:\n - contents_pillar: nginx:sites:{{ name }}\n - makedirs: True\n - watch_in:\n - service: nginx\n{% endfor %}\n```\n\nThe yamlet renderer `!include` macro does accept context variables too:\n\n```yaml\nnginx:\n sites:\n my-app: !include\n source: ../files/site.conf\n context:\n listen_ip: 127.0.0.1\n```\n\n```\n#!jinja | text strip\nserver {\n listen {{ listen_ip }}:80;\n root /var/www/my-app;\n}\n```\n\n### Text renderer\n\nThe text renderer (used above) renders a file as plain text. It stripes the shebang and can optionally strip whitespace from the beginning and end.\n\n```\n#!text strip\n\nHello World\n```\n\nThis will return:\n\n```\nHello World\n```\n\nThe text renderer usually is used for embedding rendered configuration files into a Yamlet template.\n\n### Advanced usage (very dangerous)\n\nThe pillar object passed to the python template engine is the actual mutable dict reference used to process and merge the data. It is possible to modify this dict e.g. in a python template without returning anything:\n\n```python\n#!py\n\nimport copy\n\ndef run():\n databases = __pillar__['databases']\n default = databases.pop('default') # Deletes from actual pillar\n\n for name, config in databases.items():\n databases[name] = dict(default, **config)\n\n return {}\n```\n\n*Note 1:* Do not return `None`. Otherwise [Salt will render the template twice](https://github.com/saltstack/salt/blame/v2019.2.0/salt/template.py#L108) and all side-effects will be applied twice.\n\n*Note 2:* The `__pillar__` object in Python templates is different to other template engines. It is a dict and does not allow to traverse using `get`.\n\n```py\n#!py\n\ndef run():\n return {\n 'wrong': __pilar__.get('tenant:name'),\n 'python': __pillar__['tenant']['name'],\n 'alternative': tower.get('tenant:name')\n }\n```\n\nThe above example demonstrates different usages. The first example will only work if the pillar contains an actual `tenant:name` top-level key. The second example is idiomatic-python but will raise an error if the keys do not exist. The third example uses the additional `tower` helper module to traverse the pillar data.\n\nThe `tower` pillar object is available in all rendering engines and can be used for low-level interaction with the ext_pillar engine. Some available functions are:\n\n##### tower.get(key, default=None)\n\nGet a pillar value by given traverse path:\n\n```python\ntower.get('my:pillar:key')\n```\n\n##### tower.update(dict)\n\nMerges given dictionary into the pillar data.\n\n```python\ntower.update({'my': {'pillar': 'data'}})\n\nassert tower.get('my:pillar') == 'data'\n```\n\n##### tower.merge(tgt, *objects)\n\nMerges given dictionaries or lists into the first one.\n\nNote: The first given dictionary or list is *mutated* and returned.\n\n```python\ntgt = {}\n\nret = tower.merge(tgt, {'a': 1})\n\nassert ret is tgt\nassert tgt['a'] == 1\n```\n\n##### tower.format(obj, *args, **kwargs)\n\nPerforms recursive late-bind string formatting using tower pillar and given arguments ad keywords for resolving. Uses `string.Formatter` internally.\n\n```python\ntower.update({\n 'database': {\n 'password': 'secret'\n }\n})\n\nret = tower.format('postgres://user@{database.password}/db')\n\nassert ret == 'postgres://user@secret/db'\n```\n\nFormat accept dictionaries and list as well an can therefore be used to format full or partial pillar data, this can be used to e.g. format defaults with extra variables:\n\n```python\n#!py\n\ndef run():\n returns = {}\n defaults = __pillar__['default_app_config']\n # e.g. {\n # 'database': 'sqlite:///opt/{name}.sqlite'\n # 'listen': '0.0.0.0:{app.port}'\n # }\n\n for name, conf in __pillar__['applications'].items():\n # Merge defaults with conf into new dictionary\n conf = tower.merge({}, defaults, conf)\n\n # Format late-bind defaults with application config\n conf = tower.format(conf, name=name, app=conf)\n\n returns[name] = conf\n\n return {'applications': returns}\n```\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/jgraichen/salt-tower", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "salt-tower", "package_url": "https://pypi.org/project/salt-tower/", "platform": "", "project_url": "https://pypi.org/project/salt-tower/", "project_urls": { "Homepage": "https://github.com/jgraichen/salt-tower" }, "release_url": "https://pypi.org/project/salt-tower/1.1.0/", "requires_dist": null, "requires_python": "", "summary": "A Flexible External Salt Pillar Module", "version": "1.1.0" }, "last_serial": 5292491, "releases": { "1.1.0": [ { "comment_text": "", "digests": { "md5": "6535d4ada8a3c06fba286c4f6711afab", "sha256": "e67ddcc01820babc2b6ad725008988298332da3297c110f2eb8554be4f6b048b" }, "downloads": -1, "filename": "salt_tower-1.1.0-py2.py3-none-any.whl", "has_sig": true, "md5_digest": "6535d4ada8a3c06fba286c4f6711afab", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 11991, "upload_time": "2019-05-20T13:55:26", "url": "https://files.pythonhosted.org/packages/0a/fc/c4ef05d87d3f6a549c05b853e5674cd4f63fae1f13b9a815e030a57d2dd2/salt_tower-1.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0ec2937ce694ec1a5ca02d0ca2a9b03d", "sha256": "5911e29654c880d545ea5fbbdb035f93c9c9bb59cb74c69d3191a515cd63abeb" }, "downloads": -1, "filename": "salt-tower-1.1.0.tar.gz", "has_sig": true, "md5_digest": "0ec2937ce694ec1a5ca02d0ca2a9b03d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13838, "upload_time": "2019-05-20T13:55:28", "url": "https://files.pythonhosted.org/packages/7b/23/09175b746fdfa062b41d1db629c2b1fa8bd12be25365225e46a1601a8540/salt-tower-1.1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "6535d4ada8a3c06fba286c4f6711afab", "sha256": "e67ddcc01820babc2b6ad725008988298332da3297c110f2eb8554be4f6b048b" }, "downloads": -1, "filename": "salt_tower-1.1.0-py2.py3-none-any.whl", "has_sig": true, "md5_digest": "6535d4ada8a3c06fba286c4f6711afab", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 11991, "upload_time": "2019-05-20T13:55:26", "url": "https://files.pythonhosted.org/packages/0a/fc/c4ef05d87d3f6a549c05b853e5674cd4f63fae1f13b9a815e030a57d2dd2/salt_tower-1.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0ec2937ce694ec1a5ca02d0ca2a9b03d", "sha256": "5911e29654c880d545ea5fbbdb035f93c9c9bb59cb74c69d3191a515cd63abeb" }, "downloads": -1, "filename": "salt-tower-1.1.0.tar.gz", "has_sig": true, "md5_digest": "0ec2937ce694ec1a5ca02d0ca2a9b03d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13838, "upload_time": "2019-05-20T13:55:28", "url": "https://files.pythonhosted.org/packages/7b/23/09175b746fdfa062b41d1db629c2b1fa8bd12be25365225e46a1601a8540/salt-tower-1.1.0.tar.gz" } ] }