{ "info": { "author": "William W. Kimball, Jr., MBA, MSIS", "author_email": "github-yamlpath@kimballstuff.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "License :: OSI Approved :: ISC License (ISCL)", "Operating System :: OS Independent", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "# YAML Path and Command-Line Tools\n\n[![Build Status](https://travis-ci.org/wwkimball/yamlpath.svg?branch=master)](https://travis-ci.org/wwkimball/yamlpath)\n[![Python versions](https://img.shields.io/pypi/pyversions/yamlpath.svg)](https://pypi.org/project/yamlpath/)\n[![PyPI version](https://badge.fury.io/py/yamlpath.svg)](https://pypi.org/project/yamlpath/)\n[![Coverage Status](https://coveralls.io/repos/github/wwkimball/yamlpath/badge.svg?branch=master)](https://coveralls.io/github/wwkimball/yamlpath?branch=master)\n\nContents:\n\n1. [Introduction](#introduction)\n2. [Illustration](#illustration)\n3. [Installing](#installing)\n4. [Supported YAML Path Segments](#supported-yaml-path-segments)\n5. [Based on ruamel.yaml and Python 3](#based-on-ruamelyaml-and-python-3)\n6. [The Files of This Project](#the-files-of-this-project)\n 1. [Command-Line Tools](#command-line-tools)\n 2. [Libraries](#libraries)\n7. [Basic Usage](#basic-usage)\n 1. [Basic Usage: Command-Line Tools](#basic-usage--command-line-tools)\n 1. [Rotate Your EYAML Keys](#rotate-your-eyaml-keys)\n 2. [Get a YAML Value](#get-a-yaml-value)\n 3. [Search For YAML Paths](#search-for-yaml-paths)\n 4. [Change a YAML Value](#change-a-yaml-value)\n 2. [Basic Usage: Libraries](#basic-usage--libraries)\n 1. [Initialize ruamel.yaml and These Helpers](#initialize-ruamelyaml-and-these-helpers)\n 2. [Searching for YAML Nodes](#searching-for-yaml-nodes)\n 3. [Changing Values](#changing-values)\n\n## Introduction\n\nThis project presents and utilizes YAML Paths, which are a powerful, intuitive\nmeans of identifying one *or more* nodes within [YAML](https://yaml.org/),\n[EYAML](https://github.com/voxpupuli/hiera-eyaml), or compatible data structures\nlike [JSON](https://www.json.org/). Both dot-notation (inspired by\n[Hiera](https://github.com/puppetlabs/hiera)) and forward-slash-notation\n(influenced by [XPath](https://www.w3schools.com/xml/xml_xpath.asp)) are\nsupported. The [libraries](#libraries) (modules) and several [command-line tool\nimplementations](#command-line-tools) are provided. With these, you can build\nYAML Path support right into your own application or easily use its capabilities\nright away from the command-line to retrieve or update YAML/Compatible data.\n\nThis implementation of YAML Path is a *query language* in addition to a *node\ndescriptor*. With it, you can describe or select a single precise node or\nsearch for any number of nodes that match some criteria. Keys, values, and\nelements can all be searched at any number of levels within the data structure\nusing the same query. Collectors can also be used to gather and further select\nfrom otherwise disparate parts of the source data.\n\n## Illustration\n\nTo illustrate some of these concepts, consider these samples:\n\n```yaml\n---\nhash:\n child_attr:\n key: 5280\n```\n\nThis value, `5280`, can be identified via YAML Path as any of:\n\n1. `hash.child_attr.key` (dot-notation)\n2. `hash.child_attr[.=key]` (search all child keys for one named, `key`, and\n yield its value)\n3. `/hash/child_attr/key` (same as 1 but in forward-slash notation)\n4. `/hash/child_attr[.=key]` (same as 2 but in forward-slash notation)\n\n```yaml\n---\naliases:\n - &first_anchor Simple string value\n```\n\nWith YAML Path, you can select this anchored value by any of these equivalent\nexpressions:\n\n1. `aliases[0]` (explicit array element number)\n2. `aliases.0` (implicit array element number in dot-notation)\n3. `aliases[&first_anchor]` (search by Anchor name)\n4. `aliases[.^Simple]` (search for any elements starting with \"Simple\")\n5. `aliases[.%string]` (search for any elements containing \"string\")\n6. `aliases[.$value]` (search for any elements ending with \"value\")\n7. `aliases[.=~/^(\\b[Ss][a-z]+\\s){2}[a-z]+$/]` (search for any elements matching\n a complex Regular Expression, which happens to match the example)\n8. `/aliases[0]` (same as 1 but in forward-slash notation)\n9. `/aliases/0` (same as 2 but in forward-slash notation)\n10. `/aliases[&first_anchor]` (same as 3 but in forward-slash notation)\n\n```yaml\n---\nusers:\n - name: User One\n password: ENC[PKCS7,MIIBiQY...Jk==]\n roles:\n - Writers\n - name: User Two\n password: ENC[PKCS7,MIIBiQY...vF==]\n roles:\n - Power Users\n - Editors\n```\n\nWith an example like this, YAML Path enables:\n\n* selection of single nodes: `/users/0/roles/0` = `Writers`\n* all children nodes of any given parent: `/users/1/roles` =\n `[\"Power Users\", \"Editors\"]`\n* searching by a child attribute: `/users[name=\"User One\"]/password` =\n `Some decrypted value, provided you have the appropriate EYAML keys`\n* pass-through selections against arrays-of-hashes: `/users/roles` =\n `[\"Writers\"]\\n[\"Power Users\", \"Editors\"]` (each user's list of roles are a\n seperate result)\n* collection of disparate results: `(/users/name)` =\n `[\"User One\", \"User Two\"]` (all names appear in a single result instead of\n one per line)\n\nFor a deeper exploration of YAML Path's capabilities, please visit the\n[project Wiki](https://github.com/wwkimball/yamlpath/wiki).\n\n## Supported YAML Path Segments\n\nA YAML Path *segment* is the text between seperators which identifies a parent\nor leaf node within the data structure. For dot-notation, a path like\n`hash.key` identifies two segments: `hash` (a parent node) and `key` (a leaf\nnode). The same path in forward-slash notation would be: `/hash/key`.\n\nYAML Path understands these segment types:\n\n* Top-level Hash key selection: `key`\n* Explicit top-level array element selection: `[#]` where `#` is the zero-based\n element number; `#` can also be negative, causing the element to be selected\n from the end of the Array\n* Implicit array element selection **or** numbered hash key selection: `#`\n where `#` is the 0-based element number **or** exact name of a hash key which\n is itself a number\n* Top-level (Hash) Anchor lookups: `&anchor_name` (the `&` is required to\n indicate you are seeking an Anchor by name)\n* Hash sub-keys: `hash.child.key` or `/hash/child/key`\n* Demarcation for dotted Hash keys: `hash.'dotted.child.key'` or\n `hash.\"dotted.child.key\"` (not necessary when using forward-slash notation,\n `/hash/dotted.child.key`)\n* Named Array element selection: `array[#]`, `array.#`, `/array[#]`, or\n `/array/#` where `array` is the name of the Hash key containing Array data\n and `#` is the 0-based element number\n* Anchor lookups in named Arrays: `array[&anchor_name]` where `array` is the\n name of the Hash key containing Array data and both of the `[]` pair and `&`\n are required to indicate you are seeking an Anchor by name within an Array\n* Array slicing: `array[start#:stop#]` where `start#` is the first inclusive,\n zero-based element and `stop#` is the last exclusive element to select;\n either or both can be negative, causing the elements to be selected from the\n end of the Array; when `start#` and `stop#` are identical, it is the same as\n `array[start#]`\n* Hash slicing: `hash[min:max]` where `min` and `max` are alphanumeric terms\n between which the Hash's keys are compared\n* Escape symbol recognition: `hash.dotted\\.child\\.key`,\n `/hash/whacked\\/child\\/key`, and `keys_with_\\\\slashes`\n* Hash attribute searches (which can return zero or more matches):\n * Exact match: `hash[name=admin]`\n * Starts With match: `hash[name^adm]`\n * Ends With match: `hash[name$min]`\n * Contains match: `hash[name%dmi]`\n * Less Than match: `hash[access_level<500]`\n * Greater Than match: `hash[access_level>0]`\n * Less Than or Equal match: `hash[access_level<=100]`\n * Greater Than or Equal match: `hash[access_level>=0]`\n * Regular Expression matches: `hash[access_level=~/^\\D+$/]` (the `/` Regular\n Expression delimiter can be substituted for any character you need, except\n white-space; note that `/` does not interfere with forward-slash notation\n *and it does not need to be escaped* because the entire search expression is\n contained within a `[]` pair)\n * Invert any match with `!`, like: `hash[name!=admin]` or even\n `hash[!name=admin]` (the former syntax is used when YAML Paths are\n stringified but both forms are equivalent)\n * Demarcate and/or escape expression operands, like:\n `hash[full\\ name=\"Some User\\'s Name\"]` (note that embedded, single `'` and\n `\"` must be escaped lest they be deemed unmatched demarcation pairings)\n * Multi-level matching: `hash[name%admin].pass[encrypted!^ENC\\[]` or\n `/hash[name%admin]/pass[encrypted!^ENC\\[]`\n* Array element searches with all of the search methods above via `.` (yields\n any matching elements): `array[.>9000]`\n* Hash key-name searches with all of the search methods above via `.` (yields\n their values, not the keys themselves): `hash[.^app_]`\n* Array-of-Hashes Pass-Through Selection: Omit a selector for the elements of\n an Array-of-Hashes and all matching Hash attributes at that level will be\n yielded (or searched when there is more to the path). For example,\n `warriors[1].power_level` or `/warriors[1]/power_level` will return the\n power_level attribute of only the second Hash in an Array-of-Hashes while\n `warriors.power_level` or `/warriors/power_level` will return the power_level\n attribute of every Hash in the same Array-of-Hashes. Of course these results\n can be filtered in multiple ways, like `warriors[power_level>9000]`,\n `/warriors[power_level>9000]`, `warriors.power_level[.>9000]`, and\n `/warriors/power_level[.>9000]` all yield only the power_level from *all*\n warriors with power_levels over 9,000 within the same array of warrior hashes.\n* Collectors: Placing any portion of the YAML Path within parenthesis defines a\n virtual list collector, like `(YAML Path)`; concatenation and exclusion\n operators are supported -- `+` and `-`, respectively -- along with nesting,\n like `(...)-((...)+(...))`\n* Complex combinations:\n `some::deep.hierarchy[with!=\"\"].'any.valid'[.=~/(yaml|json)/][data%structure].or.complexity[4].2`\n or `/some::deep/hierarchy[with!=\"\"]/any.valid[.=~/(yaml|json)/][data%structure]/or/complexity[4]/2`\n\nThis implementation of YAML Path encourages creativity. Use whichever notation\nand segment types that make the most sense to you in each application.\n\n## Installing\n\nThis project requires [Python](https://www.python.org/) 3.6. Most operating\nsystems and distributions have access to Python 3 even if only Python 2 -- or\nno Python, at all -- came pre-installed. It is generally safe to have more\nthan one version of Python on your system at the same time, especially when\nusing\n[virtual Python environments](https://docs.python.org/3/library/venv.html).\n\nEach published version of this project can be installed from\n[PyPI](https://pypi.org/) using `pip`. Note that on systems with more than one\nversion of Python, you will probably need to use `pip3`, or equivalent (e.g.:\nCygwin users may need to use `pip3.6`).\n\n```shell\npip3 install yamlpath\n```\n\nEYAML support is entirely optional. You do not need EYAML to use YAML Path.\nThat YAML Path supports EYAML is a service to a substantial audience: Puppet\nusers. At the time of this writing, EYAML (classified as a Hiera\nback-end/plug-in) is available only as a Ruby Gem. That said, it provides a\ncommand-line tool, `eyaml`, which can be employed by this otherwise Python\nproject. To enjoy EYAML support, install compatible versions of ruby and\nrubygems, then execute:\n\n```shell\ngem install hiera-eyaml\n```\n\nIf this puts the `eyaml` command on your system `PATH`, nothing more need be\ndone apart from generating or obtaining your encryption keys. Otherwise, you\ncan tell YAML Path library and tools where to find the `eyaml` command.\n\n## Based on ruamel.yaml and Python 3\n\nIn order to support the best available YAML editing capability (so called,\nround-trip editing with support for comment preservation), this project is based\non [ruamel.yaml](https://bitbucket.org/ruamel/yaml/overview) for\nPython 3.6. While ruamel.yaml is based on PyYAML --\nPython's \"standard\" YAML library -- ruamel.yaml is [objectively better than\nPyYAML](https://yaml.readthedocs.io/en/latest/pyyaml.html), which lacks critical\nround-trip editing capabilities as well as up-to-date YAML/Compatible data\nparsing capabilities (at the time of this writing).\n\nShould PyYAML ever merge with -- or at least, catch up with -- ruamel.yaml, this\nproject can be (lightly) adapted to depend on it, instead. These conversations\nmay offer some insight into when or whether this might happen:\n\n* [Is this time to pass the baton?](https://github.com/yaml/pyyaml/issues/31)\n* [Rebase off ruamel? - many new valuable features](https://github.com/yaml/pyyaml/issues/46)\n\n## The Files of This Project\n\nThis repository contains:\n\n1. Generally-useful Python library files. These contain the reusable core of\n this project's YAML Path capabilities.\n2. Some implementations of those libraries, exhibiting their capabilities and\n simple-to-use APIs as command-line tools.\n3. Various support, documentation, and build files.\n\n### Command-Line Tools\n\nThis project provides some command-line tool implementations which utilize YAML\nPath. For some use-case examples of these tools,\n[see below](#basic-usage--command-line-tools).\n\nThe supplied command-line tools include:\n\n* [eyaml-rotate-keys](yamlpath/commands/eyaml_rotate_keys.py)\n\n```text\nusage: eyaml-rotate-keys [-h] [-V] [-d | -v | -q] [-b] [-x EYAML]\n -i OLDPRIVATEKEY -c OLDPUBLICKEY\n -r NEWPRIVATEKEY -u NEWPUBLICKEY\n YAML_FILE [YAML_FILE ...]\n\nRotates the encryption keys used for all EYAML values within a set of YAML\nfiles, decrypting with old keys and re-encrypting using replacement keys.\n\npositional arguments:\n YAML_FILE one or more YAML files containing EYAML values\n\noptional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n -d, --debug output debugging details\n -v, --verbose increase output verbosity\n -q, --quiet suppress all output except errors\n -b, --backup save a backup of each modified YAML_FILE with an extra\n .bak file-extension\n -x EYAML, --eyaml EYAML\n the eyaml binary to use when it isn't on the PATH\n\nEYAML_KEYS:\n All key arguments are required\n\n -r NEWPRIVATEKEY, --newprivatekey NEWPRIVATEKEY\n the new EYAML private key\n -u NEWPUBLICKEY, --newpublickey NEWPUBLICKEY\n the new EYAML public key\n -i OLDPRIVATEKEY, --oldprivatekey OLDPRIVATEKEY\n the old EYAML private key\n -c OLDPUBLICKEY, --oldpublickey OLDPUBLICKEY\n the old EYAML public key\n\nAny YAML_FILEs lacking EYAML values will not be modified (or backed up, even\nwhen -b/--backup is specified).\n```\n\n* [yaml-get](yamlpath/commands/yaml_get.py)\n\n```text\nusage: yaml-get [-h] [-V] -p YAML_PATH\n [-t ['.', '/', 'auto', 'dot', 'fslash']] [-x EYAML]\n [-r PRIVATEKEY] [-u PUBLICKEY] [-d | -v | -q]\n YAML_FILE\n\nRetrieves one or more values from a YAML file at a specified YAML Path. Output\nis printed to STDOUT, one line per result. When a result is a complex data-\ntype (Array or Hash), a JSON dump is produced to represent it. EYAML can be\nemployed to decrypt the values.\n\npositional arguments:\n YAML_FILE the YAML file to query\n\noptional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n -t ['.', '/', 'auto', 'dot', 'fslash'], --pathsep ['.', '/', 'auto', 'dot', 'fslash']\n indicate which YAML Path seperator to use when\n rendering results; default=dot\n -d, --debug output debugging details\n -v, --verbose increase output verbosity\n -q, --quiet suppress all output except errors\n\nrequired settings:\n -p YAML_PATH, --query YAML_PATH\n YAML Path to query\n\nEYAML options:\n Left unset, the EYAML keys will default to your system or user defaults.\n Both keys must be set either here or in your system or user EYAML\n configuration file when using EYAML.\n\n -x EYAML, --eyaml EYAML\n the eyaml binary to use when it isn't on the PATH\n -r PRIVATEKEY, --privatekey PRIVATEKEY\n EYAML private key\n -u PUBLICKEY, --publickey PUBLICKEY\n EYAML public key\n\nFor more information about YAML Paths, please visit\nhttps://github.com/wwkimball/yamlpath.\n```\n\n* [yaml-paths](yamlpath/commands/yaml_paths.py)\n\n```text\nusage: yaml-paths [-h] [-V] -s EXPRESSION [-c EXPRESSION] [-d | -v | -q] [-p]\n [-m] [-t ['.', '/', 'auto', 'dot', 'fslash']] [-i | -k | -K]\n [-a] [-A | -Y | -y | -l] [-e] [-x EYAML] [-r PRIVATEKEY]\n [-u PUBLICKEY]\n YAML_FILE [YAML_FILE ...]\n\nReturns zero or more YAML Paths indicating where in given YAML/Compatible data\none or more search expressions match. Values, keys, and/or anchors can be\nsearched. EYAML can be employed to search encrypted values.\n\npositional arguments:\n YAML_FILE one or more YAML files to search\n\noptional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n -c EXPRESSION, --except EXPRESSION\n except results matching this search expression; can be\n set more than once\n -d, --debug output debugging details\n -v, --verbose increase output verbosity\n -q, --quiet suppress all non-result output except errors\n -p, --pathonly print results without any search expression decorators\n -m, --expand expand matching parent nodes to list all permissible\n child leaf nodes (see \"Reference handling options\" for\n restrictions)\n -t ['.', '/', 'auto', 'dot', 'fslash'], --pathsep ['.', '/', 'auto', 'dot', 'fslash']\n indicate which YAML Path seperator to use when\n rendering results; default=dot\n -a, --refnames also search the names of &anchor and *alias references\n\nrequired settings:\n -s EXPRESSION, --search EXPRESSION\n the search expression; can be set more than once\n\nKey name searching options:\n -i, --ignorekeynames (default) do not search key names\n -k, --keynames search key names in addition to values and array\n elements\n -K, --onlykeynames only search key names (ignore all values and array\n elements)\n\nReference handling options:\n Indicate how to treat anchor and alias references. An anchor is an\n original, reusable key or value. All aliases become replaced by the\n anchors they reference when YAML data is read. These options specify how\n to handle this duplication of keys and values. Note that the default\n behavior includes all aliased keys but not aliased values.\n\n -A, --anchorsonly include only original matching key and value anchors\n in results, discarding all aliased keys and values\n (including child nodes)\n -Y, --allowkeyaliases\n (default) include matching key aliases, permitting\n search traversal into their child nodes\n -y, --allowvaluealiases\n include matching value aliases (does not permit search\n traversal into aliased keys)\n -l, --allowaliases include all matching key and value aliases\n\nEYAML options:\n Left unset, the EYAML keys will default to your system or user defaults.\n Both keys must be set either here or in your system or user EYAML\n configuration file when using EYAML.\n\n -e, --decrypt decrypt EYAML values in order to search them\n (otherwise, search the encrypted blob)\n -x EYAML, --eyaml EYAML\n the eyaml binary to use when it isn't on the PATH\n -r PRIVATEKEY, --privatekey PRIVATEKEY\n EYAML private key\n -u PUBLICKEY, --publickey PUBLICKEY\n EYAML public key\n\nA search or exception EXPRESSION takes the form of a YAML Path search operator\n-- %, $, =, ^, >, <, >=, <=, =~, or ! -- followed by the search term, omitting\nthe left-hand operand. For more information about YAML Paths, please visit\nhttps://github.com/wwkimball/yamlpath.\n```\n\n* [yaml-set](yamlpath/commands/yaml_set.py)\n\n```text\nusage: yaml-set [-h] [-V] -g YAML_PATH [-a VALUE | -f FILE | -i | -R LENGTH]\n [-F {bare,boolean,default,dquote,float,folded,int,literal,squote}]\n [-c CHECK] [-s YAML_PATH] [-m] [-b]\n [-t ['.', '/', 'auto', 'dot', 'fslash']] [-e] [-x EYAML]\n [-r PRIVATEKEY] [-u PUBLICKEY] [-d | -v | -q]\n YAML_FILE\n\nChanges one or more values in a YAML file at a specified YAML Path. Matched\nvalues can be checked before they are replaced to mitigate accidental change.\nWhen matching singular results, the value can be archived to another key\nbefore it is replaced. Further, EYAML can be employed to encrypt the new\nvalues and/or decrypt an old value before checking them.\n\npositional arguments:\n YAML_FILE the YAML file to update\n\noptional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n -F {bare,boolean,default,dquote,float,folded,int,literal,squote}, --format {bare,boolean,default,dquote,float,folded,int,literal,squote}\n override automatic formatting of the new value\n -c CHECK, --check CHECK\n check the value before replacing it\n -s YAML_PATH, --saveto YAML_PATH\n save the old value to YAML_PATH before replacing it;\n implies --mustexist\n -m, --mustexist require that the --change YAML_PATH already exist in\n YAML_FILE\n -b, --backup save a backup YAML_FILE with an extra .bak file-\n extension\n -t ['.', '/', 'auto', 'dot', 'fslash'], --pathsep ['.', '/', 'auto', 'dot', 'fslash']\n indicate which YAML Path seperator to use when\n rendering results; default=dot\n -d, --debug output debugging details\n -v, --verbose increase output verbosity\n -q, --quiet suppress all output except errors\n\nrequired settings:\n -g YAML_PATH, --change YAML_PATH\n YAML Path where the target value is found\n\ninput options:\n -a VALUE, --value VALUE\n set the new value from the command-line instead of\n STDIN\n -f FILE, --file FILE read the new value from file (discarding any trailing\n new-lines)\n -i, --stdin accept the new value from STDIN (best for sensitive\n data)\n -R LENGTH, --random LENGTH\n randomly generate a replacement value of a set length\n\nEYAML options:\n Left unset, the EYAML keys will default to your system or user defaults.\n You do not need to supply a private key unless you enable --check and the\n old value is encrypted.\n\n -e, --eyamlcrypt encrypt the new value using EYAML\n -x EYAML, --eyaml EYAML\n the eyaml binary to use when it isn't on the PATH\n -r PRIVATEKEY, --privatekey PRIVATEKEY\n EYAML private key\n -u PUBLICKEY, --publickey PUBLICKEY\n EYAML public key\n\nWhen no changes are made, no backup is created, even when -b/--backup is\nspecified. For more information about YAML Paths, please visit\nhttps://github.com/wwkimball/yamlpath.\n```\n\n### Libraries\n\nWhile there are several supporting library files like enumerations, types, and\nexceptions, the most interesting library files include:\n\n* [yamlpath.py](yamlpath/yamlpath.py) -- The core YAML Path parser logic.\n* [processor.py](yamlpath/processor.py) -- Processes YAMLPath instances to read\n or write data to YAML/Compatible sources.\n* [eyamlprocessor.py](yamlpath/eyaml/eyamlprocessor.py) -- Extends the\n Processor class to support EYAML data encryption and decryption.\n\n## Basic Usage\n\nThe files of this project can be used either as command-line tools or as\nlibraries to supplement your own work.\n\n### Basic Usage: Command-Line Tools\n\nThe command-line tools are self-documented and [their documentation is captured\nabove](#command-line-tools) for easy reference. Simply pass `--help` to them in\norder to obtain the same detailed documentation.\n\nPlease review [the comprehensive test_commands_*.py unit tests](/tests/) to\nexplore samples of YAML files and the many ways these tools help get and set\ntheir data.\n\nThe following are some simple examples of their typical use-cases.\n\n#### Rotate Your EYAML Keys\n\nIf the eyaml command is already on your PATH (if not, be sure to also supply\nthe optional `--eyaml` or `-x` argument):\n\n```shell\neyaml-rotate-keys \\\n --oldprivatekey=~/old-keys/private_key.pkcs7.pem \\\n --oldpublickey=~/old-keys/public_key.pkcs7.pem \\\n --newprivatekey=~/new-keys/private_key.pkcs7.pem \\\n --newpublickey=~/new-keys/public_key.pkcs7.pem \\\n my_1st_yaml_file.yaml my_2nd_yaml_file.eyaml ... my_Nth_yaml_file.yaml\n```\n\nYou could combine this with `find` and `xargs` if your E/YAML file are\ndispersed through a directory hierarchy, as with Hiera data.\n\n#### Get a YAML Value\n\nAt its simplest:\n\n```shell\nyaml-get \\\n --query=see.documentation.above.for.many.samples \\\n my_yaml_file.yaml\n```\n\n#### Search For YAML Paths\n\nSimplest use:\n\n```shell\nyaml-paths \\\n --search=%word \\\n /some/directory/*.yaml\n```\n\nSearch for multiple expressions and exclude unwanted results:\n\n```shell\nyaml-paths \\\n --search=^another \\\n --search=$word \\\n --except=%bad \\\n /some/directory/*.yaml\n```\n\nReturn all leaf nodes under matching parents (most useful when matching against Hash keys and you only want the original leaf nodes beneath them):\n\n```shell\nyaml-paths \\\n --expand \\\n --keynames \\\n --search==parent_node \\\n /some/directory/*.yaml\n```\n\n#### Change a YAML Value\n\nFor a no-frills change to a YAML file with deeply nested Hash structures:\n\n```shell\nyaml-set \\\n --change=see.documentation.above.for.many.samples \\\n --value=\"New Value\" \\\n my_yaml_file.yaml\n```\n\nTo rotate a password, preserving the old password perhaps so your automation can\napply the new password to your application(s):\n\n```shell\nyaml-set \\\n --mustexist \\\n --change=the.new.password \\\n --saveto=the.old.password \\\n --value=\"New Password\" \\\n my_yaml_file.yaml\n```\n\nFor the extremely cautious, you could check the old password before rotating\nit and save a backup of the original file:\n\n```shell\nyaml-set \\\n --mustexist \\\n --change=the.new.password \\\n --saveto=the.old.password \\\n --check=\"Old Password\" \\\n --value=\"New Password\" \\\n --backup \\\n my_yaml_file.yaml\n```\n\nYou can also add EYAML encryption (assuming the `eyaml` command is on your\nPATH; if not, you can pass `--eyaml` to specify its location). In this example,\nI add the optional `--format=folded` so that the long EYAML value is broken up\ninto a multi-line value rather than one very long string. This is the preferred\nformat for human legibility as well as EYAML consumers like\n[Puppet](http://puppet.com). Note that `--format` has several other settings\nand applies only to new values.\n\n```shell\nyaml-set \\\n --change=the.new.password \\\n --mustexist \\\n --saveto=the.old.password \\\n --check=\"Old Password\" \\\n --value=\"New Password\" \\\n --eyamlcrypt \\\n --format=folded \\\n --backup \\\n my_yaml_file.yaml\n```\n\nYou can even tell EYAML which keys to use, if not your default system or user\nkeys:\n\n```shell\nyaml-set \\\n --change=the.new.password \\\n --mustexist \\\n --saveto=the.old.password \\\n --check=\"Old Password\" \\\n --value=\"New Password\" \\\n --eyamlcrypt \\\n --format=folded \\\n --privatekey=/secret/keys/private_key.pkcs7.pem \\\n --publickey=/secret/keys/public_key.pkcs7.pem \\\n --backup \\\n my_yaml_file.yaml\n```\n\nNote that for even greater security scenarios, you can keep the new value off of\nyour command-line, process list, and command history by swapping out `--value`\nfor one of `--stdin`, `--file`, or even `--random LENGTH` (use Python's\nstrongest random value generator if you don't need to specify the replacement\nvalue in advance).\n\n### Basic Usage: Libraries\n\nAs for the libraries, they are also heavily documented and the example\nimplementations may perhaps serve as good copy-paste fodder (provided you give\ncredit to the source). That said, here's a general flow/synopsis.\n\n#### Initialize ruamel.yaml and These Helpers\n\nYour preferences may differ, but I use this setup for round-trip YAML parsing\nand editing with ruamel.yaml. When you need to process EYAML encrypted data,\nreplace `yamlpath.Processor` with `yamlpath.eyaml.EYAMLProcessor` and add error\nhandling for `yamlpath.eyaml.EYAMLCommandException`.\n\nNote that `import yamlpath.patches` is entirely optional. I wrote and use it to\nblock ruamel.yaml's Emitter from injecting unnecessary newlines into folded\nvalues (it improperly converts every single new-line into two for left-flushed\nmulti-line values, at the time of this writing). Since \"block\" output EYAML\nvalues are left-flushed multi-line folded strings, this fix is necessary when\nusing EYAML features. At least, until ruamel.yaml has its own fix for this\nissue.\n\nNote also that these examples use `ConsolePrinter` to handle STDOUT and STDERR\nmessaging. You don't have to. However, some kind of logger must be passed to\nthese libraries so they can write messages _somewhere_. Your custom message\nhandler or logger must provide the same API as `ConsolePrinter`; review the\nheader documentation in [consoleprinter.py](yamlpath/wrappers/consoleprinter.py)\nfor details. Generally speaking, it would be trivial to write your own custom\nwrapper for Python's standard logging facilities if you require targets other\nthan STDOUT and STDERR.\n\n```python\nimport sys\n\nfrom ruamel.yaml import YAML\nfrom ruamel.yaml.parser import ParserError\n\nimport yamlpath.patches\nfrom yamlpath.func import get_yaml_data, get_yaml_editor\nfrom yamlpath.wrappers import ConsolePrinter\nfrom yamlpath import Processor\n\n# Process command-line arguments and initialize the output writer\nargs = processcli()\nlog = ConsolePrinter(args)\n\n# Prep the YAML parser and round-trip editor (tweak to your needs)\nyaml = get_yaml_editor()\n\n# At this point, you'd load or parse your YAML file, stream, or string. When\n# loading from file, I typically follow this pattern:\nyaml_data = get_yaml_data(yaml, log, yaml_file)\nif yaml_data is None:\n # There was an issue loading the file; an error message has already been\n # printed.\n exit(1)\n\n# Pass the log writer and parsed YAML data to the YAMLPath processor\nprocessor = Processor(log, yaml_data)\n\n# At this point, the processor is ready to handle YAMLPaths\n```\n\n#### Searching for YAML Nodes\n\nThese libraries use [Generators](https://wiki.python.org/moin/Generators) to get\nnodes from parsed YAML data. Identify which node(s) to get via YAML Path\nstrings. You should also catch `yamlpath.exceptions.YAMLPathException`s\nunless you prefer Python's native stack traces. When using EYAML, you should\nalso catch `yamlpath.eyaml.exceptions.EYAMLCommandException`s for the same\nreason. Whether you are working with a single result or many, you should\nconsume the Generator output with a pattern similar to:\n\n```python\nfrom yamlpath import YAMLPath\nfrom yamlpath.exceptions import YAMLPathException\n\nyaml_path = YAMLPath(\"see.documentation.above.for.many.samples\")\ntry:\n for node in processor.get_nodes(yaml_path):\n log.debug(\"Got {} from '{}'.\".format(node, yaml_path))\n # Do something with each node...\nexcept YAMLPathException as ex:\n # If merely retrieving data, this exception may be deemed non-critical\n # unless your later code absolutely depends upon a result.\n log.error(ex)\n```\n\n#### Changing Values\n\nAt its simplest, you only need to supply the the YAML Path to one or more nodes\nto update, and the value to apply to them. Catching\n`yamlpath.exceptions.YAMLPathException` is optional but usually preferred over\nallowing Python to dump the call stack in front of your users. When using\nEYAML, the same applies to `yamlpath.eyaml.exceptions.EYAMLCommandException`.\n\n```python\nfrom yamlpath.exceptions import YAMLPathException\n\ntry:\n processor.set_value(yaml_path, new_value)\nexcept YAMLPathException as ex:\n log.critical(ex, 119)\nexcept EYAMLCommandException as ex:\n log.critical(ex, 120)\n```", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/wwkimball/yamlpath", "keywords": "yaml eyaml yaml-path", "license": "ISC", "maintainer": "", "maintainer_email": "", "name": "yamlpath", "package_url": "https://pypi.org/project/yamlpath/", "platform": "", "project_url": "https://pypi.org/project/yamlpath/", "project_urls": { "Homepage": "https://github.com/wwkimball/yamlpath" }, "release_url": "https://pypi.org/project/yamlpath/2.3.0/", "requires_dist": null, "requires_python": ">3.6.0", "summary": "Read and change YAML/Compatible data using powerful, intuitive, command-line friendly syntax", "version": "2.3.0" }, "last_serial": 5867625, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "957c61b7fe468b446a828e2b68145e70", "sha256": "2abd622f403eb27d6780109857e6b2427025227217afffd1c9463435abd53a3e" }, "downloads": -1, "filename": "yamlpath-1.0.0.tar.gz", "has_sig": false, "md5_digest": "957c61b7fe468b446a828e2b68145e70", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33823, "upload_time": "2019-05-08T03:38:09", "url": "https://files.pythonhosted.org/packages/c6/11/c91e5514c8530cc307b4d381c92c7b0b89f70aa2285beec4a940bb17347a/yamlpath-1.0.0.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "d81b290f14e302732ec85683c741df33", "sha256": "f5d9370da28ba7f562a191b6b7dbfe86ba1aa4720e74392ed2f168d14d398c73" }, "downloads": -1, "filename": "yamlpath-1.1.0.tar.gz", "has_sig": false, "md5_digest": "d81b290f14e302732ec85683c741df33", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34748, "upload_time": "2019-05-09T22:35:11", "url": "https://files.pythonhosted.org/packages/e0/6b/7d56e78ec0fd12f957f0789f93d1c6df1afdaaface8d29cac4757291a728/yamlpath-1.1.0.tar.gz" } ], "1.1.1": [ { "comment_text": "", "digests": { "md5": "f43a268434bb970c9377dd03b1192107", "sha256": "c9ebb31a717c9ae67d825338f76356a1bde3e8484af208a205e751e40f3d6df7" }, "downloads": -1, "filename": "yamlpath-1.1.1.tar.gz", "has_sig": false, "md5_digest": "f43a268434bb970c9377dd03b1192107", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42219, "upload_time": "2019-05-10T06:29:08", "url": "https://files.pythonhosted.org/packages/75/f5/281ab60c1dd7a183bd10c5b45306894919042afdb5d8ee054a23f2bf19d2/yamlpath-1.1.1.tar.gz" } ], "1.1.2": [ { "comment_text": "", "digests": { "md5": "a4fc1c7162f318d2818521e6c3fa38dd", "sha256": "b6bc27ff17905c8f3cb3d7819eeead63537d4a052b133e43c3cd05ef4bfa9400" }, "downloads": -1, "filename": "yamlpath-1.1.2.tar.gz", "has_sig": false, "md5_digest": "a4fc1c7162f318d2818521e6c3fa38dd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42530, "upload_time": "2019-05-10T23:53:31", "url": "https://files.pythonhosted.org/packages/8e/5e/71cb98565ba564a908ced722232e6f5efde89fa65a178c68000864f466b6/yamlpath-1.1.2.tar.gz" } ], "1.2.0": [ { "comment_text": "", "digests": { "md5": "c35b5f38d69901df874e9f2a2121f42f", "sha256": "1e0d44f803fb190f897370e7889003268159fa3ff8db07fb1c2e9b7554c0c30b" }, "downloads": -1, "filename": "yamlpath-1.2.0.tar.gz", "has_sig": false, "md5_digest": "c35b5f38d69901df874e9f2a2121f42f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 46429, "upload_time": "2019-05-13T05:51:11", "url": "https://files.pythonhosted.org/packages/71/c0/e7831c500c4577d58e31723087d64ecc0cacd811bc2224344bc273ede2b3/yamlpath-1.2.0.tar.gz" } ], "1.2.1": [ { "comment_text": "", "digests": { "md5": "de69356ecd3ca67457383b58f67de568", "sha256": "f7d5461968058aa7c8ab6898b2fd6f8129dcd02a09af1d77d730a9e711ad0242" }, "downloads": -1, "filename": "yamlpath-1.2.1.tar.gz", "has_sig": false, "md5_digest": "de69356ecd3ca67457383b58f67de568", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 47009, "upload_time": "2019-05-13T20:57:30", "url": "https://files.pythonhosted.org/packages/eb/af/6bcc31135726d6d4a39ee7210d5b25589e79708705c3db59ed49f69231ed/yamlpath-1.2.1.tar.gz" } ], "1.2.2": [ { "comment_text": "", "digests": { "md5": "9bc9b54c491189a6e050d572d5c9b23c", "sha256": "1419b08e4280740625b8ca3dc5fecd870f3ea9322a95e263c6547707e37e23f0" }, "downloads": -1, "filename": "yamlpath-1.2.2.tar.gz", "has_sig": false, "md5_digest": "9bc9b54c491189a6e050d572d5c9b23c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 49721, "upload_time": "2019-05-14T20:17:33", "url": "https://files.pythonhosted.org/packages/5a/c6/a68296d7c1253958db892035a0c1817d407ec64130e9e5deaadeb5bf378f/yamlpath-1.2.2.tar.gz" } ], "1.2.3": [ { "comment_text": "", "digests": { "md5": "85a257eaf4836e334b50e4b9e6db6878", "sha256": "ddf009f55c565c40b15a6c60fab6a38491a47a434fb92afe705edf260577db6d" }, "downloads": -1, "filename": "yamlpath-1.2.3.tar.gz", "has_sig": false, "md5_digest": "85a257eaf4836e334b50e4b9e6db6878", "packagetype": "sdist", "python_version": "source", "requires_python": ">3.6.0", "size": 49744, "upload_time": "2019-05-16T04:17:13", "url": "https://files.pythonhosted.org/packages/e5/ed/1255e8ac378f007bd9a1e51fe0cff3aa9df5f758850756b83fba9f2e3878/yamlpath-1.2.3.tar.gz" } ], "1.2.4": [ { "comment_text": "", "digests": { "md5": "872b4d46bf321aed8de3934ec6356931", "sha256": "847818273945be81ab17222beb4826709c8c13f5b95c5424dfc4f293702b21ea" }, "downloads": -1, "filename": "yamlpath-1.2.4.tar.gz", "has_sig": false, "md5_digest": "872b4d46bf321aed8de3934ec6356931", "packagetype": "sdist", "python_version": "source", "requires_python": ">3.6.0", "size": 50429, "upload_time": "2019-05-17T23:06:01", "url": "https://files.pythonhosted.org/packages/d1/f3/249efd648a08ad286575893fd3aadf4027cd16dafe49c6be90d6467fa795/yamlpath-1.2.4.tar.gz" } ], "1.2.5": [ { "comment_text": "", "digests": { "md5": "74f81572216afa21381ad83d772f0738", "sha256": "1b82a0efd6fc3ea634a7179fedeb2490ad2a3b22122e5760f95b1f1f27b39746" }, "downloads": -1, "filename": "yamlpath-1.2.5.tar.gz", "has_sig": false, "md5_digest": "74f81572216afa21381ad83d772f0738", "packagetype": "sdist", "python_version": "source", "requires_python": ">3.6.0", "size": 50548, "upload_time": "2019-05-23T19:04:58", "url": "https://files.pythonhosted.org/packages/0a/61/05ca33b6f29a3e878e7a4c366f7aee51e4df842278946519df2e5bd7d36f/yamlpath-1.2.5.tar.gz" } ], "2.0.0": [ { "comment_text": "", "digests": { "md5": "0df0a7b2bf131b0e10cf6e5506f5c0bf", "sha256": "7bf0effc671ce9490ef66c79585a0c93c737599022315855a4e79490be06aaf9" }, "downloads": -1, "filename": "yamlpath-2.0.0.tar.gz", "has_sig": false, "md5_digest": "0df0a7b2bf131b0e10cf6e5506f5c0bf", "packagetype": "sdist", "python_version": "source", "requires_python": ">3.6.0", "size": 56545, "upload_time": "2019-05-27T19:31:10", "url": "https://files.pythonhosted.org/packages/4f/44/61842c8f788c12c416906141fc6e8c57e87b13f284c4559b5f767604730f/yamlpath-2.0.0.tar.gz" } ], "2.0.1": [ { "comment_text": "", "digests": { "md5": "c9a387ac0bfdb5d5de08bbbf30c01a72", "sha256": "dd46fb3f330d62e70b7d6e4750bd7b9db5920601c2949fbed0727c9177330380" }, "downloads": -1, "filename": "yamlpath-2.0.1.tar.gz", "has_sig": false, "md5_digest": "c9a387ac0bfdb5d5de08bbbf30c01a72", "packagetype": "sdist", "python_version": "source", "requires_python": ">3.6.0", "size": 56662, "upload_time": "2019-05-29T15:42:48", "url": "https://files.pythonhosted.org/packages/d5/93/88baebde6b36d002f039f757ead59668d75c5258ef5f2a1568c312371eea/yamlpath-2.0.1.tar.gz" } ], "2.0.2": [ { "comment_text": "", "digests": { "md5": "2ed2957ac17885f243ceb19f508fe605", "sha256": "8088d58995574a83844a1787cb83f11d4472a6ab5f7e3b1032bc30303e72c3a3" }, "downloads": -1, "filename": "yamlpath-2.0.2.tar.gz", "has_sig": false, "md5_digest": "2ed2957ac17885f243ceb19f508fe605", "packagetype": "sdist", "python_version": "source", "requires_python": ">3.6.0", "size": 57368, "upload_time": "2019-05-30T21:13:17", "url": "https://files.pythonhosted.org/packages/a8/b5/e9bef7eb3b37d541d24853dd8b163e7bf5c556242e2e4dccb09894c65419/yamlpath-2.0.2.tar.gz" } ], "2.1.0": [ { "comment_text": "", "digests": { "md5": "1714f52c8c59b0fe1961408fc5fc4266", "sha256": "b3a323dd11ff208c31fd2872ef070caa8c4b332a1512ed40c614598f46acd237" }, "downloads": -1, "filename": "yamlpath-2.1.0.tar.gz", "has_sig": false, "md5_digest": "1714f52c8c59b0fe1961408fc5fc4266", "packagetype": "sdist", "python_version": "source", "requires_python": ">3.6.0", "size": 63281, "upload_time": "2019-06-06T04:16:34", "url": "https://files.pythonhosted.org/packages/dd/77/b6764edc579117c9a6f82490882ce7db0009767745500469d69ef40abe06/yamlpath-2.1.0.tar.gz" } ], "2.1.1": [ { "comment_text": "", "digests": { "md5": "ba9b9a1877235492132de3e57b888e1f", "sha256": "5766ce79aa34193c1ba825f05d4153b1aac738f98d4863560f542737e109fb77" }, "downloads": -1, "filename": "yamlpath-2.1.1.tar.gz", "has_sig": false, "md5_digest": "ba9b9a1877235492132de3e57b888e1f", "packagetype": "sdist", "python_version": "source", "requires_python": ">3.6.0", "size": 65429, "upload_time": "2019-06-07T02:19:12", "url": "https://files.pythonhosted.org/packages/ee/59/3c72da21685a3d4850c9a48b2d2736e7b23d5ce198be2206f742ac098b0d/yamlpath-2.1.1.tar.gz" } ], "2.2.0": [ { "comment_text": "", "digests": { "md5": "ed2142d6293feb1fa0da19aaa4146c96", "sha256": "d95d1d1c33f8c3f69932bc8b604c50dd61fde21cc14e58a00cb31b30a1fbb031" }, "downloads": -1, "filename": "yamlpath-2.2.0.tar.gz", "has_sig": false, "md5_digest": "ed2142d6293feb1fa0da19aaa4146c96", "packagetype": "sdist", "python_version": "source", "requires_python": ">3.6.0", "size": 68182, "upload_time": "2019-06-09T23:32:46", "url": "https://files.pythonhosted.org/packages/be/39/9886ae1accfb506f5bde4e5e6bbd1d41678e446b7c2e006531d9cf7e14ec/yamlpath-2.2.0.tar.gz" } ], "2.3.0": [ { "comment_text": "", "digests": { "md5": "7b1ee77a37e21096a039258a783c8bfa", "sha256": "2348975655dcd0fca1c52857ef0c76da00724db92d43505893d5c0e02c82c868" }, "downloads": -1, "filename": "yamlpath-2.3.0.tar.gz", "has_sig": false, "md5_digest": "7b1ee77a37e21096a039258a783c8bfa", "packagetype": "sdist", "python_version": "source", "requires_python": ">3.6.0", "size": 69629, "upload_time": "2019-09-22T01:54:57", "url": "https://files.pythonhosted.org/packages/8a/83/9b423563de491ba2eaa29edbc5643a2ebf328350d0f76928d2fa19492d6b/yamlpath-2.3.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "7b1ee77a37e21096a039258a783c8bfa", "sha256": "2348975655dcd0fca1c52857ef0c76da00724db92d43505893d5c0e02c82c868" }, "downloads": -1, "filename": "yamlpath-2.3.0.tar.gz", "has_sig": false, "md5_digest": "7b1ee77a37e21096a039258a783c8bfa", "packagetype": "sdist", "python_version": "source", "requires_python": ">3.6.0", "size": 69629, "upload_time": "2019-09-22T01:54:57", "url": "https://files.pythonhosted.org/packages/8a/83/9b423563de491ba2eaa29edbc5643a2ebf328350d0f76928d2fa19492d6b/yamlpath-2.3.0.tar.gz" } ] }