\n\nThis iconic [STIX2 Pattern](http://docs.oasis-open.org/cti/stix/v2.0/cs01/part5-stix-patterning/stix-v2.0-cs01-part5-stix-patterning.html#_t7hu3hrkvmff) visualization is based upon this example expression:\n```\n(\n [ipv4-addr:value = '198.51.100.1/32' OR\n ipv4-addr:value = '203.0.113.33/32' OR\n ipv6-addr:value = '2001:0db8:dead:beef:dead:beef:dead:0001/128']\n\n FOLLOWEDBY [\n domain-name:value = 'example.com']\n\n) WITHIN 600 SECONDS\n```\n\nUsing the formal STIX2 Pattern grammar, that expression is converted into this parse tree:\n\n\n\ndendrol will convert that expression (by way of that parse tree) into this more human-readable and machine-actionable form:\n```yaml\npattern:\n expression:\n join: FOLLOWEDBY\n qualifiers:\n - within:\n value: 600\n unit: SECONDS\n expressions:\n - observation:\n objects:\n ? ipv4-addr\n ? ipv6-addr\n join: OR\n qualifiers:\n expressions:\n - comparison:\n object: ipv4-addr\n path: [value]\n negated:\n operator: '='\n value: 198.51.100.1/32\n - comparison:\n object: ipv4-addr\n path: [value]\n negated:\n operator: '='\n value: 203.0.113.33/32\n - comparison:\n object: ipv6-addr\n path: [value]\n negated:\n operator: '='\n value: 2001:0db8:dead:beef:dead:beef:dead:0001/128\n - observation:\n objects: {domain-name}\n join:\n qualifiers:\n expressions:\n - comparison:\n object: domain-name\n path: [value]\n negated:\n operator: '='\n value: example.com\n```\n\n\n# How do I use it?\n\ndendrol provides an interface for parsing STIX2 Pattern Expressions much like [cti-pattern-validator](https://github.com/oasis-open/cti-pattern-validator), with the `dendrol.Pattern` class. This class has a method, `to_dict_tree()`, which converts the ANTLR parse tree to a dict-based tree structure, PatternTree.\n\n```py\nfrom dendrol import Pattern\n\npattern = Pattern(\"[domain-name:value = 'http://xyz.com/download']\")\n\nassert pattern.to_dict_tree() == {\n 'pattern': {\n 'observation': {\n 'objects': {'domain-name'},\n 'join': None,\n 'qualifiers': None,\n 'expressions': [\n {'comparison': {\n 'object': 'domain-name',\n 'path': ['value'],\n 'negated': None,\n 'operator': '=',\n 'value': 'http://xyz.com/download',\n }}\n ]\n }\n }\n}\n```\n\nA specialized YAML representation is also proposed, to make visualization of this data a little less cumbersome:\n\n```py\nfrom dendrol import Pattern\n\npattern = Pattern(\"[domain-name:value = 'http://xyz.com/download']\")\n\nassert str(pattern.to_dict_tree()) == '''\\\npattern:\n observation:\n objects: {domain-name}\n join:\n qualifiers:\n expressions:\n - comparison:\n object: domain-name\n path: [value]\n negated:\n operator: '='\n value: http://xyz.com/download\n'''\n```\n\nFor more info, read [The Spec](#the-spec) below, or check out the tests.\n\n\n# Development\nTo develop dendrol and run its tests, first clone the repo. Then install the dev and testing dependencies:\n```bash\npip install .[dev] .[test]\n```\n\npytest is used for testing:\n```bash\npy.test\n```\n\nReported issues and pull requests welcomed! From new features and suggestions to typo fixes and poor naming choices, fresh eyes bolster software eternally in development.\n\nIf submitting a pull request, please add yourself to the CONTRIBUTORS file for a piece of that sweet, sweet street cred!\n\n\n# The Spec\n## Brief\nA PatternTree begins with a `'pattern'` key. Below it is an observation expression, with an `'observation'` or `'expression'` key (which may contain more observation expressions joined by AND/OR/FOLLOWEDBY). Below `'observation'` keys are comparison expressions, marked by a `'comparison'` or `'expression'` key (which may contain more comparison expressions joined by AND/OR). `'comparison'` keys denote a single comparison between an object property and a literal value.\n\n## Pattern\n```py\n{'pattern': {...}}\n```\n```yaml\npattern:\n ...\n```\nA PatternTree is a dict with one top-level key, `'pattern'`. This paradigm of a dict with a single key identifying its contents is seen throughout this spec.\n\nThe value of this `'pattern'` key is an [observation expression](#spec-observation-expressions).\n\n\n## Observation Expressions\n\nAn Observation Expression is a dict with a single key of either `'expression'` or `'observation'`. An [`'expression'`](#spec-observation-expressions-expression) SHALL contain two or more observation expressions joined by AND/OR/FOLLOWEDBY, whereas an [`'observation'`](#spec-observation-expressions-observation) SHALL contain only comparison expressions.\n\n\n### Expression\n```py\n{'expression': {\n 'join': oneOf('AND', 'OR', 'FOLLOWEDBY'),\n 'qualifiers': [...],\n 'expressions': [...],\n}}\n```\n```yaml\nexpression:\n join: AND | OR | FOLLOWEDBY\n qualifiers:\n expressions:\n - a\n - b\n - ...\n```\nAn `'expression'` is a container for other observation expressions, joined by an observation operator in `'join'`. It MAY have a list of qualifiers in the `'qualifiers'` key, or `None` if there are none.\n\nIts children are in `'expressions'`, whose values SHALL be dicts with single keys (of either `'observation'` or `'expression'`).\n\n\n### Observation\n```py\n{'observation': {\n 'objects': {'ipv4-addr', 'ipv6-addr', ...},\n 'join': oneOf('AND', 'OR'),\n 'qualifiers': [...],\n 'expressions': [...],\n}}\n```\n```yaml\nobservation:\n objects:\n ? ipv4-addr\n ? ipv6-addr\n ? ...\n join: AND | OR\n qualifiers:\n expressions:\n - a\n - ...\n```\nAn `'observation'` is analogous to square brackets in STIX2 Pattern Expressions, e.g.: `[ipv4-addr:value = '1.2.3.4']`. Children of an observation (in the `'expressions'` key) SHALL only be comparisons or comparison expressions.\n\nAn `'observation'` MAY have qualifiers, but its children MUST NOT.\n\nAn `'observation'` MAY have a join method, which denotes how its child [comparison expressions](#spec-comparison-expressions) are to be joined. This method MAY be AND or OR, but MUST NOT be FOLLOWEDBY, because the join method applies to comparison expressions, not observation expressions. If there is only a single child comparison expression, `'join'` MAY be `None`.\n\nAn `'observation'` SHALL contain a set of all the object types of its child comparison expressions. This is mainly for human consumption. A STIX2 observation is allowed to contain comparisons on disparate object types, provided they're joined by OR\u2014 this is why `'objects'` is a set, not a single string.\n\nIf `'objects'` contains only a single object type, it MAY be compacted into set literal form:\n```yaml\nobservation:\n objects: {ipv4-addr}\n join: AND | OR\n qualifiers:\n expressions:\n - a\n - ...\n```\n\n\n### Qualifiers\nA Qualifier is a dict having a single key identifying its Qualifier type. Currently, this SHALL be one of:\n - [`'start_stop'`](#spec-observation-expressions-qualifiers-start-stop)\n - [`'within'`](#spec-observation-expressions-qualifiers-within)\n - [`'repeats'`](#spec-observation-expressions-qualifiers-repeats)\n\n\n#### Start/Stop Qualifier\n```py\n{'start_stop': {\n 'start': datetime(2018, 10, 7, 0, 0, tzinfo=tzutc()),\n 'stop': datetime(2018, 10, 7, 23, 59, tzinfo=tzutc()),\n}}\n```\n```yaml\nstart_stop:\n start: 2018-10-07T00:00:00Z\n stop: 2018-10-08T23:59:00Z\n```\nThe `'start_stop'` qualifier constrains the timeframe in which its associated observation expressions MUST occur within to evaluate true. Unlike `WITHIN`, `START ... STOP ...` denotes absolute points in time, using datetime literals.\n\nExample STIX2 expression:\n```\n[a:b = 12] START t'2018-10-07T00:00:00Z' STOP t'2018-10-08T23:59:00Z'\n```\nIn STIX2 Pattern Expressions, all datetimes MUST be in [RFC3339](https://tools.ietf.org/html/rfc3339) format, and MUST be in UTC timezone. datetime literals resemble Python strings with `t` as their modifying char (like an f-string, or a bytestring). Because they must be in UTC timezone, datetime literals MUST end with the `Z` char.\n\nWhen parsed into Python, they SHALL have a `tzinfo` object with a `dstoffset` of 0.\n\n\n#### Within Qualifier\n```py\n{'within': {\n 'value': 600,\n 'unit': 'SECONDS',\n}}\n```\n```yaml\nwithin:\n value: 600\n unit: SECONDS\n```\nThe `'within'` qualifier constrains the timeframe in which its associated observation expressions MUST occur within to evaluate true. Unlike `START ... STOP ...`, `WITHIN` denotes relative timeframes, where the latest observation expression MUST occur within the specified number of seconds from the earliest observation expression.\n\nExample STIX2 expression:\n```\n[a:b = 12] WITHIN 600 SECONDS\n```\n\n`SECONDS` is hard-coded into the STIX2 Pattern Expression grammar, and MUST be included in pattern expressions. However, to avoid ambiguity for the reader, and to allow for future STIX2 spec changes, the unit is also included in the Pattern Tree.\n\n\n#### Repeated Qualifier\n```py\n{'repeats': {\n 'value': 9000,\n}}\n```\n```yaml\nrepeats:\n value: 9000\n```\nThe `'repeats'` qualifier REQUIRES that its associated observation expressions evaluate true at different occasions, for a specified number of times.\n\nExample STIX2 expression:\n```\n[a:b = 12] REPEATS 9000 TIMES\n```\n\n`TIMES` is hard-coded into the STIX2 Pattern Expression grammar, and MUST be included in pattern expressions. However, since there aren't any other obvious units of multiplicity, other than \"X times\", it has been omitted from the Pattern Tree output \u2014\u00a0unlike `SECONDS` of `WITHIN`.\n\n\n## Comparison Expressions\nA Comparison Expression is a dict with a single key of either `'expression'` or `'comparison'`. An [`'expression'`](#spec-comparison-expressions-expression) SHALL contain two or more comparison expressions joined by AND/OR, whereas a [`'comparison'`](#spec-comparison-expressions-comparison) contains no children, and only marks a comparison of one variable to one literal value.\n\n\n### Expression\n```py\n{'expression': {\n 'join': oneOf('AND', 'OR'),\n 'expressions': [a, b, ...],\n}}\n```\n```yaml\nexpression:\n join: AND | OR\n expressions:\n - a\n - b\n - ...\n```\nAn `'expression'` is a container for other comparison expressions, joined by either AND or OR in `'join'` \u2014 comparison expressions do not have FOLLOWEDBY, as they are intended to reference a single object at a single point in time.\n\nAn `'expression'` MUST NOT have qualifiers.\n\nIts children are in `'expressions'`, whose values SHALL be dicts with single keys (of either `'comparison'` or `'expression'`).\n\n\n### Comparison\n```py\n{'comparison': {\n 'object': 'email-message',\n 'path': ['from_ref', 'value'],\n 'negated': None,\n 'operator': 'MATCHES',\n 'value': '.+@malicio\\\\.us',\n}}\n```\n```yaml\ncomparison:\n object: email-message\n path:\n - from_ref\n - value\n negated:\n operator: MATCHES\n value: .+@malicio\\.us\n```\nA `'comparison'` represents a single comparison between a STIX2 object property and a literal value. A single string object type SHALL be placed in the `'object'` key.\n\n`'path'` SHALL be a list beginning with a top-level property of the object type denoted in `'object'`, as a string. Following this MAY be any number of child properties, as strings, or list index components/dereferences, denoted as Python `slice()` objects, where `[1]` is equivalent to `slice(start=None, stop=1, step=None)`. The special _match any_ list index from STIX2 (e.g. `file:sections[*]`) is equivalent to `slice(start=None, stop='*', step=None)`.\n\n`'negated'` SHALL be a bool denoting whether the operator SHALL be negated during evaluation. STIX2 allows a `NOT` keyword before the operator: `file:name NOT MATCHES 'james.*'`. If the operator is not negated, `'negated'` MAY be `None`. (This allows for a more compact YAML representation \u2014 where the value may simply be omitted.)\n\n`'operator'` SHALL be a string representing the operator, e.g. `'>'`, `'LIKE'`, or `'='`.\n\n`'value'` MAY be any static Python value. Currently, only strings, bools, ints, floats, datetimes, and bytes are outputted, but this could change in the future (e.g. if compiled regular expressions are deemed useful).\n\nIf `'path'` contains only a single property, it MAY be compacted into list literal form:\n```yaml\ncomparison:\n object: domain-name\n path: [value]\n negated:\n operator: =\n value: cnn.com\n```",
"description_content_type": "text/markdown",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/usePF/dendrol",
"keywords": "",
"license": "MIT",
"maintainer": "Zach \"theY4Kman\" Kanzler",
"maintainer_email": "z@perchsecurity.com",
"name": "dendrol",
"package_url": "https://pypi.org/project/dendrol/",
"platform": "",
"project_url": "https://pypi.org/project/dendrol/",
"project_urls": {
"Homepage": "https://github.com/usePF/dendrol"
},
"release_url": "https://pypi.org/project/dendrol/0.0.5/",
"requires_dist": null,
"requires_python": "",
"summary": "The STIX2 Pattern expression library for humans",
"version": "0.0.5"
},
"last_serial": 5464232,
"releases": {
"0.0.1": [
{
"comment_text": "",
"digests": {
"md5": "aba445509c2e7dee7a6f8cb784c9166f",
"sha256": "9bec614d3fad493fcd84f76248e0c00089f0e98f7d98d559cd6de174dd12a2ef"
},
"downloads": -1,
"filename": "dendrol-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "aba445509c2e7dee7a6f8cb784c9166f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 40596,
"upload_time": "2018-11-25T17:56:35",
"url": "https://files.pythonhosted.org/packages/22/ce/296a7e5299580e3a2d9ed68768b6822265769cd9c15c35224c03c99a7c4e/dendrol-0.0.1.tar.gz"
}
],
"0.0.2": [
{
"comment_text": "",
"digests": {
"md5": "804ed2534179d837e3ce93ba0fdd94ce",
"sha256": "73512081dd59bbc66521167707af04f7fdbb104df0fab1023a0a2a8f98020934"
},
"downloads": -1,
"filename": "dendrol-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "804ed2534179d837e3ce93ba0fdd94ce",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 41561,
"upload_time": "2018-11-26T15:55:07",
"url": "https://files.pythonhosted.org/packages/b1/66/2ab7df4a873fee43e636408194ac9661c848065b602aec27780205657739/dendrol-0.0.2.tar.gz"
}
],
"0.0.3": [
{
"comment_text": "",
"digests": {
"md5": "6ad598f333736f6afe76cd51518fd9be",
"sha256": "6614166d95bacf05f285b9c6daf651dd443aef250fbec3b03fee5e782ec9a581"
},
"downloads": -1,
"filename": "dendrol-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "6ad598f333736f6afe76cd51518fd9be",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 41927,
"upload_time": "2018-11-30T15:21:40",
"url": "https://files.pythonhosted.org/packages/a7/dc/fe7a63236d3fc8ef75a5152ec8c44364de77d6d59c42d9b84dde4bfd0e25/dendrol-0.0.3.tar.gz"
}
],
"0.0.4": [
{
"comment_text": "",
"digests": {
"md5": "946065dc206e92bbb170ea4571939533",
"sha256": "af7ddbfa07ba0f6d6fbe3a0054a9bb21e0b7cf4cc0949f293b6108596cdf901a"
},
"downloads": -1,
"filename": "dendrol-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "946065dc206e92bbb170ea4571939533",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 42057,
"upload_time": "2018-11-30T16:22:42",
"url": "https://files.pythonhosted.org/packages/c2/59/9312fdc82d99662407eb5ddcb00e8cb2ada3c7b07c255c5489f83e5ceb81/dendrol-0.0.4.tar.gz"
}
],
"0.0.5": [
{
"comment_text": "",
"digests": {
"md5": "a60eb8835799e15e5fb3f2cc2dbc9fec",
"sha256": "ca085589b12636a56e0f0d148d993224f005257e0a99397243b2187433fa2902"
},
"downloads": -1,
"filename": "dendrol-0.0.5.tar.gz",
"has_sig": false,
"md5_digest": "a60eb8835799e15e5fb3f2cc2dbc9fec",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 38121,
"upload_time": "2019-03-06T22:56:29",
"url": "https://files.pythonhosted.org/packages/4d/a1/a73be4cc261d412b7976db18d8e5f54c62cb74fe870f39bd2c0ef190b6c7/dendrol-0.0.5.tar.gz"
}
],
"0.0.5.dev2": [
{
"comment_text": "",
"digests": {
"md5": "d21e9118fa6d8ee368fdbe7371d65e14",
"sha256": "5d77d632b0be71c4186ae5c14d419097fce409b621d44943ac405927f9157bca"
},
"downloads": -1,
"filename": "dendrol-0.0.5.dev2.tar.gz",
"has_sig": false,
"md5_digest": "d21e9118fa6d8ee368fdbe7371d65e14",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 37518,
"upload_time": "2019-06-27T05:24:22",
"url": "https://files.pythonhosted.org/packages/26/d1/8477184295bea4e69a4d1d49ac45eed880f32eacfc9020a9fffa2d9fc7c0/dendrol-0.0.5.dev2.tar.gz"
}
],
"0.0.5.dev35": [
{
"comment_text": "",
"digests": {
"md5": "f057207ee9719902430132ab25c4fb5f",
"sha256": "82bdd055f72daff6fe74e05dc55dceb9537049748e834c5c09d189c36fbe5c59"
},
"downloads": -1,
"filename": "dendrol-0.0.5.dev35.tar.gz",
"has_sig": false,
"md5_digest": "f057207ee9719902430132ab25c4fb5f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 37657,
"upload_time": "2019-06-29T02:23:01",
"url": "https://files.pythonhosted.org/packages/9b/52/a1b7e9c3cf0388e6c2b68fda523384deee1b147c31cf1c4cb2b0fc39f7de/dendrol-0.0.5.dev35.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "a60eb8835799e15e5fb3f2cc2dbc9fec",
"sha256": "ca085589b12636a56e0f0d148d993224f005257e0a99397243b2187433fa2902"
},
"downloads": -1,
"filename": "dendrol-0.0.5.tar.gz",
"has_sig": false,
"md5_digest": "a60eb8835799e15e5fb3f2cc2dbc9fec",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 38121,
"upload_time": "2019-03-06T22:56:29",
"url": "https://files.pythonhosted.org/packages/4d/a1/a73be4cc261d412b7976db18d8e5f54c62cb74fe870f39bd2c0ef190b6c7/dendrol-0.0.5.tar.gz"
}
]
}
![\"dendrol\"](\"https://user-images.githubusercontent.com/33840/49189283-aadc8980-f33b-11e8-9505-77f074448474.png\")
\n ![\"Iconic](\"https://user-images.githubusercontent.com/33840/47131128-027ed400-d26b-11e8-950b-4c494c9cf4c5.png\")
\n \n