{ "info": { "author": "Bruno Lange", "author_email": "blangeram@gmail.com", "bugtrack_url": null, "classifiers": [], "description": "# artifax\n\nartifax is a Python package to evaluate nodes in a computation graph where\nthe dependencies associated with each node are extracted directly from their\nfunction signatures.\n\nA computation graph can be entirely encoded in a standard python dictionary.\nEach key represents a node or an artifact, that will eventually be computed\nonce all of its dependencies have been calculated. The value associated with\neach key can be either a constant - a string, a number or an instance of a class,\nor a function. In the latter case, the function arguments map to other nodes\nin the computation graph to establish a direct dependency between the nodes.\n\nFor example, the following dictionary:\n\n```python\nartifacts = {\n 'A': 42,\n 'B': 7,\n 'C': lambda: 10,\n 'AB': lambda A, B: A*B,\n 'C-B': lambda B, C: C() - B,\n 'greeting': 'Hello',\n 'message': lambda greeting, A: '{} World! The answer is {}.'.format(greeting, A)\n}\n```\nyields the following computation graph:\n\n![Screenshot](sample-dag.png)\n
Figure 1. Example of a computation graph.
\n\nThe `build` function evalutes the entire computation graph and returns a new dictionary\nwith the same keys as the original one and with the calculated values for each of the nodes\nin the computation graph.\n\n```python\nfrom artifax import build\n\nartifacts = {\n 'A': 42,\n 'B': 7,\n 'C': lambda: 10,\n 'AB': lambda A, B: A*B,\n 'C-B': lambda B, C: C() - B,\n 'greeting': 'Hello',\n 'message': lambda greeting, A: '{} World! The answer is {}.'.format(greeting, A)\n}\nresult = build(artifacts)\n\nfor k, v in result.items():\n print('{:<10}: {}'.format(k, v))\n```\noutputs\n```shell\nA : 42\nB : 7\nC : functools.partial( at 0x102c4fae8>)\nAB : 294\nC-B : 3\ngreeting : Hello\nmessage : Hello World! The answer is 42.\n```\n\n# Artifax class\n\nThe `build` function represents the core transformation that yields artifacts.\nIt is entirely stateless and has no side-effects. Given the same input graph, it will always\nevaluate every single node and generate the same results.\n\nWhilst these features are highly desirable from any core component, the stateful `Artifax`\nclass can be employed to interface with the build function and provide some additional features\nand performance enhancements.\n\n```python\nfrom artifax import Artifax, At\n\ndef double(x):\n return x*2\n\nafx = Artifax()\nafx.set('a', 42)\nafx.set('b', At('a', double))\n# set also accepts named arguments\nafx.set(c=lambda b: -b)\n\nassert len(afx) == 3\nassert 'b' in afx\n\nresults = afx.build()\nfor k, v in results.items():\n print(k, v)\n\n# c -84\n# a 42\n# b 84\n```\n\n## Lazy builds\n\nArtifax instances optimize sequential builds by only re-evaluating nodes that\nhave become stale due to an update. For example, given the graph illustrated in\nFigure 1, if node `B` is updated, e.g, `afx.set('B', -5))`, nodes `B`, `AB` and\n`C-B` get re-evaluated when the build method is invoked, but not any other\nnodes.\n\nIn the example below, the second call to the `build` method triggers a\nre-evaluation of node `p1` and all the nodes that depend on it. Nodes `v2` and\n`m2`, on the other hand, do not require re-evaluation since they do not depend\non the updated node.\n\n```python\nimport artifax\nimport math\n\nclass Vector:\n def __init__(self, u, v):\n self.u = u\n self.v = v\n def magnitude(self):\n print('Calculating magnitude of vector {}...'.format(self))\n return math.sqrt(self.u**2 + self.v**2)\n def __repr__(self):\n return '({}, {})'.format(self.u, self.v)\n\nafx = artifax.Artifax(\n p1=(3, 4),\n v1=lambda p1: Vector(*p1),\n m1=lambda v1: v1.magnitude(),\n v2=Vector(5, 12),\n m2=lambda v2: v2.magnitude()\n)\n_ = afx.build()\nprint('Updating p1...')\nafx.set(p1=(1, 1))\n_ = afx.build()\n```\n\n```\nCalculating magnitude of vector (3, 4)...\nCalculating magnitude of vector (5, 12)...\nUpdating p1...\nCalculating magnitude of vector (1, 1)...\n```\n\n## Targeted builds\nThe `build` method accepts an optional argument that specifies which node in\nyour computation graph should be built. Instead of returning the usual dictionary,\ntargeted builds return a tuple containing the value associated with each of the\ntarget nodes.\n\n```python\nterminal_node_value = afx.build(targets='terminal_node')\nsome_node, another_node = afx.build(targets=('node1', 'node2'))\n\n```\n\nTargeted builds only evaluate dependencies for the target node and the target node itself.\nAny other nodes in the computation graph do not get evaluated.\n\n```python\nfrom artifax import Artifax\nafx = Artifax({\n 'name': 'World',\n 'punctuation': '?',\n 'greeting': lambda name, punctuation: 'Hello, {}{}'.format(name, punctuation),\n})\ngreeting = afx.build(targets='greeting')\nprint(greeting) # prints \"Hello, World?\"\nafx.set('punctuation', '!')\ngreeting, punctuation = afx.build(targets=('greeting', 'punctuation'))\nprint(greeting) # prints \"Hello, World!\"\nprint('Cool beans{}'.format(punctuation)) # prints \"Cool beans!\"\n```\n\nTargeted builds are an efficient way of retrieving certain nodes without\nevaluating the entire computation graph.\n\n# Solvers\n\nDepending on the use case, different solvers can be employed to increase performance.\nThe `build` function and methods accept an optional `solver` parameter which defaults to\n`linear`.\n\n## The `linear` solver\n\nThe linear solver topologically sorts the computation graph in order to generate a sequence\nof nodes to be calculated in order such that for any node, all of its dependencies appear\nbefore in the sequence.\n\n## The `parallel` solver\n\nThe `parallel` solver consumes the computation graph starting from the nodes that have\nno dependencies and processes them all in parallel. When this initial set of nodes is resolved,\ntheir immediate neighbors make up the new frontier which also gets processed in parallel.\nThis procedure continues until there are no more nodes to be calculated. At any step, the\nsolver spawns one new process for each node at the frontier without exceeding the number of\navailable cores minus 1.\n\n## The `async` solver\n\nThe `async` solver takes the parallelism of the `parallel` solver one step further. It is triggered\neach time a node evaluation is completed, looking for new nodes that can be started and evaluating\nthem in a new process immediately.\n\n# Error handling\n\nIf the computation graph represented by the artifacts dictionary is not a DAG\n(Direct Acyclic Graph), a `CircularDependencyError` exception is thrown.\n\n```python\nimport artifax\ntry:\n _ = artifax.build({'x': lambda x: x+1})\nexcept artifax.CircularDependencyError as err:\n print('Cannot build artifacts: {}'.format(err))\n```\n```\nCannot build artifacts: artifact graph is not a DAG\n```\n\nIf a particular node is represented by a function for which any of its arguments isn't part\nof the computation graph, an `UnresolvedDependencyError` exception is thrown.\n\n```python\n_ = artifax.build({\n 'x': 42,\n 'p': lambda x, y: x + y\n}) # raises UnresolvedDependencyError due to missing 'y' node\n```\n\nHowever, sometimes this behavior might be desirable if we want nodes to resolve to partially\napplied functions that can be used elsewhere. If that's the case, the exception can be suppressed\nby setting the `allow_partial_functions` optional parameter to `build` to `True`.\n\n```python\nresults = artifax.build({\n 'x': 42,\n 'p': lambda x, y: x + y\n}, allow_partial_functions=True)\nprint(results['p'](100)) # prints 142\n```\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://gitlab.com/brunolange/artifax", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "artifax", "package_url": "https://pypi.org/project/artifax/", "platform": "", "project_url": "https://pypi.org/project/artifax/", "project_urls": { "Homepage": "https://gitlab.com/brunolange/artifax" }, "release_url": "https://pypi.org/project/artifax/0.1.3.1/", "requires_dist": [ "pathos", "exos", "Jinja2", "pylint ; extra == 'dev'" ], "requires_python": ">=3", "summary": "python package for building artifacts from a computational graph", "version": "0.1.3.1" }, "last_serial": 5865353, "releases": { "0.1.1": [ { "comment_text": "", "digests": { "md5": "8a1c2c5148a17896a19a572254f2935a", "sha256": "d759c86c72ebcb7005cdf1653d25f15672803f3eb3907987c4246ea1f37b67c0" }, "downloads": -1, "filename": "artifax-0.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "8a1c2c5148a17896a19a572254f2935a", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 9045, "upload_time": "2019-09-09T03:10:24", "url": "https://files.pythonhosted.org/packages/64/53/aa156271f2d773423333334a18f8613afd0ea5410d07121d1c847eb106a9/artifax-0.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0efce889acee64e709c128daa94c1588", "sha256": "a83581e4014f0f989df1f1a263c80c09d78006d68dd943546edc811879e4befd" }, "downloads": -1, "filename": "artifax-0.1.1.tar.gz", "has_sig": false, "md5_digest": "0efce889acee64e709c128daa94c1588", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 11506, "upload_time": "2019-09-09T03:10:27", "url": "https://files.pythonhosted.org/packages/b9/e6/400e6af855bcbb10a00f84756c8dbf56a46295caa9013084b7651b19e703/artifax-0.1.1.tar.gz" } ], "0.1.1.1": [ { "comment_text": "", "digests": { "md5": "d34ef566f9e286adc378871028ff0514", "sha256": "3d5b8d976913522885e6bde076d10223c55396eb75303f938833c2ffa165ee4f" }, "downloads": -1, "filename": "artifax-0.1.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "d34ef566f9e286adc378871028ff0514", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 9072, "upload_time": "2019-09-09T03:19:45", "url": "https://files.pythonhosted.org/packages/6c/7c/a41a6a72e84ddd3e71494c3cc0d97d9593cc7eac0533039ff6a6e26129c2/artifax-0.1.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e1a5fa4bfbfed6ebc6fd5ebe30a041f5", "sha256": "ffb63794768597101954aa9635ef040b0504504c7dc97a7b9e9cec8954408d9a" }, "downloads": -1, "filename": "artifax-0.1.1.1.tar.gz", "has_sig": false, "md5_digest": "e1a5fa4bfbfed6ebc6fd5ebe30a041f5", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 11522, "upload_time": "2019-09-09T03:19:48", "url": "https://files.pythonhosted.org/packages/34/9a/d0a09f05923aa417eb6c783fff8352f6016c11a2252df13619dc281a9f1a/artifax-0.1.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "2e92325116dbca9f6e76107ac0bbcb8e", "sha256": "fc467bc6bc4827d1f0b819616a9db1c4148be4c625f9d8a2aa6077608916d70c" }, "downloads": -1, "filename": "artifax-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "2e92325116dbca9f6e76107ac0bbcb8e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 9052, "upload_time": "2019-09-09T03:19:46", "url": "https://files.pythonhosted.org/packages/a6/b1/a849a196bd55dfe1d09f38e99792fdde9ec979c131dd1d43d20aac653797/artifax-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9313495f9fddbf3703f7ffc56a9f3260", "sha256": "f06762d366cc46bdc897d3480c401105f49f99cd9f55440887e608f283735f2f" }, "downloads": -1, "filename": "artifax-0.1.2.tar.gz", "has_sig": false, "md5_digest": "9313495f9fddbf3703f7ffc56a9f3260", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 11518, "upload_time": "2019-09-09T03:19:50", "url": "https://files.pythonhosted.org/packages/1e/b3/f03ba8114013f572dadc1d4cc7ce998a8931b27671795f3e15d172cf2777/artifax-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "9b77d49bb7fb49909a4667b40a93c822", "sha256": "012fe022ff4b6f78de7a4d107bfce9d758815ffcabf27ca51f2e1bb2a0b1a1bd" }, "downloads": -1, "filename": "artifax-0.1.3-py3-none-any.whl", "has_sig": false, "md5_digest": "9b77d49bb7fb49909a4667b40a93c822", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 9065, "upload_time": "2019-09-21T06:41:16", "url": "https://files.pythonhosted.org/packages/d2/47/55a0673a406cd38fc14286cbccc84cd5c1355065eb6dbe791d5117f5fd96/artifax-0.1.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "963faa2129501c5a191fee154ab649fa", "sha256": "83417f810cdafbc622f4e4692f9c444ac97e81c40e3d24b2710b26b4ae546136" }, "downloads": -1, "filename": "artifax-0.1.3.tar.gz", "has_sig": false, "md5_digest": "963faa2129501c5a191fee154ab649fa", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 11337, "upload_time": "2019-09-21T06:41:17", "url": "https://files.pythonhosted.org/packages/43/33/6261a361335440cb200cee707c0e3dfd2ddaffdc7d77c896e0968f1b5d1b/artifax-0.1.3.tar.gz" } ], "0.1.3.1": [ { "comment_text": "", "digests": { "md5": "7272400d1331dc32f85fa0e834344fdc", "sha256": "3ef5700efa0eaa12bc957c821c34d7096bc71c02d4e4a6dca3341cebf9eeb799" }, "downloads": -1, "filename": "artifax-0.1.3.1-py3-none-any.whl", "has_sig": false, "md5_digest": "7272400d1331dc32f85fa0e834344fdc", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 11934, "upload_time": "2019-09-21T06:59:54", "url": "https://files.pythonhosted.org/packages/e8/af/3c02cf5859872067618387fcf3fe363d8e31193edab802c8205014e9c4b2/artifax-0.1.3.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2e59e8fe2c7f4f33b3f6cfefcf98b7a0", "sha256": "2d5378a379730954c68a3a9083b6dddb12e4693d768cff72996be11219f98ba3" }, "downloads": -1, "filename": "artifax-0.1.3.1.tar.gz", "has_sig": false, "md5_digest": "2e59e8fe2c7f4f33b3f6cfefcf98b7a0", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 14025, "upload_time": "2019-09-21T06:59:56", "url": "https://files.pythonhosted.org/packages/63/f9/65e39ae4c81d4bbc439a4af2a586c66904f23cab0b3ff43e8fc7243e94f5/artifax-0.1.3.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "7272400d1331dc32f85fa0e834344fdc", "sha256": "3ef5700efa0eaa12bc957c821c34d7096bc71c02d4e4a6dca3341cebf9eeb799" }, "downloads": -1, "filename": "artifax-0.1.3.1-py3-none-any.whl", "has_sig": false, "md5_digest": "7272400d1331dc32f85fa0e834344fdc", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 11934, "upload_time": "2019-09-21T06:59:54", "url": "https://files.pythonhosted.org/packages/e8/af/3c02cf5859872067618387fcf3fe363d8e31193edab802c8205014e9c4b2/artifax-0.1.3.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2e59e8fe2c7f4f33b3f6cfefcf98b7a0", "sha256": "2d5378a379730954c68a3a9083b6dddb12e4693d768cff72996be11219f98ba3" }, "downloads": -1, "filename": "artifax-0.1.3.1.tar.gz", "has_sig": false, "md5_digest": "2e59e8fe2c7f4f33b3f6cfefcf98b7a0", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 14025, "upload_time": "2019-09-21T06:59:56", "url": "https://files.pythonhosted.org/packages/63/f9/65e39ae4c81d4bbc439a4af2a586c66904f23cab0b3ff43e8fc7243e94f5/artifax-0.1.3.1.tar.gz" } ] }