{ "info": { "author": "Sasho Savkov", "author_email": "me@sasho.io", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Operating System :: Unix", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.2", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Scientific/Engineering" ], "description": "bratutils\n=========\n[![CircleCI](https://circleci.com/gh/savkov/bratutils.svg?style=svg&circle-token=9a7bdcb066c87c45017fe2214c71f2e2f9672c94)](https://circleci.com/gh/savkov/bratutils)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\nA collection of utilities for manipulating data and calculating inter-annotator \nagreement in brat annotation files.\n\n### Installation\n\nInstall as a normal package from the source directory.\n\n```bash\n$ pip install bratutils\n```\n\n\n### Agreement Definition\n\nAgreement in multi-token annotations is commonly evaluated using [f-score][fsc].\ndue to various problems with computing the traditional [Krippendorf's alpha][al] \nand [Cohen's kappa][ka]. [Hripcsak][hripcsak] prove the validity of the metric \nfor very large populations, i.e. for unrestricted text annotations.\n\nThis library roughly follows the definitions of precision and recall calculation\nfrom the [MUC-7 test scoring][muc]. The basic definitions along with some \nadditional restrictions are laid out below:\n\n* `CORRECT` - when annotation tags and indices match completely\n* `INCORRECT` - when annotation tags do not match, but the indices coincide\n* `PARTIAL` - when the annotation tags are the same but one of the annotations\nhas the same end index and a different start index\n* `MISSING` - annotations exising only in the gold standard annotation set\n* `SPURIOUS` - annotations existing only in the candidate annotation set\n\n_Note_: the gold standard is considered the collections/document from which the \n comparison is invoked, while the supplied parallel annotation is considered \n the candidate set.\n\n_*Disclaimer:*_ the current definition of the `PARTIAL` category accomodates \nworking with syntactic chunks. A different arrangement (e.g. pick largest \ncontained tag as partial match instead of rightmost) might be more suitable for \nother tasks, for example some types of semantic annotation.\n\n\n### Examples\n\nSimple example:\n\n```python\nfrom bratutils import agreement as a\n\ndoc = a.Document('res/samples/A/data-sample-1.ann')\ndoc2 = a.Document('res/samples/B/data-sample-1.ann')\n\ndoc.make_gold()\nstatistics = doc2.compare_to_gold(doc)\n\nprint(statistics)\n```\n\nOutput:\n\n```shell\n-------------------MUC-Table--------------------\n------------------------------------------------\npos:135\nact:134\ncor:115\npar:5\ninc:4\nmis:11\nspu:10\n------------------------------------------------\npre:0.858208955224\nrec:0.851851851852\nfsc:0.855018587361\n------------------------------------------------\nund:0.0814814814815\novg:0.0746268656716\nsub:0.0725806451613\n------------------------------------------------\nbor:119\nibo:15\n------------------------------------------------\n------------------------------------------------\n```\n\n\n[fsc]: \n[al]: \n[ka]: \n[hripcsak]: \n[muc]: \n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/savkov/transcriptor", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "bratutils", "package_url": "https://pypi.org/project/bratutils/", "platform": "", "project_url": "https://pypi.org/project/bratutils/", "project_urls": { "Homepage": "https://github.com/savkov/transcriptor" }, "release_url": "https://pypi.org/project/bratutils/0.2.2/", "requires_dist": null, "requires_python": "", "summary": "brat utilities", "version": "0.2.2" }, "last_serial": 5319967, "releases": { "0.2.1": [ { "comment_text": "", "digests": { "md5": "7b681572092cb8d54bb9ae13665dc94f", "sha256": "27d7b83a5701540111677ba6c51a4a67217e14f2882401248f613157b6a6c3c2" }, "downloads": -1, "filename": "bratutils-0.2.1-py3-none-any.whl", "has_sig": false, "md5_digest": "7b681572092cb8d54bb9ae13665dc94f", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 26945, "upload_time": "2019-05-26T21:14:21", "url": "https://files.pythonhosted.org/packages/20/b2/81cb8f5827532e3aadade55ca4d9184859d2933e83747f742324c6fb5e01/bratutils-0.2.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9c56f0a79d8dd2bdfb9139034cdeb067", "sha256": "6b8a07f23663988c2933ce7861a470abe8799fdf3929087e2d63d3fabbf6799f" }, "downloads": -1, "filename": "bratutils-0.2.1.tar.gz", "has_sig": false, "md5_digest": "9c56f0a79d8dd2bdfb9139034cdeb067", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13025, "upload_time": "2019-05-26T21:14:23", "url": "https://files.pythonhosted.org/packages/67/7d/5eca0bf2ef1115d448bc85a2a9de070cd291ba91778d007289a5986aacba/bratutils-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "3a65f4ac98119421535dcfd489b86a19", "sha256": "b99115b4806a7f1cc8a6499d5dee9b03b2af0b1d353370af4a0e88a289d08c7a" }, "downloads": -1, "filename": "bratutils-0.2.2-py3-none-any.whl", "has_sig": false, "md5_digest": "3a65f4ac98119421535dcfd489b86a19", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 17393, "upload_time": "2019-05-26T22:47:27", "url": "https://files.pythonhosted.org/packages/b8/36/ef2c5ef29e0aaa5c2fe19e99bb9567f70de86c8e9ad35352b9a828d82e17/bratutils-0.2.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e60cf2f9a8e796aeb97bd1f5c1e6ed2d", "sha256": "6eb242b537598cf8acc06baffb84e5593dc17c6fdca6658516f0fb9ec72fa467" }, "downloads": -1, "filename": "bratutils-0.2.2.tar.gz", "has_sig": false, "md5_digest": "e60cf2f9a8e796aeb97bd1f5c1e6ed2d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13604, "upload_time": "2019-05-26T22:47:28", "url": "https://files.pythonhosted.org/packages/e6/67/62cc3b15db0c0d71523700769452c42b2995ac9327312c354945b1a5ce45/bratutils-0.2.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "3a65f4ac98119421535dcfd489b86a19", "sha256": "b99115b4806a7f1cc8a6499d5dee9b03b2af0b1d353370af4a0e88a289d08c7a" }, "downloads": -1, "filename": "bratutils-0.2.2-py3-none-any.whl", "has_sig": false, "md5_digest": "3a65f4ac98119421535dcfd489b86a19", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 17393, "upload_time": "2019-05-26T22:47:27", "url": "https://files.pythonhosted.org/packages/b8/36/ef2c5ef29e0aaa5c2fe19e99bb9567f70de86c8e9ad35352b9a828d82e17/bratutils-0.2.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e60cf2f9a8e796aeb97bd1f5c1e6ed2d", "sha256": "6eb242b537598cf8acc06baffb84e5593dc17c6fdca6658516f0fb9ec72fa467" }, "downloads": -1, "filename": "bratutils-0.2.2.tar.gz", "has_sig": false, "md5_digest": "e60cf2f9a8e796aeb97bd1f5c1e6ed2d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13604, "upload_time": "2019-05-26T22:47:28", "url": "https://files.pythonhosted.org/packages/e6/67/62cc3b15db0c0d71523700769452c42b2995ac9327312c354945b1a5ce45/bratutils-0.2.2.tar.gz" } ] }