{ "info": { "author": "sai-prasanna", "author_email": "sai.r.prasanna@gmail.com", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Science/Research", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Topic :: Scientific/Engineering :: Artificial Intelligence" ], "description": "## lmproof - Language Model Proof Reader\n\nLibrary to do proof-reading corrections for Grammatical Errors, spelling errors, confused word errors and other errors using pre-trained Language Models.\n\n## Usage\nInstall spacy model by `python -m spacy download en`.\nThen try out this snippet.\n\n``` python\nimport lmproof\nproof_reader = lmproof.load(\"en\")\nsource = \"The foxes living on the Shire is brown.'\"\ncorrected = proof_reader.proofread(source) # \"The foxes living in the Shire are brown.\"\n```\n\n## How it works?\n\nWe use the language model based scoring approach mentioned in [Christopher Bryant and Ted Briscoe. 2018](http://aclweb.org/anthology/W18-0529) with few changes.\n\nUnlike many approaches to GEC, this approach does NOT require annotated training data and mainly depends on a monolingual language model. The program works by iteratively comparing certain words in a text against alternative candidates and applying a correction if one of these candidates is more probable than the original word. These correction candidates are variously generated by a word inflection library or are otherwise defined manually. Currently, this system only corrects:\n\n Non-words (e.g. freind and informations)\n Morphology (e.g. eat, ate, eaten, eating, etc.)\n Common Determiners and Prepositions (e.g. the, a, in, at, to, etc.)\n Commonly Confused Words (e.g. bear/bare, lose/loose, etc.)\n\nThis work builds upon https://github.com/chrisjbryant/lmgec-lite/\n\n## Components\n\n### Language Models\n* [transformers](https://github.com/huggingface/transformers)\n### Inflection generators\n* [LemmInflect](https://github.com/bjascob/LemmInflect) is used to lemmatize and generate inflections for candidate proposals to the language model.\n### Spell Checker\n* [symspellpy](https://github.com/mammothb/symspellpy) is used for obtaining spell check candidates.\n\nThe components are highly modularised to facilitate experimentation with newer scorers and support more languages.\nPre-trained language models for other languages, inflectors, common error patterns can be easily added to support more languages.\n\n## TODOs\n\n* Use edits in existing GEC corpus to generate candidates.\n* Tests\n* Publish benchmarks of the model.\n* Think of simple ways to generate insertion candidates.\n* Add more languages.\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/sai-prasanna/lmproof", "keywords": "NLP,language model,Grammatical error correction", "license": "Apache-2.0", "maintainer": "", "maintainer_email": "", "name": "lmproof", "package_url": "https://pypi.org/project/lmproof/", "platform": "", "project_url": "https://pypi.org/project/lmproof/", "project_urls": { "Homepage": "https://github.com/sai-prasanna/lmproof", "Repository": "https://github.com/sai-prasanna/lmproof" }, "release_url": "https://pypi.org/project/lmproof/0.3.0/", "requires_dist": [ "spacy (>=2.0)", "torch (>=1.2.0)", "lemminflect (>=0.2.0)", "transformers (>=2.0)", "symspellpy (>=6.0)", "dataclasses (>=0.7); python_version >= \"3.6\" and python_version < \"3.7\"" ], "requires_python": ">=3.6", "summary": "Language model powered proof reader for correcting contextual errors in natural language.", "version": "0.3.0", "yanked": false, "yanked_reason": null }, "last_serial": 7163764, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "fe7d3135b24880dd52e76f21a504fd9a", "sha256": "4da9b265065f185b1fea230f10696daab6e3c921c553f74e22f2fd5cc47a2724" }, "downloads": -1, "filename": "lmproof-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "fe7d3135b24880dd52e76f21a504fd9a", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.7", "size": 638757, "upload_time": "2019-10-22T19:25:42", "upload_time_iso_8601": "2019-10-22T19:25:42.498065Z", "url": "https://files.pythonhosted.org/packages/31/9b/154652c0e03aae4fb8eee48b8e4d40a7d50b8c65315643afe1b95eb8253d/lmproof-0.1.0-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "8998de411998b8c5cc97c1e5260d3186", "sha256": "dbc600014022460272b785828d59298a7d1511f7506fda4fc65b34b66ede68db" }, "downloads": -1, "filename": "lmproof-0.1.0.tar.gz", "has_sig": false, "md5_digest": "8998de411998b8c5cc97c1e5260d3186", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.7", "size": 639359, "upload_time": "2019-10-22T19:25:46", "upload_time_iso_8601": "2019-10-22T19:25:46.413209Z", "url": "https://files.pythonhosted.org/packages/6a/f2/918121f9e683ab53ad7c69e7c272c3a37adc9712d15c1c991ab2b34c8103/lmproof-0.1.0.tar.gz", "yanked": false, "yanked_reason": null } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "597589ba861965767604ba02be389e61", "sha256": "dd65c006a657a86df2f95d08523a51d739beb601d65a95f28e87234c4d0f2cc2" }, "downloads": -1, "filename": "lmproof-0.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "597589ba861965767604ba02be389e61", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.7", "size": 639813, "upload_time": "2019-11-27T10:59:27", "upload_time_iso_8601": "2019-11-27T10:59:27.243789Z", "url": "https://files.pythonhosted.org/packages/21/58/2e0668c3875f046d0269d26ad49030c48c0027635b93b8e331fafe85a35c/lmproof-0.2.0-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "7100eee858cb404a967c7d3b29300ecb", "sha256": "49e28034cea4f7d030fcd4f8dde2f1c0a0e45f87c52a2f030febb5a619cf60e1" }, "downloads": -1, "filename": "lmproof-0.2.0.tar.gz", "has_sig": false, "md5_digest": "7100eee858cb404a967c7d3b29300ecb", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.7", "size": 640457, "upload_time": "2019-11-27T10:59:30", "upload_time_iso_8601": "2019-11-27T10:59:30.794786Z", "url": "https://files.pythonhosted.org/packages/d4/30/f4f2832a669010cd79e07e1a064502e999de15a91ea88fb0aa8729e3e907/lmproof-0.2.0.tar.gz", "yanked": false, "yanked_reason": null } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "8a5f25b1dc2419602910c9804551755e", "sha256": "5ac2a50ccd36b8cfd5e9e518a1db63aac5df7d5fdcbe66df2fbd0161b931d01f" }, "downloads": -1, "filename": "lmproof-0.2.1-py3-none-any.whl", "has_sig": false, "md5_digest": "8a5f25b1dc2419602910c9804551755e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6", "size": 639877, "upload_time": "2020-03-06T11:43:57", "upload_time_iso_8601": "2020-03-06T11:43:57.982329Z", "url": "https://files.pythonhosted.org/packages/d3/08/899e3471d60064d1896d3c3e17874b1eca49ae03e3d94e68cd9a7c5e0e80/lmproof-0.2.1-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "96fd994339934b39dc5d98a9da4a89f1", "sha256": "4ea924a3bbceed079e778328eb68912a8bc1b881b747e91c087852ee1dc9c5aa" }, "downloads": -1, "filename": "lmproof-0.2.1.tar.gz", "has_sig": false, "md5_digest": "96fd994339934b39dc5d98a9da4a89f1", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 640627, "upload_time": "2020-03-06T11:44:00", "upload_time_iso_8601": "2020-03-06T11:44:00.113129Z", "url": "https://files.pythonhosted.org/packages/a4/7c/1b0250c4b33207fef85628db7dbcfde2208d29610ca6fabe9eef7135780b/lmproof-0.2.1.tar.gz", "yanked": false, "yanked_reason": null } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "b1f3e612ac7a5e7c87dc29c4096bee84", "sha256": "5e4ad6108f7b9af04e0e59e05ee3e6f70f370bb8e15d3f55c6365768d080b94b" }, "downloads": -1, "filename": "lmproof-0.3.0-py3-none-any.whl", "has_sig": false, "md5_digest": "b1f3e612ac7a5e7c87dc29c4096bee84", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6", "size": 639846, "upload_time": "2020-05-04T11:57:39", "upload_time_iso_8601": "2020-05-04T11:57:39.084408Z", "url": "https://files.pythonhosted.org/packages/15/db/627c4ad1b1b0631d22290a09bb108a26ef08369337c3be31ba61fd436d8d/lmproof-0.3.0-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "351a9931a81ae0b91b302ba5f20e14ef", "sha256": "eb52c8e6a319ee91d9b8192ced3b0f9b3f70272f739ee1f6cb2dad7bd0210b5e" }, "downloads": -1, "filename": "lmproof-0.3.0.tar.gz", "has_sig": false, "md5_digest": "351a9931a81ae0b91b302ba5f20e14ef", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 640503, "upload_time": "2020-05-04T11:57:40", "upload_time_iso_8601": "2020-05-04T11:57:40.881944Z", "url": "https://files.pythonhosted.org/packages/e6/99/b17a800fc017cf9af09cd3face05d2d7357dd6000824484b88f0f6e7daa6/lmproof-0.3.0.tar.gz", "yanked": false, "yanked_reason": null } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "b1f3e612ac7a5e7c87dc29c4096bee84", "sha256": "5e4ad6108f7b9af04e0e59e05ee3e6f70f370bb8e15d3f55c6365768d080b94b" }, "downloads": -1, "filename": "lmproof-0.3.0-py3-none-any.whl", "has_sig": false, "md5_digest": "b1f3e612ac7a5e7c87dc29c4096bee84", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6", "size": 639846, "upload_time": "2020-05-04T11:57:39", "upload_time_iso_8601": "2020-05-04T11:57:39.084408Z", "url": "https://files.pythonhosted.org/packages/15/db/627c4ad1b1b0631d22290a09bb108a26ef08369337c3be31ba61fd436d8d/lmproof-0.3.0-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "351a9931a81ae0b91b302ba5f20e14ef", "sha256": "eb52c8e6a319ee91d9b8192ced3b0f9b3f70272f739ee1f6cb2dad7bd0210b5e" }, "downloads": -1, "filename": "lmproof-0.3.0.tar.gz", "has_sig": false, "md5_digest": "351a9931a81ae0b91b302ba5f20e14ef", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 640503, "upload_time": "2020-05-04T11:57:40", "upload_time_iso_8601": "2020-05-04T11:57:40.881944Z", "url": "https://files.pythonhosted.org/packages/e6/99/b17a800fc017cf9af09cd3face05d2d7357dd6000824484b88f0f6e7daa6/lmproof-0.3.0.tar.gz", "yanked": false, "yanked_reason": null } ], "vulnerabilities": [] }