{
"info": {
"author": "Kyubyong Park",
"author_email": "kbpark.linguist@gmail.com",
"bugtrack_url": null,
"classifiers": [
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: Apache Software License",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7"
],
"description": "[](https://pypi.org/project/g2pC/)\n[](https://pypi.org/project/g2pC/)\n[](https://pypi.org/project/g2pc/)\n# g2pC: A Context-aware Grapheme-to-Phoneme for Chinese\nThere are several open source libraries of Chinese grapheme-to-phoneme \nconversion such as [python-pinyin](https://github.com/mozillazg/python-pinyin) or [xpinyin](https://github.com/lxneng/xpinyin). \nHowever, none of them seem to disambiguate Chinese polyphonic words like \"\u884c\" \n(\"x\u00edng\" (go, walk) vs. \"h\u00e1ng\" (line)) or \"\u4e86\" (\"le\" (completed action marker) \nvs. \"li\u01ceo\" (finish, achieve)). Instead, they pick up the most frequent pronunciation.\nAlthough that may be a simple and economic strategy, machine learning techniques can be of help.\nWe use CRF to determine the pronunciation of polyphonic words. In addition to the target word itself and its part-of-speech, which are tagged by pkuseg, its neighboring words are also featurized.\n## Requirements\n* python >= 3.6\n* pkuseg\n* sklearn_crfsuite\n## Installation\n```\npip install g2pc\n```\n## Main Features\n* Disambiguate polyphonic Chinese characters/words and return the most likely pinyin in the\n context using CRF implemented with [sklearn_crfsuite](https://github.com/TeamHG-Memex/sklearn-crfsuite).\n* By associating segmentation results provided by [pkuseg](https://arxiv.org/abs/1906.11455) with an open-source dictionary [CC-CEDICT](https://cc-cedict.org/wiki/),\ndisplay the following comprehensive information.\n * word\n * part-of-speech\n * pinyin\n * descriptive pinyin: where Chinese tone change rules are applied\n * English meaning\n * traditional equivalent\n## Algorithm (illustrated with an example)\ne.g., Input: \u6211\u5199\u4e86\u51e0\u884c\u4ee3\u7801\u3002 (I wrote a few lines of codes.)\n* STEP 1. Segment input string using [pkuseg](https://arxiv.org/abs/1906.11455).\n * -> [('\u6211', 'r'), ('\u5199', 'v'), ('\u4e86', 'u'), ('\u51e0', 'm'), ('\u884c', 'q'), ('\u4ee3\u7801', 'n'), ('\u3002', 'w')]\n* STEP 2. Look up the [CC-CEDICT](https://cc-cedict.org/wiki/). Each token, a tuple, consists of\nword, pos, pronunciation candidates, meaning candidates, traditional character candidates.\n * -> [('\u6211', 'r', ['wo3'], ['/I/me/my/'], ['\u6211']),
\n('\u5199', 'v', ['xie3'], ['/to write/'], ['\u5beb']),
\n('\u4e86', 'u', ['le5', 'liao3', 'liao4'], [dal particle ..], ['\u4e86', '\u4e86', '\u77ad']),
\n('\u51e0', 'm', ['ji3', 'ji1'], ['/how much/..'], ['\u5e7e', '\u51e0']),
\n('\u884c', 'q', ['xing2', 'hang2'], ['/to walk/..\"], ['\u884c', '\u884c']),
\n('\u4ee3\u7801', 'n', ['dai4 ma3'], ['/code/'], ['\u4ee3\u78bc']),
\n('\u3002', 'w', ['\u3002'], [''], ['\u3002'])]\n* STEP 3. For polyphonic words, we disambiguate them, using our pre-trained CRF model.\n * -> [('\u6211', 'r', 'wo3', '/I/me/my/', '\u6211'),
\n('\u5199', 'v', 'xie3', '/to write/', '\u5beb'),
\n('\u4e86', 'u', 'le5', '/(modal particle ..', '\u4e86'),
\n('\u51e0', 'm', 'ji3', '/how much/..', '\u5e7e'),
\n('\u884c', 'q', 'hang2', \"/row/..\", '\u884c'),
\n('\u4ee3\u7801', 'n', 'dai4 ma3', '/code/', '\u4ee3\u78bc'),
\n('\u3002', 'w', '\u3002', '\u3002', '', '\u3002')]\n\n* STEP 4. Tone change rules are applied.\n * -> [('\u6211', 'r', 'wo3', 'wo2', '/I/me/my/', '\u6211'),
\n('\u5199', 'v', 'xie3', 'xie3', '/to write/', '\u5beb'),
\n('\u4e86', 'u', 'le5', 'le5', '/(modal particle ..', '\u4e86'),
\n('\u51e0', 'm', 'ji3', 'ji3', '/how much/..', '\u5e7e'),
\n('\u884c', 'q', 'hang2', 'hang2, \"/row/..\", '\u884c'),
\n('\u4ee3\u7801', 'n', 'dai4 ma3', 'dai4 ma3', '/code/', '\u4ee3\u78bc'),
\n('\u3002', 'w', '\u3002', '\u3002', '', '\u3002')]\n## Usage\n```\n>>> from g2pc import G2pC\n>>> g2p = G2pC()\n>>> g2p(\"\u4e00\u5fc3\u4e00\u610f\")\n# This returns a list of tuples, each of which consists of\n# word, pos, pinyin, (tone changed) descriptive pinyin, English meaning, and equivanlent traditional character.\n[[('\u4e00\u5fc3\u4e00\u610f', \n'i', \n'yi1 xin1 yi1 yi4', \n'yi4 xin1 yi2 yi4', \n\"/concentrating one's thoughts and efforts/single-minded/bent on/intently/\", \n'\u4e00\u5fc3\u4e00\u610f')]\n```\n## Respectful comparison with other libraries\n```\n>>> text1 = \"\u6211\u5199\u4e86\u51e0\u884c\u4ee3\u7801\u3002\" # pay attention to the \u884c, which should be read as 'hang2', not 'xing2'\n>>> text2 = \"\u6765\u4e0d\u4e86\" # pay attention to the \u4e86, which should be read as 'liao3', not 'le'\n# python-pinyin\n>>> pip install pypinyin\n>>> from pypinyin import pinyin\n>>> pinyin(text1)\n[['w\u01d2'], ['xi\u011b'], ['le'], ['j\u01d0'], ['x\u00edng'], ['d\u00e0i'], ['m\u01ce'], ['\u3002']]\n>>> pinyin(text2)\n[['l\u00e1i'], ['b\u00f9'], ['le']]\n# xpinyin\n>>> pip install xpinyin\n>>> from xpinyin import Pinyin\n>>> p = Pinyin()\n>>> p.get_pinyin(text1, tone_marks=\"numbers\") \n'wo3-xie3-le5-ji1-xing2-dai4-ma3-\u3002'\n>>> p.get_pinyin(text2, tone_marks=\"numbers\") \n'lai2-bu4-le5'\n```\n## Changelog\n### 0.9.9.3 July 10, 2019\n* Refined the tone change rules.\n\n### 0.9.9.2 July 10, 2019\n* Refined the `cedict.pkl`. \n\n### 0.9.9.1 July 9, 2019\n* Fixed a bug of failing to find Chinese characters for names. (See [this](https://github.com/Kyubyong/g2pC/issues/3))\n\n### 0.9.6. July 7, 2019\n* Fixed a bug of failing to converting words not found in the dictionary.\n* Rearragned the `cedict.pkl`.\n* Refined the CRF model.\n* Added tone change rules. (See [this](https://github.com/Kyubyong/g2pC/issues/1))\n### 0.9.4. July 4, 2019\n* Initial launch\n## References\nIf you use our software for research, please cite:\n```\n@misc{gp2C2019,\n author = {Park, Kyubyong},\n title = {g2pC},\n year = {2019},\n publisher = {GitHub},\n journal = {GitHub repository},\n howpublished = {\\url{https://github.com/Kyubyong/g2pC}}\n}\n```\n\n\n",
"description_content_type": "text/markdown",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/Kyubyong/g2pC",
"keywords": "",
"license": "Apache License 2.0",
"maintainer": "",
"maintainer_email": "",
"name": "g2pC",
"package_url": "https://pypi.org/project/g2pC/",
"platform": "",
"project_url": "https://pypi.org/project/g2pC/",
"project_urls": {
"Homepage": "https://github.com/Kyubyong/g2pC"
},
"release_url": "https://pypi.org/project/g2pC/0.9.9.3/",
"requires_dist": [
"pkuseg (>=0.0.22)",
"sklearn-crfsuite (>=0.3.6)"
],
"requires_python": ">=3.6",
"summary": "g2pC: A Context-aware g2p module for Chinese",
"version": "0.9.9.3"
},
"last_serial": 5511420,
"releases": {
"0.9.4": [
{
"comment_text": "",
"digests": {
"md5": "642c4787a42fe11c3a277384323d34ae",
"sha256": "d22628c33a268c9be67b2862aaeee5387bb50bbf740706c6188fc418fac2c83a"
},
"downloads": -1,
"filename": "g2pC-0.9.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "642c4787a42fe11c3a277384323d34ae",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 6336727,
"upload_time": "2019-07-05T15:33:01",
"url": "https://files.pythonhosted.org/packages/f5/ef/d4cb12734c4a76192294d2894efc3cffd882a53356c60d364b6f741fe7f8/g2pC-0.9.4-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "f8072946bf4aefa4a65e840aed0b0289",
"sha256": "7aa49ecdf49b04c703ca08b0dc28d7995bd16cdffc76e42c3c04c9f33b9af0db"
},
"downloads": -1,
"filename": "g2pC-0.9.4.tar.gz",
"has_sig": false,
"md5_digest": "f8072946bf4aefa4a65e840aed0b0289",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 6316445,
"upload_time": "2019-07-05T15:33:05",
"url": "https://files.pythonhosted.org/packages/60/98/2ba9099d21c05f1fb898a6115c580131b8be2fb24cd88badb5f32baa3f4f/g2pC-0.9.4.tar.gz"
}
],
"0.9.6": [
{
"comment_text": "",
"digests": {
"md5": "c4f870aa9676e2e54bd1330f5d31c85d",
"sha256": "57d8edda2f6646deeece3ee9ff27b9e6534d68fad9c1df3525f48db08ed2a28d"
},
"downloads": -1,
"filename": "g2pC-0.9.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c4f870aa9676e2e54bd1330f5d31c85d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 6299386,
"upload_time": "2019-07-07T09:18:03",
"url": "https://files.pythonhosted.org/packages/ba/ef/52d2f104e7792e2c5b69735f815e640feaebbdbb2c153b17b3559732997b/g2pC-0.9.6-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "997f87858b9d4d4400c307aa71ecce9e",
"sha256": "81737396fb1998440cac5011e46ec43992e93d5b13de5f08d59be3df598f2696"
},
"downloads": -1,
"filename": "g2pC-0.9.6.tar.gz",
"has_sig": false,
"md5_digest": "997f87858b9d4d4400c307aa71ecce9e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 6278415,
"upload_time": "2019-07-07T09:18:06",
"url": "https://files.pythonhosted.org/packages/77/69/21c2580ff9e4a9f33fc52bc60b3e41209ce4895cd5bb3f283bb9f7382fbc/g2pC-0.9.6.tar.gz"
}
],
"0.9.9.1": [
{
"comment_text": "",
"digests": {
"md5": "b6b014b9e6a788b70f6e6cf45beba9bc",
"sha256": "672ae4724bfe60d9127e86e0ed23feb0f48ec19e055f5e730bb3b5ef2c40db75"
},
"downloads": -1,
"filename": "g2pC-0.9.9.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b6b014b9e6a788b70f6e6cf45beba9bc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 7710420,
"upload_time": "2019-07-09T01:09:54",
"url": "https://files.pythonhosted.org/packages/89/38/567d1b51ed2b8a3ccdf01e46a7792c167b9ee582face502f5bc745b9179a/g2pC-0.9.9.1-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "ad84d58a635cf0d384d54b9dc9139807",
"sha256": "1737328c04c7eee580e21d7ed3f85b345c3d578aba507fe5d7d4800bcad65ac1"
},
"downloads": -1,
"filename": "g2pC-0.9.9.1.tar.gz",
"has_sig": false,
"md5_digest": "ad84d58a635cf0d384d54b9dc9139807",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 7688385,
"upload_time": "2019-07-09T01:09:59",
"url": "https://files.pythonhosted.org/packages/55/aa/e68ceba31793275a297ce7d1692f29161fa895807a2c9283bc49969b7a42/g2pC-0.9.9.1.tar.gz"
}
],
"0.9.9.2": [
{
"comment_text": "",
"digests": {
"md5": "2ba6525ac2ca11aad6d3e5c8e88976a1",
"sha256": "d29832c217db1d268552dcc239f717aeb72073e89883bf396f982fea92c8ade5"
},
"downloads": -1,
"filename": "g2pC-0.9.9.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2ba6525ac2ca11aad6d3e5c8e88976a1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 7710611,
"upload_time": "2019-07-10T03:14:59",
"url": "https://files.pythonhosted.org/packages/99/86/7c1342c6f64c6a72030320eb4f27bc083be8fd98f4044644dd56888a4d4e/g2pC-0.9.9.2-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "c6c28b16253f5836f9346199b98e6f3e",
"sha256": "8ee1194b2af035dc7f7884a6eeaf56f35f83b46a9bdc7c13916c486cf0a0fccc"
},
"downloads": -1,
"filename": "g2pC-0.9.9.2.tar.gz",
"has_sig": false,
"md5_digest": "c6c28b16253f5836f9346199b98e6f3e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 7688528,
"upload_time": "2019-07-10T03:15:03",
"url": "https://files.pythonhosted.org/packages/75/fd/ca4929536bc9048e3242fc9b2e62b28e8adb49ed36fa35272c9ebd49576c/g2pC-0.9.9.2.tar.gz"
}
],
"0.9.9.3": [
{
"comment_text": "",
"digests": {
"md5": "dd22d3f0f7dac9919bcbf40b7cd6556a",
"sha256": "395f9237b4c3d0cace7fca63f9b3693e5071d7e894cf7e0ea2a38d6f8017b30a"
},
"downloads": -1,
"filename": "g2pC-0.9.9.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "dd22d3f0f7dac9919bcbf40b7cd6556a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 7710723,
"upload_time": "2019-07-10T10:54:17",
"url": "https://files.pythonhosted.org/packages/91/e4/d825a92b0bc684b03554969cbe39584a744b2b0cf3e28ebd89076b82d496/g2pC-0.9.9.3-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "50ba313602a2f0ea9ae6a8346a00cad3",
"sha256": "4695d169512a24a16516658dbeb161af756f93ebdbc23ae5d7071ea5d8202ea4"
},
"downloads": -1,
"filename": "g2pC-0.9.9.3.tar.gz",
"has_sig": false,
"md5_digest": "50ba313602a2f0ea9ae6a8346a00cad3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 7688615,
"upload_time": "2019-07-10T10:54:21",
"url": "https://files.pythonhosted.org/packages/83/84/b5a1d5300ffbb491506581db72f4cd872b9027a5fca72bd7a2ed660de117/g2pC-0.9.9.3.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "dd22d3f0f7dac9919bcbf40b7cd6556a",
"sha256": "395f9237b4c3d0cace7fca63f9b3693e5071d7e894cf7e0ea2a38d6f8017b30a"
},
"downloads": -1,
"filename": "g2pC-0.9.9.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "dd22d3f0f7dac9919bcbf40b7cd6556a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 7710723,
"upload_time": "2019-07-10T10:54:17",
"url": "https://files.pythonhosted.org/packages/91/e4/d825a92b0bc684b03554969cbe39584a744b2b0cf3e28ebd89076b82d496/g2pC-0.9.9.3-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "50ba313602a2f0ea9ae6a8346a00cad3",
"sha256": "4695d169512a24a16516658dbeb161af756f93ebdbc23ae5d7071ea5d8202ea4"
},
"downloads": -1,
"filename": "g2pC-0.9.9.3.tar.gz",
"has_sig": false,
"md5_digest": "50ba313602a2f0ea9ae6a8346a00cad3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 7688615,
"upload_time": "2019-07-10T10:54:21",
"url": "https://files.pythonhosted.org/packages/83/84/b5a1d5300ffbb491506581db72f4cd872b9027a5fca72bd7a2ed660de117/g2pC-0.9.9.3.tar.gz"
}
]
}