{ "info": { "author": "Paul Logston", "author_email": "code@logston.me", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)", "Programming Language :: Python", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: Implementation :: CPython", "Topic :: Text Processing", "Topic :: Text Processing :: Filters" ], "description": "cUnidecode\n==========\nlossy ASCII transliterations of Unicode text\n--------------------------------------------\n\n.. image:: https://travis-ci.org/logston/cunidecode.svg?branch=master\n :target: https://travis-ci.org/logston/cunidecode\n\nIt often happens that you have text data in Unicode, but you need to\nrepresent it in ASCII. For example when integrating with legacy code that\ndoesn't support Unicode, or for ease of entry of non-Roman names on a US\nkeyboard, or when constructing ASCII machine identifiers from\nhuman-readable Unicode strings that should still be somewhat intelligible\n(a popular example of this is when making an URL slug from an article\ntitle). \n\nIn most of these examples you could represent Unicode characters as\n\"???\" or \"\\\\15BA\\\\15A0\\\\1610\", to mention two extreme cases. But that's\nnearly useless to someone who actually wants to read what the text says.\n\nWhat Unidecode provides is a middle road: function unidecode() takes\nUnicode data and tries to represent it in ASCII characters (i.e., the\nuniversally displayable characters between 0x00 and 0x7F), where the\ncompromises taken when mapping between two character sets are chosen to be\nnear what a human with a US keyboard would choose.\n\nThe quality of resulting ASCII representation varies. For languages of\nwestern origin it should be between perfect and good. On the other hand\ntransliteration (i.e., conveying, in Roman letters, the pronunciation\nexpressed by the text in some other writing system) of languages like\nChinese, Japanese or Korean is a very complex issue and this library does\nnot even attempt to address it. It draws the line at context-free\ncharacter-by-character mapping. So a good rule of thumb is that the further\nthe script you are transliterating is from Latin alphabet, the worse the\ntransliteration will be.\n\nNote that this module generally produces better results than simply\nstripping accents from characters (which can be done in Python with\nbuilt-in functions). It is based on hand-tuned character mappings that for\nexample also contain ASCII approximations for symbols and non-Latin\nalphabets.\n\nThis is a C port of Unidecode by Paul Logston \n\nUnidecode (by Tomaz Solc) is a Python port of the Text::Unidecode Perl module by\nSean M. Burke and later\n\n\nModule content\n--------------\n\nThe module exports a single function that takes an Unicode object (Python\n2.x) and returns a string::\n\n >>> from cunidecode import unidecode\n >>> unidecode(u'ko\\u017eu\\u0161\\u010dek')\n 'kozuscek'\n >>> unidecode(u'30 \\U0001d5c4\\U0001d5c6/\\U0001d5c1')\n '30 km/h'\n >>> unidecode(u\"\\u5317\\u4EB0\")\n 'Bei Jing '\n\n\nRequirements\n------------\n\nPython 2.x >= 2.6 and a C compiler/linker.\n \nThis implementation of unidecode does not \"decode\" characters outside\nof the Basic Multilingual Plane.\nA Python build with \"wide\" Unicode characters may lead to a segmentation\nfault if an attempt to decode a character outside the BMP is made.\n\nSurrogate pair encoding of \"narrow\" builds is not supported.\n\n\nInstallation\n------------\n\nYou install Unidecode, as you would install any Python module, by running\nthese commands::\n\n python setup.py install\n python setup.py test\n\n\nSource\n------\n\nYou can get the latest development version of cUnidecode with::\n\n git clone git@github.com:logston/cunidecode.git\n\n\nSupport\n-------\n\nQuestions, bug reports, useful code bits, and suggestions for Unidecode\nshould be sent to tomaz.solc@tablix.org\n\nQuestions, bug reports, useful code bits, and suggestions for cUnidecode\nshould be sent to code@logston.me\n\n\nCopyright\n---------\n\nOriginal character transliteration tables:\n\nCopyright 2001, Sean M. Burke , all rights reserved.\n\nPython code and later additions to Unidecode:\n\nCopyright 2014, Tomaz Solc \n\nC code and later additions to cUnidecode:\n\nCopyright 2014, Paul Logston \n\nThis program is free software; you can redistribute it and/or modify it\nunder the terms of the GNU General Public License as published by the Free\nSoftware Foundation; either version 2 of the License, or (at your option)\nany later version.\n\nThis program is distributed in the hope that it will be useful, but WITHOUT\nANY WARRANTY; without even the implied warranty of MERCHANTABILITY or\nFITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for\nmore details.\n\nYou should have received a copy of the GNU General Public License along\nwith this program; if not, write to the Free Software Foundation, Inc., 51\nFranklin Street, Fifth Floor, Boston, MA 02110-1301 USA. The programs and\ndocumentation in this dist are distributed in the hope that they will be\nuseful, but without any warranty; without even the implied warranty of\nmerchantability or fitness for a particular purpose.", "description_content_type": null, "docs_url": null, "download_url": "https://github.com/logston/cunidecode/tarball/0.0.2", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/logston/cunidecode", "keywords": "unicode,c,unidecode", "license": "GPL", "maintainer": null, "maintainer_email": null, "name": "cunidecode", "package_url": "https://pypi.org/project/cunidecode/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/cunidecode/", "project_urls": { "Download": "https://github.com/logston/cunidecode/tarball/0.0.2", "Homepage": "https://github.com/logston/cunidecode" }, "release_url": "https://pypi.org/project/cunidecode/0.0.2/", "requires_dist": null, "requires_python": null, "summary": "ASCII transliterations of Unicode text", "version": "0.0.2" }, "last_serial": 1203148, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "709efeb1fb59c8866e9ac3aaa33b2d6b", "sha256": "f7226f4193cf74fa20a94743403162b7f48c2b3ba34f830f2f38ec43c614b23c" }, "downloads": -1, "filename": "cunidecode-0.0.1.tar.gz", "has_sig": false, "md5_digest": "709efeb1fb59c8866e9ac3aaa33b2d6b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14952, "upload_time": "2014-08-26T17:17:21", "url": "https://files.pythonhosted.org/packages/fc/7a/668f8a48ef077bedbf7d6c7854f3140dadd4e9b6300509c0cc3e8dd4dc48/cunidecode-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "753a0ef6b8de9d8f5ccd710dd4792ee7", "sha256": "343bccb26473da1639dbd336f52395b02457466bd4474e4808ede48cdf8de3c4" }, "downloads": -1, "filename": "cunidecode-0.0.2.tar.gz", "has_sig": false, "md5_digest": "753a0ef6b8de9d8f5ccd710dd4792ee7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 95731, "upload_time": "2014-08-26T17:21:33", "url": "https://files.pythonhosted.org/packages/ae/ba/ebc3ed127d8cebf11b8a266d46b311399584a2a5a7c4316b94e5fba66c35/cunidecode-0.0.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "753a0ef6b8de9d8f5ccd710dd4792ee7", "sha256": "343bccb26473da1639dbd336f52395b02457466bd4474e4808ede48cdf8de3c4" }, "downloads": -1, "filename": "cunidecode-0.0.2.tar.gz", "has_sig": false, "md5_digest": "753a0ef6b8de9d8f5ccd710dd4792ee7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 95731, "upload_time": "2014-08-26T17:21:33", "url": "https://files.pythonhosted.org/packages/ae/ba/ebc3ed127d8cebf11b8a266d46b311399584a2a5a7c4316b94e5fba66c35/cunidecode-0.0.2.tar.gz" } ] }