{ "info": { "author": "Jussi Judin", "author_email": "jjudin+python-libguess@iki.fi", "bugtrack_url": null, "classifiers": [ "Environment :: Console", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Programming Language :: Python :: 2", "Topic :: Software Development :: Libraries" ], "description": "libguess_ enables finding out in what encoding some text is for\r\ngiven language. This library is especially useful for short text strings\r\nthat themselves don't carry reliable encoding information, like ID3 tags\r\nin MP3 files. For example we can encode the same Japanese string in 3\r\ndifferent encodings (UTF-8, SHIFT-JIS, EUC-JP) and libguess_ will\r\nhopefully guess the encoding correctly just by looking at the first\r\nfew bytes of the given string.\r\n\r\n.. _libguess: http://www.atheme.org/project/libguess\r\n\r\nYou can also use this as a regular command line program by giving\r\nregion as the first parameter and possible target files after that::\r\n\r\n Usage: python -m guess REGION [INPUT_FILE]\r\n\r\n If input file name is not given, this program reads from the standard input.\r\n\r\nFunctions\r\n=========\r\n\r\ndetermine_encoding(in_string, region)\r\n-------------------------------------\r\n\r\nDetermines encoding of a string for the given language region.\r\n\r\n| Arguments:\r\n| ``in_string`` -- a raw byte string for which encoding needs to be guessed.\r\n| ``region`` -- one of the ``REGION_*`` constants for which encoding is guessed.\r\n\r\n| Returns:\r\n| String value indicating the guessed encoding of in_string argument for given region or ``None`` if error happened.\r\n\r\nAs an usage example we might see what happens when given Japanese\r\ntext in 2 different encodings::\r\n\r\n >>> import guess\r\n >>> guess.determine_encoding(u'\\u3042'.encode('euc-jp'), guess.REGION_JP)\r\n 'EUC-JP'\r\n >>> guess.determine_encoding(u'\\u3042'.encode('utf-8'), guess.REGION_JP)\r\n 'UTF-8'\r\n\r\nThe output string of this function can be given directly\r\nto ``iconv_open()`` C function and the resulting names should be\r\ncompatible with the encoding string of ``str.decode()`` function::\r\n\r\n >>> encoded_value = u'\\u3042'.encode('shift-jis')\r\n >>> encoding = guess.determine_encoding(encoded_value, guess.REGION_JP)\r\n >>> encoding\r\n 'SJIS'\r\n >>> encoded_value.decode(encoding)\r\n u'\\u3042'\r\n\r\nIn case the given region name is invalid or the underlying\r\n``libguess_determine_encoding()`` call fails for any other reason,\r\nNone value is returned::\r\n\r\n >>> encoding = guess.determine_encoding(\"asdf\", \"UNKNOWN\")\r\n >>> encoding is None\r\n True\r\n\r\nUse ``REGION_*`` constants for region names.\r\n\r\nvalidate_utf8(in_string)\r\n------------------------\r\n\r\nChecks if the given string is a valid UTF-8 byte sequence.\r\n\r\n| Arguments:\r\n| ``in_string`` -- a raw byte string to be inspected for UTF-8 validity.\r\n\r\n| Return value:\r\n| ``True`` if given string is a valid UTF-8 byte sequence, ``False`` otherwise.\r\n\r\nThis function is included here for the completeness with libguess\r\ninterface. It gives more precise results for UTF-8 validity than\r\nfor example functions in glib.\r\n\r\n::\r\n\r\n >>> import guess\r\n >>> guess.validate_utf8(u'\\u3042'.encode('EUC-JP'))\r\n False\r\n >>> guess.validate_utf8(u'\\u3042'.encode('UTF-8'))\r\n True\r\n\r\nRegion names\r\n============\r\n\r\n| ``REGION_AR`` = 'arabic'\r\n| ``REGION_BL`` = 'baltic'\r\n| ``REGION_CN`` = 'chinese'\r\n| ``REGION_GR`` = 'greek'\r\n| ``REGION_HW`` = 'hebrew'\r\n| ``REGION_JP`` = 'japanese'\r\n| ``REGION_KR`` = 'korean'\r\n| ``REGION_PL`` = 'polish'\r\n| ``REGION_RU`` = 'russian'\r\n| ``REGION_TR`` = 'turkish'\r\n| ``REGION_TW`` = 'taiwanese'", "description_content_type": null, "docs_url": "https://pythonhosted.org/python-libguess/", "download_url": "https://bitbucket.org/barro/python-libguess/", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "UNKNOWN", "keywords": "", "license": "UNKNOWN", "maintainer": "", "maintainer_email": "", "name": "python-libguess", "package_url": "https://pypi.org/project/python-libguess/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/python-libguess/", "project_urls": { "Download": "https://bitbucket.org/barro/python-libguess/", "Homepage": "UNKNOWN" }, "release_url": "https://pypi.org/project/python-libguess/1.1/", "requires_dist": null, "requires_python": null, "summary": "Python wrapper for libguess, a library to determine character encoding of a string.", "version": "1.1" }, "last_serial": 797973, "releases": { "1.0": [ { "comment_text": "", "digests": { "md5": "09b1ff97228dc0011b23c745d8455f5e", "sha256": "704a41ddcc655de68b649eeb22c646098eaa228da1a0a4740aadc8c41daed97e" }, "downloads": -1, "filename": "python-libguess-1.0.tar.gz", "has_sig": false, "md5_digest": "09b1ff97228dc0011b23c745d8455f5e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2684, "upload_time": "2011-04-10T23:18:34", "url": "https://files.pythonhosted.org/packages/be/72/469b33a973118be3639ad45b9080c79679e5c2111e7b6686f8eb78597670/python-libguess-1.0.tar.gz" } ], "1.1": [ { "comment_text": "", "digests": { "md5": "622f6f8a094b30e156b27d9d4faa0c19", "sha256": "4359f6eb30782d9f27dde37eafc9520c5b1be97184664b61278ae786a0c6bed9" }, "downloads": -1, "filename": "python-libguess-1.1.tar.gz", "has_sig": false, "md5_digest": "622f6f8a094b30e156b27d9d4faa0c19", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4054, "upload_time": "2011-04-19T17:11:05", "url": "https://files.pythonhosted.org/packages/5a/9c/6881c04d966afde868871ed5673bcb8d4bd2b129d88217800d61317661c9/python-libguess-1.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "622f6f8a094b30e156b27d9d4faa0c19", "sha256": "4359f6eb30782d9f27dde37eafc9520c5b1be97184664b61278ae786a0c6bed9" }, "downloads": -1, "filename": "python-libguess-1.1.tar.gz", "has_sig": false, "md5_digest": "622f6f8a094b30e156b27d9d4faa0c19", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4054, "upload_time": "2011-04-19T17:11:05", "url": "https://files.pythonhosted.org/packages/5a/9c/6881c04d966afde868871ed5673bcb8d4bd2b129d88217800d61317661c9/python-libguess-1.1.tar.gz" } ] }