{ "info": { "author": "Stefan H. Holek", "author_email": "stefan@epy.co.at", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Framework :: Zope2", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python :: 2" ], "description": "==============\r\nUnicodeLexicon\r\n==============\r\n---------------------------------------\r\nA Unicode aware lexicon for ZCTextIndex\r\n---------------------------------------\r\n\r\nMotivation\r\n==========\r\n\r\nThe standard ZCTextIndex lexicon only deals with 8-bit strings (and only\r\nif you get the zope.conf *locale* setting right).\r\nIt does not handle Unicode or UTF-8. UnicodeLexicon fills this gap.\r\n\r\nInstallation\r\n============\r\n\r\nThis product adds a ZCTextIndex Unicode Lexicon type to Zope. The\r\nlexicon comes with word splitters, stop word removers, a case normalizer,\r\nand two accent normalizers.\r\n\r\nIf you have GenericSetup installed, you can use the included extension\r\nprofile to create a UnicodeLexicon in your portal_catalog and update\r\nthe *Title*, *Description*, and *SearchableText* ZCTextIndexes.\r\n\r\nThere is no upgrade path from UnicodeLexicon 1.0. If you have 1.0 on your\r\nsystem, you have to delete and recreate the lexicon.\r\n\r\nPipeline Elements\r\n=================\r\n\r\nThe splitter works with all languages that separate words with\r\nwhitespace characters.\r\n\r\nThe stop word remover knows about English language stop words only.\r\n\r\nThe accent normalizer comes in two flavors. There is a normalizer for Latin\r\nand Western European text (fr, es, pt, it, en, nl), and one for German and\r\nScandinavian text (de, dk, no, se, fi, is). The latter keeps the umlaut\r\ncharacters \u00e4, \u00f6, and \u00fc in tact.\r\n\r\nCustom Pipeline Elements\r\n------------------------\r\n\r\nAdditional pipeline elements can be registered via ZCML. E.g.::\r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\nDefault Encoding\r\n----------------\r\n\r\nThe lexicon assumes either Unicode or UTF-8. If your application uses a\r\ndifferent encoding, you can override the default by registering the encoding\r\nas a utility::\r\n\r\n \r\n\r\n \r\n\r\n \r\n\r\nRelated\r\n=======\r\n\r\nFor CJK text you will want to use a ZCTextIndex lexicon in\r\nconjunction with CJKSplitter_. For Greek text you will want a\r\nZCTextIndex lexicon with GRSplitter_.\r\n\r\n.. _CJKSplitter: http://www.zope.org/Members/panjunyong/CJKSplitter\r\n.. _GRSplitter: http://pypi.python.org/pypi/qi.GRSplitter\r\n\r\n\r\nChangelog\r\n=========\r\n\r\n2.2 - 2011-01-30\r\n----------------\r\n\r\n- Allow to override the default encoding in ZCML.\r\n [stefan]\r\n\r\n2.1 - 2011-01-26\r\n----------------\r\n\r\n- Add ability to register pipeline elements in ZCML.\r\n [stefan]\r\n\r\n- Fix a bug when updating PipelineFactory.\r\n [stefan]\r\n\r\n2.0 - 2011-01-21\r\n----------------\r\n\r\n- Add an ordered PipelineFactory.\r\n [stefan]\r\n\r\n- Add an accent-normalizing pipeline element originally contributed by\r\n Marc-Aur\u00e9le Darche.\r\n [stefan]\r\n\r\n- Release as Python egg.\r\n [stefan]\r\n\r\n1.0 - 2006-08-14\r\n----------------\r\n\r\n- Initial release.\r\n [stefan]", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://pypi.python.org/pypi/Products.UnicodeLexicon", "keywords": "unicode zctextindex", "license": "BSD", "maintainer": "", "maintainer_email": "", "name": "Products.UnicodeLexicon", "package_url": "https://pypi.org/project/Products.UnicodeLexicon/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/Products.UnicodeLexicon/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://pypi.python.org/pypi/Products.UnicodeLexicon" }, "release_url": "https://pypi.org/project/Products.UnicodeLexicon/2.2/", "requires_dist": null, "requires_python": null, "summary": "Unicode aware lexicon for ZCTextIndex", "version": "2.2" }, "last_serial": 785138, "releases": { "2.0": [ { "comment_text": "", "digests": { "md5": "0da5741b3e433076231ee54f645889b3", "sha256": "84bea466d9f164723a44fc4202bb01bfeea052f822244bb6b1779b0c4d6d2a8d" }, "downloads": -1, "filename": "Products.UnicodeLexicon-2.0.zip", "has_sig": false, "md5_digest": "0da5741b3e433076231ee54f645889b3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22822, "upload_time": "2011-01-21T12:17:08", "url": "https://files.pythonhosted.org/packages/1f/f4/0ee6e16e96ca95de40a012dba974bd1b7afc286f2b33c2a2d55261360c0a/Products.UnicodeLexicon-2.0.zip" } ], "2.1": [ { "comment_text": "", "digests": { "md5": "a8d118c6f99ec1e45fe46bbc07f411cc", "sha256": "9ef682e5a5828c832f3d600d13ef3b00bbeb1e58974cecceba9283badbb3e590" }, "downloads": -1, "filename": "Products.UnicodeLexicon-2.1.zip", "has_sig": false, "md5_digest": "a8d118c6f99ec1e45fe46bbc07f411cc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 25274, "upload_time": "2011-01-26T16:07:11", "url": "https://files.pythonhosted.org/packages/c4/f8/173efdcb015b2a016f6c4d17075c0a2f1e1f5873802315e1db6414553c85/Products.UnicodeLexicon-2.1.zip" } ], "2.2": [ { "comment_text": "", "digests": { "md5": "67cfba7757f9a7b14c4d23ee43ef4c0b", "sha256": "98488a725a281679674c0fa80bf9a160b1a25a29207a669df115de8c1e311ed5" }, "downloads": -1, "filename": "Products.UnicodeLexicon-2.2.zip", "has_sig": false, "md5_digest": "67cfba7757f9a7b14c4d23ee43ef4c0b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 26455, "upload_time": "2011-01-30T11:54:00", "url": "https://files.pythonhosted.org/packages/4d/11/e2fa879ab94baa7aa01bb6dd7b28de78f101f6679f8907e6abc25cf6eb04/Products.UnicodeLexicon-2.2.zip" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "67cfba7757f9a7b14c4d23ee43ef4c0b", "sha256": "98488a725a281679674c0fa80bf9a160b1a25a29207a669df115de8c1e311ed5" }, "downloads": -1, "filename": "Products.UnicodeLexicon-2.2.zip", "has_sig": false, "md5_digest": "67cfba7757f9a7b14c4d23ee43ef4c0b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 26455, "upload_time": "2011-01-30T11:54:00", "url": "https://files.pythonhosted.org/packages/4d/11/e2fa879ab94baa7aa01bb6dd7b28de78f101f6679f8907e6abc25cf6eb04/Products.UnicodeLexicon-2.2.zip" } ] }