{ "info": { "author": "Sumudu Tennakoon", "author_email": "", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Education", "Intended Audience :: Science/Research", "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python", "Topic :: Scientific/Engineering", "Topic :: Software Development" ], "description": "# TextAnalyticsLab (Text Analytics Toolkit for Python)\n## Current release: TextLab [v0.1.2]\n\n\n\n## TextAnalyticsLab - a collection of Text Analytics tools for Python.\n\n\n\n## Introduction\n'TextAnalyticsLab' is a Python package providing a set of text analytics tools \nfor data mining and machine learning projects and end-to-end text analytics \napplication development. It is compatible with and interoperate with data \nanalysis and manipulation library Pandas, natural language processing library \nnltk, Machine Lerning TookKit (pymltoolkit|mltk), and many other AI and machine \nlearning platforms. \n\n## Installation\n```\npip install TextAnalyticsLab\n```\nIf the installation failed with dependancy issues, execute the above command with --no-dependencies\n\n```\npip install TextAnalyticsLab --no-dependencies\n```\n\n## Functions\n- Text Similarity\n- Text Mining and Information Extraction (in v0.2.0)\n- Cleaning Text content (in v0.1.5)\n- Web Scraping (in v0.1.5)\n- Classification of Text Conent (in v0.2.0)\n\n\n## Usage\n```python\nimport textlab\n```\n\n### Warning: Python Variable, Function or Class names \nThe Python interpreter has a number of built-in functions. It is possible to overwrite thier definitions when coding without any rasing a warning from the Python interpriter. (https://docs.python.org/3/library/functions.html)\nTherfore, AVOID THESE NAMES as your variable, function or class names.\n\n\n\n\n\n\n\n\n\n\n
absallanyasciibinboolbytearraybytes
callablechrclassmethodcompilecomplexdelattrdictdir
divmodenumerateevalexecfilterfloatformatfrozenset
getattrglobalshasattrhashhelphexidinput
intisinstanceissubclassiterlenlistlocalsmap
maxmemoryviewminnextobjectoctopenord
powprintpropertyrangereprreversedroundset
setattrslicesortedstaticmethodstrsumsupertuple
typevarszip__import__
\n\nIf you accedently overwrite any of the built-in function (e.g. list), execute the following to bring built-in defition.\n```python\ndel(list)\n```\n\n```\n\n## Text Analytics Example\n\n### Text Similarity\n```python\nimport textlab\n\nstr1 = 'Hello'\nstr2 = 'Hola'\n\ndl_distance = textlab.damerau_levenshtein_distance(str1, str2, case_sensitive=True, normalized=False)\nprint('damerau_levenshtein_distance: ', dl_distance)\n\ndl_distance_normalized = textlab.damerau_levenshtein_distance(str1, str2, case_sensitive=True, normalized=True)\nprint('damerau_levenshtein_distance (normalized): ', dl_distance_normalized)\n\nsubstrings = textlab.get_substrings(string=str1, case_sensitive=True, min_length=2, max_length=np.inf)\nprint('substrings: ', substrings)\n\nj_index = textlab.jaccard_index(str1, str2, method='substring', case_sensitive=True, min_length=1, max_length=np.inf) #method='words'\nprint('jaccard_index: ', round(j_index,3))\n```\n\n```\ndamerau_levenshtein_distance: 3\ndamerau_levenshtein_distance (normalized): 0.6\nsubstrings: ['He', 'll', 'Hel', 'el', 'llo', 'lo', 'ello', 'Hell', 'Hello', 'ell']\njaccard_index: 0.143\n```\n\n```python\n# A paragraph from Wikipedia: https://en.wikipedia.org/wiki/Albert_Einstein\ntext = \"\"\"\nAlbert Einstein; 14 March 1879 \u00e2\u20ac\u201c 18 April 1955) was a German-born theoretical physicist[5] who developed the theory of relativity, one of the two pillars of modern physics (alongside quantum mechanics).[3][6]:274 His work is also known for its influence on the philosophy of science.[7][8] He is best known to the general public for his mass\u00e2\u20ac\u201cenergy equivalence formula {\\displaystyle E=mc^{2}} E = mc^2, which has been dubbed \"the world's most famous equation\".[9] He received the 1921 Nobel Prize in Physics \"for his services to theoretical physics, and especially for his discovery of the law of the photoelectric effect\",[10] a pivotal step in the development of quantum theory.\n\"\"\"\n\ntext1 = normalize_text(text, method='str')\ntext2 = normalize_text(text, method='regex')\n```\ntext1\n```\n'albert einstein march \u00e2\u20ac\u201c april was a germanborn theoretical physicist who developed the theory of relativity one of the two pillars of modern physics alongside quantum mechanics his work is also known for its influence on the philosophy of science he is best known to the general public for his mass\u00e2\u20ac\u201cenergy equivalence formula displaystyle emc e mc which has been dubbed the worlds most famous equation he received the nobel prize in physics for his services to theoretical physics and especially for his discovery of the law of the photoelectric effect a pivotal step in the development of quantum theory'\n```\ntext2\n```\n'albert einstein march april was a germanborn theoretical physicist who developed the theory of relativity one of the two pillars of modern physics alongside quantum mechanics his work is also known for its influence on the philosophy of science he is best known to the general public for his mass energy equivalence formula displaystyle emc e mc which has been dubbed the worlds most famous equation he received the nobel prize in physics for his services to theoretical physics and especially for his discovery of the law of the photoelectric effect a pivotal step in the development of quantum theory'\n```\n```python\n#Text from Wikipedia page: https://en.wikipedia.org/wiki/Email_address\ntext = \"\"\"\nAn email address identifies an email box to which email messages are delivered. A wide variety of formats were used in early email systems, but only a single format is used today, following the standards developed for Internet mail systems since the 1980s. This article uses the term email address to refer to the addr-spec defined in RFC 5322, not to the address that is commonly used; the difference is that an address may contain a display name, a comment, or both.\n\nValid email addresses\nsimple@example.com\nvery.common@example.com\ndisposable.style.email.with+symbol@example.com\nother.email-with-hyphen@example.com\nfully-qualified-domain@example.com\nuser.name+tag+sorting@example.com (may go to user.name@example.com inbox depending on mail server)\nx@example.com (one-letter local-part)\nexample-indeed@strange-example.com\nadmin@mailserver1 (local domain name with no TLD, although ICANN highly discourages dotless email addresses)\nexample@s.example (see the List of Internet top-level domains)\n\" \"@example.org (space between the quotes)\n\"john..doe\"@example.org (quoted double dot)\nInvalid email addresses\nAbc.example.com (no @ character)\nA@b@c@example.com (only one @ is allowed outside quotation marks)\na\"b(c)d,e:f;gi[j\\k]l@example.com (none of the special characters in this local-part are allowed outside quotation marks)\njust\"not\"right@example.com (quoted strings must be dot separated or the only element making up the local-part)\nthis is\"not\\allowed@example.com (spaces, quotes, and backslashes may only exist when within quoted strings and preceded by a backslash)\nthis\\ still\\\"not\\\\allowed@example.com (even if escaped (preceded by a backslash), spaces, quotes, and backslashes must still be contained by quotes)\n\"\"\"\n\nemail_addresses = extract_email_addresses(text)\n```\n```\n['simple@example.com',\n 'very.common@example.com',\n 'disposable.style.email.with+symbol@example.com',\n 'other.email-with-hyphen@example.com',\n 'fully-qualified-domain@example.com',\n 'user.name+tag+sorting@example.com',\n 'user.name@example.com',\n 'example-indeed@strange-example.com',\n 'example@s.example',\n 'right@example.com',\n 'llowed@example.com',\n 'allowed@example.com']\n ```\n\n```python\n# Scrape Wikipedia page to get a list of countries and Codes for the representation of names of countries and their subdivisions.\n\ntablle = extract_tables_webpage(r'https://en.wikipedia.org/wiki/ISO_3166-1')[1] # Required information in the 2nd table extracted\ntablle.sample(6)\n```\n\n```\n English short name (using title case) Alpha-2 code Alpha-3 code Numeric code Link to ISO 3166-2 subdivision codes Independent\n143 Mexico MX MEX 484 ISO 3166-2:MX Yes\n220 Thailand TH THA 764 ISO 3166-2:TH Yes\n233 United Arab Emirates AE ARE 784 ISO 3166-2:AE Yes\n81 Gambia GM GMB 270 ISO 3166-2:GM Yes\n148 Montenegro ME MNE 499 ISO 3166-2:ME Yes\n21 Belgium BE BEL 56 ISO 3166-2:BE Yes\n```\n\n## License\n```\nCopyright 2019 Sumudu Tennakoon\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n```\n\n## Text Analytics Project Timeline\n- 2018-07-10 [v0.0.1]: Initial set of functions for text data analysis was published to Github. (https://github.com/sptennak/TextAnalytics).\n- 2019-01-03 [v0.0.2]: Created more functions for data exploration including web scraping and geo spacial data analysis for for IBM Coursera Data Science Capstone Project was published to Github. (https://github.com/sptennak/Coursera_Capstone).\n- 2019-07-20 [v0.1.2]: First release of the \"TextLab\" Text Analytics Python package \n\n## Future Release Plan\n- TBD [v0.1.5]: Integreate Cleaning Text content and Web scraping\n- 2019-12-31 [v0.1.6]: Comprehensive documentation, Major bug-fix version of the initial release with some enhancements.\n- [v0.2.0]: Integrate Text Mining, Information Extraction, and Classification.\n- [v0.3.0]: End-to-end Text Analytics Application Development\n\n## References\n- https://pandas.pydata.org/\n- https://www.numpy.org/\n- https://docs.python.org/3.6/library/re.html\n- https://matplotlib.org/\n\n## Other helpful text Anlytics and Natural Language Processing Python libraries\n- https://www.nltk.org/\n- https://textblob.readthedocs.io\n- https://radimrehurek.com/gensim/\n- https://spacy.io/\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/sptennak/TextAnalytics", "keywords": "", "license": "Apache License Version 2.0", "maintainer": "", "maintainer_email": "", "name": "TextAnalyticsLab", "package_url": "https://pypi.org/project/TextAnalyticsLab/", "platform": "", "project_url": "https://pypi.org/project/TextAnalyticsLab/", "project_urls": { "Homepage": "https://github.com/sptennak/TextAnalytics" }, "release_url": "https://pypi.org/project/TextAnalyticsLab/0.1.2/", "requires_dist": [ "numpy", "scipy", "matplotlib", "pandas", "pymltoolkit", "beautifulsoup4" ], "requires_python": "", "summary": "Text Analytics Toolkit (TextAnalyticsLab) for Python", "version": "0.1.2" }, "last_serial": 5561975, "releases": { "0.1.2": [ { "comment_text": "", "digests": { "md5": "635b900fe4d6e6fd892fa9c3b1f0a3a1", "sha256": "cbe024fdf527cb1292d641777ee93b7d39cc5470df8bab3cba07552eb920079f" }, "downloads": -1, "filename": "TextAnalyticsLab-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "635b900fe4d6e6fd892fa9c3b1f0a3a1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 19383, "upload_time": "2019-07-21T01:22:09", "url": "https://files.pythonhosted.org/packages/7d/f2/4547e17869dc75c78ce4655760ba1d57d0c80b0364205ff95757e6e20df1/TextAnalyticsLab-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "84eeaa842714d7e149748e865be305f2", "sha256": "0819ec02cbf55ee3cc20903b428f171d24b044280cff62e28fbbee7eae8f28c4" }, "downloads": -1, "filename": "TextAnalyticsLab-0.1.2.tar.gz", "has_sig": false, "md5_digest": "84eeaa842714d7e149748e865be305f2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11696, "upload_time": "2019-07-21T01:22:12", "url": "https://files.pythonhosted.org/packages/87/fc/bf0c08f4aa39ab35d39ff9c609113a2db8447ef096a6c1d41b5a816ee8f4/TextAnalyticsLab-0.1.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "635b900fe4d6e6fd892fa9c3b1f0a3a1", "sha256": "cbe024fdf527cb1292d641777ee93b7d39cc5470df8bab3cba07552eb920079f" }, "downloads": -1, "filename": "TextAnalyticsLab-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "635b900fe4d6e6fd892fa9c3b1f0a3a1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 19383, "upload_time": "2019-07-21T01:22:09", "url": "https://files.pythonhosted.org/packages/7d/f2/4547e17869dc75c78ce4655760ba1d57d0c80b0364205ff95757e6e20df1/TextAnalyticsLab-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "84eeaa842714d7e149748e865be305f2", "sha256": "0819ec02cbf55ee3cc20903b428f171d24b044280cff62e28fbbee7eae8f28c4" }, "downloads": -1, "filename": "TextAnalyticsLab-0.1.2.tar.gz", "has_sig": false, "md5_digest": "84eeaa842714d7e149748e865be305f2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11696, "upload_time": "2019-07-21T01:22:12", "url": "https://files.pythonhosted.org/packages/87/fc/bf0c08f4aa39ab35d39ff9c609113a2db8447ef096a6c1d41b5a816ee8f4/TextAnalyticsLab-0.1.2.tar.gz" } ] }