{
    "info": {
        "author": "Christopher Potts, H. Andrew Schwartz, Maarten Sap, Salvatore Giorgi, Caspar Chan",
        "author_email": "hansens@sas.upenn.edu, sgiorgi@sas.upenn.edu",
        "bugtrack_url": null,
        "classifiers": [
            "Environment :: Console",
            "Intended Audience :: Developers",
            "Intended Audience :: End Users/Desktop",
            "Intended Audience :: Science/Research",
            "Natural Language :: English",
            "Programming Language :: Python",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.6",
            "Topic :: Scientific/Engineering"
        ],
        "description": "# Happiest Fun Tokenizer for python3\nThis code is modified in order to adapt to Python3. And according to my own requirements, auther added some functions or params to preserve the specific tokens like nicknames, urls, and hashtags.\nThe former version of code implements a basic, Twitter-aware tokenizer. Originally developed by [Christopher Potts](http://web.stanford.edu/~cgpotts/) \n([Happy Fun Tokenizer](http://sentiment.christopherpotts.net/code-data/happyfuntokenizing.py)) and updated by [H. Andrew Schwartz](http://www3.cs.stonybrook.edu/~has/). Shared with Christopher's permission. Further updated by [Caspar Chan](https://github.com/channel960608/happiestfuntokenizing).  \n\n\n### Usage\n\n```python\nfrom happiestfuntokenizing.happiestfuntokenizing import Tokenizer\n\ntokenizer = Tokenizer()\n\nmessage = \"\"\"OMG!!!! :) I looooooove this tokenizer lololol\"\"\"\ntokens = tokenizer.tokenize(message)\nprint(tokens)\n['omg', '!', '!', '!', '!', ':)', 'i', 'looooooove', 'this', 'tokenizer', 'lololol']\n\nmessage = \"\"\"OMG!!!! :) I looooooove this tokenizer LoLoLoLoLooOOOOL\"\"\"\ntokenizer = Tokenizer(preserve_case=True)\ntokens = tokenizer.tokenize(message)\nprint(tokens)\n['OMG', '!', '!', '!', '!', ':)', 'I', 'looooooove', 'this', 'tokenizer', 'LoLoLoLoLooOOOOL']\n\nmessage = \"\"\"RT @cahnnle #happyfuncoding: this is a typical Twitter tweet :-) https://twiter/test.phd\"\"\"  \ntokenizer = Tokenizer(preserve_keywords=True)  \ntokens = tokenizer.tokenize(message)\nprint(tokens)\n['rt', '@cahnnle', '#happyfuncoding', ':', 'this', 'is', 'a', 'typical', 'twitter', 'tweet', ':-)']   \nprint(tokenizer..get_preserve_dict())\n{'username': ['@cahnnle'], 'url': ['https://twiter/test.phd'], 'hashtag': ['#happyfuncoding']}\n\n\n```\n\n### Installation\n\nThis is available through `pip`\n\n```sh\npip install happiestfuntokenizing\n```\n\nIf you do not have sudo privileges you can use the `--user` flag\n\n```sh\npip install --user happiestfuntokenizing\n```\n\n## Requirements\n\nThis uses Python 3.* Package dependencies include `re` and `html`.\n\n## License\n\nLicensed under a [GNU General Public License v3 (GPLv3)](https://www.gnu.org/licenses/gpl-3.0.en.html)\n\n## Background\n\nAdapted by the [World Well-Being Project](http://www.wwbp.org) based out of the University of Pennsylvania\nand Stony Brook University. Originally developed by [Christopher Potts](http://web.stanford.edu/~cgpotts/).",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "https://github.com/channel960608/happiestfuntokenizing",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://dlatk.wwbp.org",
        "keywords": "",
        "license": "GNU General Public License v3 (GPLv3)",
        "maintainer": "",
        "maintainer_email": "",
        "name": "happiestfuntokenizing",
        "package_url": "https://pypi.org/project/happiestfuntokenizing/",
        "platform": "",
        "project_url": "https://pypi.org/project/happiestfuntokenizing/",
        "project_urls": {
            "Download": "https://github.com/channel960608/happiestfuntokenizing",
            "Homepage": "http://dlatk.wwbp.org"
        },
        "release_url": "https://pypi.org/project/happiestfuntokenizing/0.0.7/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "This code is updated by author to adapt to Python3. The former version of code implements a basic, Twitter-aware tokenizer. Originally developed by Christopher Potts and updated by the World Well-Being Project based out of the University of Pennsylvania and Stony Brook University. Shared with permission.",
        "version": "0.0.7"
    },
    "last_serial": 4785336,
    "releases": {
        "0.0.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4678ab7040f18c644dd1fa7ff32f9e58",
                    "sha256": "7f588b8ea671c7515aeaeefc8fe71ce2d41379cbd52cac95c48c98bbef24fc96"
                },
                "downloads": -1,
                "filename": "happiestfuntokenizing-0.0.6.tar.gz",
                "has_sig": false,
                "md5_digest": "4678ab7040f18c644dd1fa7ff32f9e58",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6537,
                "upload_time": "2019-01-25T16:48:38",
                "url": "https://files.pythonhosted.org/packages/18/de/e74da04d953ef876a06272f321e3a786478a06767825f0c054d9c1d4a58c/happiestfuntokenizing-0.0.6.tar.gz"
            }
        ],
        "0.0.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ec21885d512edeca7e2e78186750aadb",
                    "sha256": "55289efd27faaa05b38cef9f6cb00ccc54e47c83addf6cc43e61ac6f86bdc213"
                },
                "downloads": -1,
                "filename": "happiestfuntokenizing-0.0.7.tar.gz",
                "has_sig": false,
                "md5_digest": "ec21885d512edeca7e2e78186750aadb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6349,
                "upload_time": "2019-02-06T07:02:51",
                "url": "https://files.pythonhosted.org/packages/38/d0/6eeab22df5e414b456b3de8fdb71b866950b66ff9f8f19ea8af4dd8554af/happiestfuntokenizing-0.0.7.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "ec21885d512edeca7e2e78186750aadb",
                "sha256": "55289efd27faaa05b38cef9f6cb00ccc54e47c83addf6cc43e61ac6f86bdc213"
            },
            "downloads": -1,
            "filename": "happiestfuntokenizing-0.0.7.tar.gz",
            "has_sig": false,
            "md5_digest": "ec21885d512edeca7e2e78186750aadb",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 6349,
            "upload_time": "2019-02-06T07:02:51",
            "url": "https://files.pythonhosted.org/packages/38/d0/6eeab22df5e414b456b3de8fdb71b866950b66ff9f8f19ea8af4dd8554af/happiestfuntokenizing-0.0.7.tar.gz"
        }
    ]
}