{
    "info": {
        "author": "Jay Yip",
        "author_email": "junpang.yip@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6"
        ],
        "description": "![python](https://img.shields.io/badge/python%20-3.6.0-brightgreen.svg) [![tensorflow](https://img.shields.io/badge/tensorflow-1.13.1-green.svg)](https://www.tensorflow.org/) [![PyPI version fury.io](https://badge.fury.io/py/ansicolortags.svg)](https://pypi.python.org/pypi/bert-multitask-learning/) [![PyPI license](https://img.shields.io/pypi/l/ansicolortags.svg)](https://pypi.python.org/pypi/bert-multitask-learning/)\n\n\n# Bert for Multi-task Learning\n\n[\u4e2d\u6587\u6587\u6863](#Bert\u591a\u4efb\u52a1\u5b66\u4e60)\n\n## Install\n\n```\npip install bert-multitask-learning\n```\n\n## What is it\n\nThis a project that uses [BERT](https://github.com/google-research/bert) to do **multi-task learning** with multiple GPU support.\n\n## Why do I need this\n\nIn the original BERT code, neither multi-task learning or multiple GPU training is possible. Plus, the original purpose of this project is NER which dose not have a working script in the original BERT code.\n\nTo sum up, compared to the original bert repo, this repo has the following features:\n\n1. Multi-task learning(major reason of re-writing the majority of code).\n2. Multiple GPU training\n3. Support sequence labeling (for example, NER) and Encoder-Decoder Seq2Seq(with transformer decoder).\n\n## What type of problems are supported?\n\n- Masked LM and next sentence prediction Pre-train(pretrain)\n- Classification(cls)\n- Sequence Labeling(seq_tag)\n- Seq2seq Labeling(seq2seq_tag)\n- Seq2seq Text Generation(seq2seq_text)\n- Multi-Label Classification(multi_cls)\n\n## How to run pre-defined problems\n\nThere are two types of chaining operations can be used to chain problems.\n\n- `&`. If two problems have the same inputs, they can be chained using `&`. Problems chained by `&` will be trained at the same time.\n- `|`. If two problems don't have the same inputs, they need to be chained using `|`. Problems chained by `|` will be sampled to train at every instance.\n\nFor example, `cws|NER|weibo_ner&weibo_cws`, one problem will be sampled at each turn, say `weibo_ner&weibo_cws`, then `weibo_ner` and `weibo_cws` will trained for this turn together. Therefore, in a particular batch, some tasks might not be sampled, and their loss could be 0 in this batch.\n\nPlease see the examples in [notebooks](notebooks/) for more details about training, evaluation and export models.\n\n\n# Bert\u591a\u4efb\u52a1\u5b66\u4e60\n\n## \u5b89\u88c5\n\n```\npip install bert-multitask-learning\n```\n\n## \u8fd9\u662f\u4ec0\u4e48\n\n\u8fd9\u662f\u5229\u7528[BERT](https://github.com/google-research/bert)\u8fdb\u884c**\u591a\u4efb\u52a1\u5b66\u4e60**\u5e76\u4e14\u652f\u6301\u591aGPU\u8bad\u7ec3\u7684\u9879\u76ee.\n\n## \u6211\u4e3a\u4ec0\u4e48\u9700\u8981\u8fd9\u4e2a\u9879\u76ee\n\n\u5728\u539f\u59cb\u7684BERT\u4ee3\u7801\u4e2d, \u662f\u6ca1\u6709\u529e\u6cd5\u76f4\u63a5\u7528\u591aGPU\u8fdb\u884c\u591a\u4efb\u52a1\u5b66\u4e60\u7684. \u53e6\u5916, BERT\u5e76\u6ca1\u6709\u7ed9\u51fa\u5e8f\u5217\u6807\u6ce8\u548cSeq2seq\u7684\u8bad\u7ec3\u4ee3\u7801.\n\n\u56e0\u6b64, \u548c\u539f\u6765\u7684BERT\u76f8\u6bd4, \u8fd9\u4e2a\u9879\u76ee\u5177\u6709\u4ee5\u4e0b\u7279\u70b9:\n\n1. \u591a\u4efb\u52a1\u5b66\u4e60\n2. \u591aGPU\u8bad\u7ec3\n3. \u5e8f\u5217\u6807\u6ce8\u4ee5\u53caEncoder-decoder seq2seq\u7684\u652f\u6301(\u7528transformer decoder)\n\n## \u76ee\u524d\u652f\u6301\u7684\u4efb\u52a1\u7c7b\u578b\n\n- Masked LM\u548cnext sentence prediction\u9884\u8bad\u7ec3(pretrain)\n- \u5355\u6807\u7b7e\u5206\u7c7b(cls)\n- \u5e8f\u5217\u6807\u6ce8(seq_tag)\n- \u5e8f\u5217\u5230\u5e8f\u5217\u6807\u7b7e\u6807\u6ce8(seq2seq_tag)\n- \u5e8f\u5217\u5230\u5e8f\u5217\u6587\u672c\u751f\u6210(seq2seq_text)\n- \u591a\u6807\u7b7e\u5206\u7c7b(multi_cls)\n\n## \u5982\u4f55\u8fd0\u884c\u9884\u5b9a\u4e49\u4efb\u52a1\n\n### \u76ee\u524d\u652f\u6301\u7684\u4efb\u52a1\n\n- \u4e2d\u6587\u547d\u540d\u5b9e\u4f53\u8bc6\u522b\n- \u4e2d\u6587\u5206\u8bcd\n- \u4e2d\u6587\u8bcd\u6027\u6807\u6ce8\n\n\n\u53ef\u4ee5\u7528\u4e24\u79cd\u65b9\u6cd5\u6765\u5c06\u591a\u4e2a\u4efb\u52a1\u8fde\u63a5\u8d77\u6765.\n\n- `&`. \u5982\u679c\u4e24\u4e2a\u4efb\u52a1\u6709\u76f8\u540c\u7684\u8f93\u5165, \u4e0d\u540c\u6807\u7b7e\u7684\u8bdd, \u90a3\u4e48\u4ed6\u4eec**\u53ef\u4ee5**\u7528`&`\u6765\u8fde\u63a5. \u88ab`&`\u8fde\u63a5\u8d77\u6765\u7684\u4efb\u52a1\u4f1a\u88ab\u540c\u65f6\u8bad\u7ec3.\n- `|`. \u5982\u679c\u4e24\u4e2a\u4efb\u52a1\u4e3a\u4e0d\u540c\u7684\u8f93\u5165, \u90a3\u4e48\u4ed6\u4eec**\u5fc5\u987b**\u7528`|`\u6765\u8fde\u63a5. \u88ab`|`\u8fde\u63a5\u8d77\u6765\u7684\u4efb\u52a1\u4f1a\u88ab\u968f\u673a\u62bd\u53d6\u6765\u8bad\u7ec3.\n\n\u4f8b\u5982, \u6211\u4eec\u5b9a\u4e49\u4efb\u52a1`cws|NER|weibo_ner&weibo_cws`, \u90a3\u4e48\u5728\u751f\u6210\u6bcf\u4e00\u6761\u6570\u636e\u65f6, \u4e00\u4e2a\u4efb\u52a1\u5757\u4f1a\u88ab\u968f\u673a\u62bd\u53d6\u51fa\u6765, \u4f8b\u5982\u5728\u8fd9\u4e00\u6b21\u62bd\u6837\u4e2d, `weibo_ner&weibo_cws`\u88ab\u9009\u4e2d. \u90a3\u4e48\u8fd9\u6b21`weibo_ner`\u548c`weibo_cws`\u4f1a\u88ab\u540c\u65f6\u8bad\u7ec3. \u56e0\u6b64, \u5728\u4e00\u4e2abatch\u4e2d, \u6709\u53ef\u80fd\u67d0\u4e9b\u4efb\u52a1\u6ca1\u6709\u88ab\u62bd\u4e2d, loss\u4e3a0.\n\n\u8bad\u7ec3, eval\u548c\u5bfc\u51fa\u6a21\u578b\u8bf7\u89c1[notebooks](notebooks/)\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/JayYip/bert-multitask-learning",
        "keywords": "",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "bert-multitask-learning",
        "package_url": "https://pypi.org/project/bert-multitask-learning/",
        "platform": "",
        "project_url": "https://pypi.org/project/bert-multitask-learning/",
        "project_urls": {
            "Homepage": "https://github.com/JayYip/bert-multitask-learning"
        },
        "release_url": "https://pypi.org/project/bert-multitask-learning/0.2.9/",
        "requires_dist": [
            "numpy",
            "joblib",
            "tqdm",
            "six",
            "pandas",
            "pytablewriter",
            "setuptools",
            "tensor2tensor",
            "nltk",
            "scikit-learn",
            "atlas",
            "pyarrow",
            "tensorflow-gpu (==1.13.1)"
        ],
        "requires_python": ">=3.5.0",
        "summary": "BERT for Multi-task Learning",
        "version": "0.2.9"
    },
    "last_serial": 5719164,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f1186e60ee2f3ebca9b9975ceb5961f9",
                    "sha256": "adc18469bfed2d869a00f965dff2bd8fd184e15556681b8969a9d75d44afe40c"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f1186e60ee2f3ebca9b9975ceb5961f9",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5.0",
                "size": 59128,
                "upload_time": "2019-05-21T13:31:35",
                "url": "https://files.pythonhosted.org/packages/26/dc/553663c9c86aee448504d484bb780a90176000c4f7d1a4a65fadb2b72452/bert_multitask_learning-0.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "1c9f12d8e32336695bf8136c57ef0342",
                    "sha256": "c7c4871da6cb5316723a77377e641437bb9578946a4e5ec7e4da6afc31608fd9"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "1c9f12d8e32336695bf8136c57ef0342",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5.0",
                "size": 48680,
                "upload_time": "2019-05-21T13:31:38",
                "url": "https://files.pythonhosted.org/packages/25/b8/ebb6c9faca7c2f1b9d4b28cd060ce08866a89534a4f4d45d24a6fc17a0e3/bert_multitask_learning-0.1.0.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "32cada911822c38f2c9546296ac98561",
                    "sha256": "8b49d8057b1cee55600ef186a4a1e45d7361553f770487fe90f21771591a5674"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.2.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "32cada911822c38f2c9546296ac98561",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5.0",
                "size": 59289,
                "upload_time": "2019-05-30T09:56:16",
                "url": "https://files.pythonhosted.org/packages/69/0d/ff00acefc21437154dc8309cf724c099b7ec7f0de73ad6260194c7eeb07e/bert_multitask_learning-0.2.0-py3-none-any.whl"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b6c42b592e085652adb6a6eb0516bafc",
                    "sha256": "f66deff03851c5293b538863a487a7e7d77e9013e4394890513a888f6ac734da"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.2.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b6c42b592e085652adb6a6eb0516bafc",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5.0",
                "size": 59167,
                "upload_time": "2019-05-31T09:35:03",
                "url": "https://files.pythonhosted.org/packages/3a/11/7956f04e74ef175f7473a7c99006e1d8baa300ed961abf98a55918e7c97c/bert_multitask_learning-0.2.1-py3-none-any.whl"
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "75f1f6e884c476373fe6a4e8ca1850f3",
                    "sha256": "44882eddc76145d820afa5e9b81e284516fc91600cdfd09238a480bc3de4e1c4"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.2.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "75f1f6e884c476373fe6a4e8ca1850f3",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5.0",
                "size": 59423,
                "upload_time": "2019-06-03T14:04:16",
                "url": "https://files.pythonhosted.org/packages/e4/e8/eb675b98df6647258b5a8e606b529b174a78655cfcf27518bf899a83e909/bert_multitask_learning-0.2.2-py3-none-any.whl"
            }
        ],
        "0.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7e8b3a473c8a74bf23f2e889bff7e89d",
                    "sha256": "7d48f3797e640e0a3d91668d3f736fa32ae1787e29bbe5890b2391fdce9bf7b0"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.2.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "7e8b3a473c8a74bf23f2e889bff7e89d",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5.0",
                "size": 62365,
                "upload_time": "2019-06-10T06:12:48",
                "url": "https://files.pythonhosted.org/packages/71/87/6a34ac77209b6981a1e68ac468dc3c9f6f63fee939409e385437b11825ad/bert_multitask_learning-0.2.3-py3-none-any.whl"
            }
        ],
        "0.2.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b7f9fe687105eda8032e297bee68a998",
                    "sha256": "53823f3d13094b65704b50d0c10bda0fd90ae91af85b8612e096b2887b03ba26"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.2.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b7f9fe687105eda8032e297bee68a998",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5.0",
                "size": 62145,
                "upload_time": "2019-06-13T11:27:28",
                "url": "https://files.pythonhosted.org/packages/0d/cd/0dee3526c02504a27f7ee6495ca4413fbe21b1d1eeccbdba63ff8f872c23/bert_multitask_learning-0.2.4-py3-none-any.whl"
            }
        ],
        "0.2.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9c646d3f283903b5558783e7a274ac54",
                    "sha256": "319822ee4298cf288be158220e4b6a31d1a011aa2a28e803fb04a08c8f2234a1"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.2.5-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "9c646d3f283903b5558783e7a274ac54",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5.0",
                "size": 62150,
                "upload_time": "2019-06-21T02:33:58",
                "url": "https://files.pythonhosted.org/packages/ab/c5/f358507955d8847845a87a2261843605be4d1d3c924876580c96c9dc2a08/bert_multitask_learning-0.2.5-py3-none-any.whl"
            }
        ],
        "0.2.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "034300fb555134b32f2f54902b8b0180",
                    "sha256": "fadff8f75e7e6f83ae54239a98bd1af6ba76e9d4549560be2dc87508fcf0fce9"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.2.6-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "034300fb555134b32f2f54902b8b0180",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5.0",
                "size": 62502,
                "upload_time": "2019-06-25T04:36:03",
                "url": "https://files.pythonhosted.org/packages/c4/24/96c8e84a013b4110330bcf2a27ad7c7cef4e0b20516526396e280ce16cc2/bert_multitask_learning-0.2.6-py3-none-any.whl"
            }
        ],
        "0.2.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "01576ee794fb728de3a8a4e477e31a46",
                    "sha256": "6f6a323ddf929c08cd0401246f4fd5d8dde8d3bc2e0d451f8f4e32e30f679782"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.2.7-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "01576ee794fb728de3a8a4e477e31a46",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5.0",
                "size": 62497,
                "upload_time": "2019-06-25T04:41:32",
                "url": "https://files.pythonhosted.org/packages/12/30/e4d6cdba2ef016b4cc142f09992bf688a307db462361c305743a1869e4bb/bert_multitask_learning-0.2.7-py3-none-any.whl"
            }
        ],
        "0.2.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3c895c2d3d00ce41c8434c0db3f2f853",
                    "sha256": "a54510fe7595614aa715616ffc2a9d899d92f32f239fd9a3501355408c588bdd"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.2.8-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "3c895c2d3d00ce41c8434c0db3f2f853",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5.0",
                "size": 72389,
                "upload_time": "2019-07-04T03:30:29",
                "url": "https://files.pythonhosted.org/packages/91/32/9609162ac330514b1547ffe3daab0f9018859db3fab4df16f2bd0f525fd8/bert_multitask_learning-0.2.8-py3-none-any.whl"
            }
        ],
        "0.2.9": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "880d98001f9a7cef86445de77495a61e",
                    "sha256": "3d87039cbb814975a5578357bddc39a8d5ba7655982b7729c30ef27d5570704a"
                },
                "downloads": -1,
                "filename": "bert_multitask_learning-0.2.9-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "880d98001f9a7cef86445de77495a61e",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5.0",
                "size": 73546,
                "upload_time": "2019-08-23T06:49:20",
                "url": "https://files.pythonhosted.org/packages/f1/8c/98e27bcda23ca46fa5cb78ce524a5d166e073718080554c2780a61e08851/bert_multitask_learning-0.2.9-py3-none-any.whl"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "880d98001f9a7cef86445de77495a61e",
                "sha256": "3d87039cbb814975a5578357bddc39a8d5ba7655982b7729c30ef27d5570704a"
            },
            "downloads": -1,
            "filename": "bert_multitask_learning-0.2.9-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "880d98001f9a7cef86445de77495a61e",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.5.0",
            "size": 73546,
            "upload_time": "2019-08-23T06:49:20",
            "url": "https://files.pythonhosted.org/packages/f1/8c/98e27bcda23ca46fa5cb78ce524a5d166e073718080554c2780a61e08851/bert_multitask_learning-0.2.9-py3-none-any.whl"
        }
    ]
}