{
    "info": {
        "author": "CyberZHG",
        "author_email": "CyberZHG@users.noreply.github.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3"
        ],
        "description": "# Keras XLNet\n\n[![Travis](https://travis-ci.org/CyberZHG/keras-xlnet.svg)](https://travis-ci.org/CyberZHG/keras-xlnet)\n[![Coverage](https://coveralls.io/repos/github/CyberZHG/keras-xlnet/badge.svg?branch=master)](https://coveralls.io/github/CyberZHG/keras-xlnet)\n[![Version](https://img.shields.io/pypi/v/keras-xlnet.svg)](https://pypi.org/project/keras-xlnet/)\n![Downloads](https://img.shields.io/pypi/dm/keras-xlnet.svg)\n![License](https://img.shields.io/pypi/l/keras-xlnet.svg)\n\n![](https://img.shields.io/badge/keras-tensorflow-blue.svg)\n![](https://img.shields.io/badge/keras-tf.keras-blue.svg)\n![](https://img.shields.io/badge/keras-tf.keras/eager-blue.svg)\n![](https://img.shields.io/badge/keras-tf.keras/2.0_beta-blue.svg)\n\n\\[[\u4e2d\u6587](https://github.com/CyberZHG/keras-xlnet/blob/master/README.zh-CN.md)|[English](https://github.com/CyberZHG/keras-xlnet/blob/master/README.md)\\]\n\nUnofficial implementation of [XLNet](https://arxiv.org/pdf/1906.08237). [Embedding extraction](demo/extract/token_embeddings.py) and [embedding extract with memory](demo/extract/token_embeddings_with_memory.py) show how to get the outputs of the last transformer layer using pre-trained checkpoints.\n\n## Install\n\n```bash\npip install keras-xlnet\n```\n\n## Usage\n\n### Fine-tuning on GLUE\n\nClick the task name to see the demos with base model:\n\n|Task Name                       |Metrics                       |Approximate Results on Dev Set|\n|:-------------------------------|:----------------------------:|----:|\n|[CoLA](demo/GLUE/CoLA/cola.py)  |Matthew Corr.                 |52   |\n|[SST-2](demo/GLUE/SST-2/sst2.py)|Accuracy                      |93   |\n|[MRPC](demo/GLUE/MRPC/mrpc.py)  |Accuracy/F1                   |86/89|\n|[STS-B](demo/GLUE/STS-B/stsb.py)|Pearson Corr. / Spearman Corr.|86/87|\n|[QQP](demo/GLUE/QQP/qqp.py)     |Accuracy/F1                   |90/86|\n|[MNLI](demo/GLUE/MNLI/mnli.py)  |Accuracy                      |84/84|\n|[QNLI](demo/GLUE/QNLI/qnli.py)  |Accuracy                      |86   |\n|[RTE](demo/GLUE/RTE/rte.py)     |Accuracy                      |64   |\n|[WNLI](demo/GLUE/WNLI/wnli.py)  |Accuracy                      |56   |\n\n(Only 0s are predicted in WNLI dataset)\n\n### Load Pretrained Checkpoints\n\n```python\nimport os\nfrom keras_xlnet import Tokenizer, load_trained_model_from_checkpoint, ATTENTION_TYPE_BI\n\ncheckpoint_path = '.../xlnet_cased_L-24_H-1024_A-16'\n\ntokenizer = Tokenizer(os.path.join(checkpoint_path, 'spiece.model'))\nmodel = load_trained_model_from_checkpoint(\n    config_path=os.path.join(checkpoint_path, 'xlnet_config.json'),\n    checkpoint_path=os.path.join(checkpoint_path, 'xlnet_model.ckpt'),\n    batch_size=16,\n    memory_len=512,\n    target_len=128,\n    in_train_phase=False,\n    attention_type=ATTENTION_TYPE_BI,\n)\nmodel.summary()\n```\n\nArguments `batch_size`, `memory_len` and `target_len` are maximum sizes used for initialization of memories. The model used for training a language model is returned if `in_train_phase` is `True`, otherwise a model used for fine-tuning will be returned.\n\n### About I/O\n\n**Note that** `shuffle` should be `False` in either `fit` or `fit_generator` if memories are used. \n\n#### `in_train_phase` is `False`\n\n3 inputs:\n\n* IDs of tokens, with shape `(batch_size, target_len)`.\n* IDs of segments, with shape `(batch_size, target_len)`.\n* Length of memories, with shape `(batch_size, 1)`.\n\n1 output:\n\n* The feature for each token, with shape `(batch_size, target_len, units)`.\n\n#### `in_train_phase` is `True`\n\n4 inputs:\n\n* IDs of tokens, with shape `(batch_size, target_len)`.\n* IDs of segments, with shape `(batch_size, target_len)`.\n* Length of memories, with shape `(batch_size, 1)`.\n* Masks of tokens, with shape `(batch_size, target_len)`.\n\n1 output:\n\n* The probability of each token in each position, with shape `(batch_size, target_len, num_token)`.",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/CyberZHG/keras-xlnet",
        "keywords": "",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "keras-xlnet",
        "package_url": "https://pypi.org/project/keras-xlnet/",
        "platform": "",
        "project_url": "https://pypi.org/project/keras-xlnet/",
        "project_urls": {
            "Homepage": "https://github.com/CyberZHG/keras-xlnet"
        },
        "release_url": "https://pypi.org/project/keras-xlnet/0.17.0/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "XLNet implemented in Keras",
        "version": "0.17.0"
    },
    "last_serial": 5773720,
    "releases": {
        "0.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2e584d094544f262494ad5688295bd87",
                    "sha256": "b4b7b35d0e366c7c250bbd6cd4d2b48c5dd89e8c995dd553fdb12c356c6fb4ae"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "2e584d094544f262494ad5688295bd87",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 11067,
                "upload_time": "2019-07-15T12:06:50",
                "url": "https://files.pythonhosted.org/packages/74/a7/0cd8cccf6aa567b35af17412c777d0e84bd28d0278b3a470ad300ad0a7cc/keras-xlnet-0.0.0.tar.gz"
            }
        ],
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a9f69c94594597f3411fb38e53cfaadf",
                    "sha256": "347f0ea0476c86c880aceec8e458e1e4dcd31509bcf2ec3805d8e1e4f6ec5ec6"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "a9f69c94594597f3411fb38e53cfaadf",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 11891,
                "upload_time": "2019-07-15T12:56:25",
                "url": "https://files.pythonhosted.org/packages/d2/06/1b9456eea81eea9aafa8a2de022dc0bbf15e44863c4f2c9226d7c3b911dc/keras-xlnet-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e139d8f398d8d49646875b82ed1e3fa4",
                    "sha256": "181bd0ef2b5c19a4366b96b005a3ef5b173ab782eb13a907a56f0473530e091a"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "e139d8f398d8d49646875b82ed1e3fa4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10958,
                "upload_time": "2019-07-15T16:53:21",
                "url": "https://files.pythonhosted.org/packages/0a/dd/65c9a7038c443ada00f2fabd6aca0e90d5c5602bd465772439b54fc43c72/keras-xlnet-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b0ed80e7814f1b3e8640e97c4e7d5132",
                    "sha256": "279921effed38f44e0296acd159e4f06722147fb649882ba44a6da96b034764c"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "b0ed80e7814f1b3e8640e97c4e7d5132",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 11187,
                "upload_time": "2019-07-15T17:20:02",
                "url": "https://files.pythonhosted.org/packages/38/7e/7c6de4f5a31158f422f6b6edf9efec90d3c3d82b4215aecf2bae57ab89bf/keras-xlnet-0.0.3.tar.gz"
            }
        ],
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d0ba65e45c52c0b45d68544bd5a847f0",
                    "sha256": "e45747fc506911b2f6ac6f8eae4af461014da280184bf98d5cd8b2590ab578b1"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "d0ba65e45c52c0b45d68544bd5a847f0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12132,
                "upload_time": "2019-07-16T06:06:42",
                "url": "https://files.pythonhosted.org/packages/c3/15/de2ab08d85e7adfc1dbccc2b7446176a5e563e3d881334902da9e3c3bc66/keras-xlnet-0.1.0.tar.gz"
            }
        ],
        "0.10.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1642d618717b8d4d0368be3bc5c46e6f",
                    "sha256": "c29cec2def16d0025b2489c0abae441d6ce40c4c5d408137ea0eb944105bf391"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.10.0.tar.gz",
                "has_sig": false,
                "md5_digest": "1642d618717b8d4d0368be3bc5c46e6f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14555,
                "upload_time": "2019-07-30T08:22:04",
                "url": "https://files.pythonhosted.org/packages/4a/00/57c49a0eb11e32ab289f6a9c60fbac11e47920b99b14b2e7c9d7d3913615/keras-xlnet-0.10.0.tar.gz"
            }
        ],
        "0.11.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "bad8856ee20cab7988db64112bd0baae",
                    "sha256": "febc697f58973bccb5a8b157c4ce7e3f7dace8e9314e69a80ffeda029f6ad66d"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.11.0.tar.gz",
                "has_sig": false,
                "md5_digest": "bad8856ee20cab7988db64112bd0baae",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14585,
                "upload_time": "2019-07-30T09:37:08",
                "url": "https://files.pythonhosted.org/packages/61/85/d14138183dfbef0bdbd1511f0354d87277e1dd1cb6042e4f04b0cc008c0d/keras-xlnet-0.11.0.tar.gz"
            }
        ],
        "0.13.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "bd25b65a46695a70707ceab24436f686",
                    "sha256": "50b69890cb682445138ac2d5d6c396cfffce057fc54666c55cb6d65165c6fa5e"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.13.0.tar.gz",
                "has_sig": false,
                "md5_digest": "bd25b65a46695a70707ceab24436f686",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 15139,
                "upload_time": "2019-08-05T07:17:12",
                "url": "https://files.pythonhosted.org/packages/c7/c5/989f21dd009e64048bc90e9bd4e9d8c0808593ea865846c9a2ffaa62df89/keras-xlnet-0.13.0.tar.gz"
            }
        ],
        "0.14.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0451e82eb64bf405e6534c9539da8f5b",
                    "sha256": "bea3aabfdb8d5508a98e4dca5dff6a194f4aba36624109c697b52ac806661ad3"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.14.0.tar.gz",
                "has_sig": false,
                "md5_digest": "0451e82eb64bf405e6534c9539da8f5b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20702,
                "upload_time": "2019-08-06T05:00:13",
                "url": "https://files.pythonhosted.org/packages/07/b0/d0b6ff20cdccee614a6ec0f8c6866b516dbb55c84fd75d0869566c16d547/keras-xlnet-0.14.0.tar.gz"
            }
        ],
        "0.14.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ad2740642a5266077a4a1ea75cd41d61",
                    "sha256": "3652564d6af8b3d641f8e1ba1781ca023d84ad48f8341848594bce86738f1636"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.14.1.tar.gz",
                "has_sig": false,
                "md5_digest": "ad2740642a5266077a4a1ea75cd41d61",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20762,
                "upload_time": "2019-08-06T11:16:31",
                "url": "https://files.pythonhosted.org/packages/23/32/b9e7bf730f79613541567c46e4cd4a1395fa8441e6b3c602882ea4612fdb/keras-xlnet-0.14.1.tar.gz"
            }
        ],
        "0.15.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5e1c14ead4492156b2a258204348a30e",
                    "sha256": "a7222c7e1fc114f26ce4406b17eaf179ec6a579297cbfd4f4b9441f0af95199d"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.15.0.tar.gz",
                "has_sig": false,
                "md5_digest": "5e1c14ead4492156b2a258204348a30e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20804,
                "upload_time": "2019-08-23T05:42:40",
                "url": "https://files.pythonhosted.org/packages/30/20/a6590b845ab7dd06ba464ff178181f12b62f53ec9546e9927495a851ae56/keras-xlnet-0.15.0.tar.gz"
            }
        ],
        "0.16.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "57d25c1375c0ac47d2ab7c832705d7d0",
                    "sha256": "188dfc472e8279728ff1a132d6f2c3c244c2a6ecc64bdf38c799cc89790c7d66"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.16.0.tar.gz",
                "has_sig": false,
                "md5_digest": "57d25c1375c0ac47d2ab7c832705d7d0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20879,
                "upload_time": "2019-08-23T06:50:59",
                "url": "https://files.pythonhosted.org/packages/9d/87/8537dcdcac1bb6704c2934b4fe895584270b65f2df563fc4226941940eec/keras-xlnet-0.16.0.tar.gz"
            }
        ],
        "0.17.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a6dfd396061f8e7c7b0c182cee692c15",
                    "sha256": "49ffa66c333019dee18b37e2564067e911b6416add180f8cff657f4deaae10dc"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.17.0.tar.gz",
                "has_sig": false,
                "md5_digest": "a6dfd396061f8e7c7b0c182cee692c15",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 21674,
                "upload_time": "2019-09-03T03:58:47",
                "url": "https://files.pythonhosted.org/packages/38/1a/1f91000643423f2cb97f6b478e0b0c77483e2e10de619a75a13071b2c12b/keras-xlnet-0.17.0.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fa109a2e9e9ff80525c42400a6ab9dfe",
                    "sha256": "9032ab53c5c87597f1cfeddc86345180f125ac33f03cfadbed7653186d68bd1b"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "fa109a2e9e9ff80525c42400a6ab9dfe",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12261,
                "upload_time": "2019-07-18T12:24:49",
                "url": "https://files.pythonhosted.org/packages/10/80/232eccfdfb826332bc1d5acf5fe3583cc4e208d6cfde355e468726336aef/keras-xlnet-0.2.0.tar.gz"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b9104c36a4763e241ff253e6c7142237",
                    "sha256": "057cb4e05068870c1139a2e65937af7b64b37cad52be9bee1611e960ebd9ebee"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "b9104c36a4763e241ff253e6c7142237",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12280,
                "upload_time": "2019-07-18T12:41:45",
                "url": "https://files.pythonhosted.org/packages/10/c8/9359561b25156ead5dcb5cc458ba5b372e03e07f86a87953768510fb97b5/keras-xlnet-0.3.0.tar.gz"
            }
        ],
        "0.4.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "02fb9c169f87ecbbd40c88f02d1bd97b",
                    "sha256": "13327d99d50024e335ffcf369768bd4f152ca4bd146ae5b2c7df9e02a19f4232"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.4.0.tar.gz",
                "has_sig": false,
                "md5_digest": "02fb9c169f87ecbbd40c88f02d1bd97b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12295,
                "upload_time": "2019-07-18T12:47:59",
                "url": "https://files.pythonhosted.org/packages/0b/25/73ca5d6ca2411ee1bbaae356350b34de0bde0e6c96d529ddafde13158d60/keras-xlnet-0.4.0.tar.gz"
            }
        ],
        "0.5.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4f78351497f70105b76e9a5a17875e2e",
                    "sha256": "f0e15692abc5b630d030e797157fe4996477432541b140a70973d0203e58ff5b"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.5.0.tar.gz",
                "has_sig": false,
                "md5_digest": "4f78351497f70105b76e9a5a17875e2e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12215,
                "upload_time": "2019-07-18T14:29:24",
                "url": "https://files.pythonhosted.org/packages/a9/0c/ac0f473f90f7022279e23d1b3936d179b110f4f400d6b0cae841c71640ca/keras-xlnet-0.5.0.tar.gz"
            }
        ],
        "0.7.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "dba10f4955ba2a4cc44955638d249191",
                    "sha256": "253df0accebae046893ce37059a0ebbfaceba08a7a1a36ed1d4366c58037e886"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.7.0.tar.gz",
                "has_sig": false,
                "md5_digest": "dba10f4955ba2a4cc44955638d249191",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14158,
                "upload_time": "2019-07-30T02:27:32",
                "url": "https://files.pythonhosted.org/packages/55/f1/144a62ed197865b728e9f832b7f8b65e4eb7d82ea8e3d160ff6e935c06cc/keras-xlnet-0.7.0.tar.gz"
            }
        ],
        "0.8.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f0b30e7ea0229fe864a7e504df4c8cdb",
                    "sha256": "dadc87b595b8708536985bcc331ef2d7cf4cc2ba579ba057ae4abc6032e547c9"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.8.0.tar.gz",
                "has_sig": false,
                "md5_digest": "f0b30e7ea0229fe864a7e504df4c8cdb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14341,
                "upload_time": "2019-07-30T05:09:02",
                "url": "https://files.pythonhosted.org/packages/01/17/d8ea7e2034bd0a870949b2c892e660441546a68bb9f2b7cc03644b5038ae/keras-xlnet-0.8.0.tar.gz"
            }
        ],
        "0.9.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ce3a801b2f7fb194b8bd2f5a77636547",
                    "sha256": "13f6f96c73774ef7847ea26fa87c3a23c22e141194a822979d03111c47abb573"
                },
                "downloads": -1,
                "filename": "keras-xlnet-0.9.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ce3a801b2f7fb194b8bd2f5a77636547",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14560,
                "upload_time": "2019-07-30T08:17:48",
                "url": "https://files.pythonhosted.org/packages/e9/f1/39b6ace7218cdc618b57c5af42d07031d613ead85a66d238c4a45c09e92c/keras-xlnet-0.9.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "a6dfd396061f8e7c7b0c182cee692c15",
                "sha256": "49ffa66c333019dee18b37e2564067e911b6416add180f8cff657f4deaae10dc"
            },
            "downloads": -1,
            "filename": "keras-xlnet-0.17.0.tar.gz",
            "has_sig": false,
            "md5_digest": "a6dfd396061f8e7c7b0c182cee692c15",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 21674,
            "upload_time": "2019-09-03T03:58:47",
            "url": "https://files.pythonhosted.org/packages/38/1a/1f91000643423f2cb97f6b478e0b0c77483e2e10de619a75a13071b2c12b/keras-xlnet-0.17.0.tar.gz"
        }
    ]
}