{ "info": { "author": "Cheng Guo", "author_email": "guocheng672@gmail.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "# Transformer Encoder\n

\n \n \n \n \n

\nThis package provides an easy-to-use interface of transformer encoder.\n\n## Installation\n\nRequirements: `python(>=3.5)`, `pytorch(>=1.0.0)`\n\nInstall from pypi:\n```\npip install tfencoder\n```\nOr from Github for the latest version:\n```\npip install git+https://github.com/guocheng2018/transformer-encoder.git\n```\n\n## Usage\n\n*TFEncoder(n_layers, d_model, d_ff, n_heads, dropout)*\n\n- `n_layers`: number of stacked layers of encoder\n- `d_model`: dimension of each word vector\n- `d_ff`: hidden dimension of feed forward layer\n- `n_heads`: number of heads in self-attention\n- `dropout`: dropout rate, default 0.1\n\n*TFEncoder.forward(x, mask)*\n\n- `x (~torch.FloatTensor)`: shape *(batch_size, max_seq_len, d_model)*\n- `mask (~torch.ByteTensor)`: shape *(batch_size, max_seq_len)*\n\nExample:\n```python\nimport torch\nimport tfencoder\n\nencoder = tfencoder.TFEncoder(6, 512, 2048, 8, dropout=0.1)\n\nx = torch.randn(64, 100, 512) # (batch_size, max_seq_len, d_model)\nmask = torch.randn(64, 100).ge(0) # a random mask\n\nout = encoder(x, mask)\n```\n\n**This package also provides the embedding, positional encoding and scheduled optimizer that are used in transformer as extra functionalities.**\n\n*TFEmbedding(d_model, n_vocab)*\n\n- `d_model`: same as TFEncoder\n- `n_vocab`: vocabulary size\n\n*TFEmbedding.forward(x)*\n\n- `x (~torch.LongTensor)`: shape *(batch_size, max_seq_len)*\n\n*TFPositionalEncoding(d_model, dropout, max_len)*\n\n- `d_model`: same as TFEncoder\n- `dropout`: dropout rate\n- `max_len`: max sequence length\n\n*TFPositionalEncoding.forward(x)*\n\n- `x (~torch.FloatTensor)`: shape *(batch_size, max_seq_len, d_model)*\n\nExample:\n```python\nimport torch\nimport torch.nn as nn\n\nfrom tfencoder.utils import TFEmbedding, TFPositionalEncoding\n\ntfembed = TFEmbedding(512, 6)\ntfpe = TFPositionalEncoding(512, 0.1, max_len=5)\ntfinput = nn.Sequential(tfembed, tfpe)\n\nx = torch.LongTensor([[1,2,3,4,5], [1,2,3,0,0]])\nout = tfinput(x)\n```\n\n*TFOptimizer(d_model, factor, warmup, optimizer)*\n\n- `d_model`: equals d_model in TFEncoder\n- `factor`: scale factor of learning rate\n- `warmup`: warmup steps \n- `optimizer (~torch.optim.Optimzier)`: e.g. adam optimzier\n\nExample:\n```python\nimport torch.optim as optim\n\nfrom tfencoder import TFEncoder\nfrom tfencoder.utils import TFOptimizer\n\nencoder = TFEncoder(6, 512, 2048, 8, dropout=0.1)\noptimizer = TFOptimizer(512, 1, 1000, optim.Adam(encoder.parameters(), lr=0))\n\noptimizer.zero_grad()\n\nloss = ...\nloss.backward()\n\noptimizer.step()\n```\n\n## Contribution\nAny contributions are welcome!\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/guocheng2018/transformer-encoder", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "tfencoder", "package_url": "https://pypi.org/project/tfencoder/", "platform": "", "project_url": "https://pypi.org/project/tfencoder/", "project_urls": { "Homepage": "https://github.com/guocheng2018/transformer-encoder" }, "release_url": "https://pypi.org/project/tfencoder/0.0.2/", "requires_dist": null, "requires_python": "", "summary": "A pytorch implementation of transformer encoder", "version": "0.0.2" }, "last_serial": 5653958, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "69ec58bfc37d4c4f294b02e2ea68427c", "sha256": "086608b137ca6346496b7d4d8c91d841b47dc023c17a2febc28d66d41cce662b" }, "downloads": -1, "filename": "tfencoder-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "69ec58bfc37d4c4f294b02e2ea68427c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9213, "upload_time": "2019-07-25T06:06:21", "url": "https://files.pythonhosted.org/packages/b5/17/8c10a4043add146ba9d59990a3cbda752380afa97799f9cf0e3bcfa9bebf/tfencoder-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b3cf435d85e49bab95bce13abbaaaeb1", "sha256": "2110acb1f14569d31c5d0d3d2a3b36d1cbc94c6e854075178b8bf2da3163aef2" }, "downloads": -1, "filename": "tfencoder-0.0.1.tar.gz", "has_sig": false, "md5_digest": "b3cf435d85e49bab95bce13abbaaaeb1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5881, "upload_time": "2019-07-25T06:06:23", "url": "https://files.pythonhosted.org/packages/80/25/e87c2dd35ea611a227343033a32bc5dcc6ecde6a64dd02179703d2c0436c/tfencoder-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "82d9db5be201d9faf51762f18cd3e49e", "sha256": "0186397b583a69df5e47c59e0d80a4734096a1c4ee2ac51fe662dbd4e3928b1d" }, "downloads": -1, "filename": "tfencoder-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "82d9db5be201d9faf51762f18cd3e49e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9176, "upload_time": "2019-08-09T07:10:00", "url": "https://files.pythonhosted.org/packages/40/f9/1478a0dfe25867d477008d154311e0e84c35869e2e8ef0a1c8bebc29b273/tfencoder-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "96bebbd4eb51897ab9a14a9988a3e918", "sha256": "5c3c2786da7af406e9416fa3e5f67c7b0061686cc198ee70039e5015ab9b022b" }, "downloads": -1, "filename": "tfencoder-0.0.2.tar.gz", "has_sig": false, "md5_digest": "96bebbd4eb51897ab9a14a9988a3e918", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5855, "upload_time": "2019-08-09T07:10:02", "url": "https://files.pythonhosted.org/packages/bf/50/93b15aaff3cc12e11880746cf39f50b5fe5bfd993ec3ebd2aa7ab6ab08f9/tfencoder-0.0.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "82d9db5be201d9faf51762f18cd3e49e", "sha256": "0186397b583a69df5e47c59e0d80a4734096a1c4ee2ac51fe662dbd4e3928b1d" }, "downloads": -1, "filename": "tfencoder-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "82d9db5be201d9faf51762f18cd3e49e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9176, "upload_time": "2019-08-09T07:10:00", "url": "https://files.pythonhosted.org/packages/40/f9/1478a0dfe25867d477008d154311e0e84c35869e2e8ef0a1c8bebc29b273/tfencoder-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "96bebbd4eb51897ab9a14a9988a3e918", "sha256": "5c3c2786da7af406e9416fa3e5f67c7b0061686cc198ee70039e5015ab9b022b" }, "downloads": -1, "filename": "tfencoder-0.0.2.tar.gz", "has_sig": false, "md5_digest": "96bebbd4eb51897ab9a14a9988a3e918", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5855, "upload_time": "2019-08-09T07:10:02", "url": "https://files.pythonhosted.org/packages/bf/50/93b15aaff3cc12e11880746cf39f50b5fe5bfd993ec3ebd2aa7ab6ab08f9/tfencoder-0.0.2.tar.gz" } ] }