{ "info": { "author": "", "author_email": "", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Scientific/Engineering :: Artificial Intelligence" ], "description": "# Introduction\n\nFairseq(-py) is a sequence modeling toolkit that allows researchers and\ndevelopers to train custom models for translation, summarization, language\nmodeling and other text generation tasks.\n\n### What's New:\n\n- July 2019: fairseq relicensed under MIT license\n- July 2019: [RoBERTa models and code release](examples/roberta/README.md)\n- June 2019: [wav2vec models and code release](examples/wav2vec/README.md)\n- April 2019: [fairseq demo paper @ NAACL 2019](https://arxiv.org/abs/1904.01038)\n\n### Features:\n\nFairseq provides reference implementations of various sequence-to-sequence models, including:\n- **Convolutional Neural Networks (CNN)**\n - [Language Modeling with Gated Convolutional Networks (Dauphin et al., 2017)](examples/language_model/conv_lm/README.md)\n - [Convolutional Sequence to Sequence Learning (Gehring et al., 2017)](examples/conv_seq2seq/README.md)\n - [Classical Structured Prediction Losses for Sequence to Sequence Learning (Edunov et al., 2018)](https://github.com/pytorch/fairseq/tree/classic_seqlevel)\n - [Hierarchical Neural Story Generation (Fan et al., 2018)](examples/stories/README.md)\n - [wav2vec: Unsupervised Pre-training for Speech Recognition (Schneider et al., 2019)](examples/wav2vec/README.md)\n- **LightConv and DynamicConv models**\n - [Pay Less Attention with Lightweight and Dynamic Convolutions (Wu et al., 2019)](examples/pay_less_attention_paper/README.md)\n- **Long Short-Term Memory (LSTM) networks**\n - Effective Approaches to Attention-based Neural Machine Translation (Luong et al., 2015)\n- **Transformer (self-attention) networks**\n - Attention Is All You Need (Vaswani et al., 2017)\n - [Scaling Neural Machine Translation (Ott et al., 2018)](examples/scaling_nmt/README.md)\n - [Understanding Back-Translation at Scale (Edunov et al., 2018)](examples/backtranslation/README.md)\n - [Adaptive Input Representations for Neural Language Modeling (Baevski and Auli, 2018)](examples/language_model/transformer_lm/README.md)\n - [Mixture Models for Diverse Machine Translation: Tricks of the Trade (Shen et al., 2019)](examples/translation_moe/README.md)\n - [RoBERTa: A Robustly Optimized BERT Pretraining Approach (Liu et al., 2019)](examples/roberta/README.md)\n\n**Additionally:**\n- multi-GPU (distributed) training on one machine or across multiple machines\n- fast generation on both CPU and GPU with multiple search algorithms implemented:\n - beam search\n - Diverse Beam Search ([Vijayakumar et al., 2016](https://arxiv.org/abs/1610.02424))\n - sampling (unconstrained, top-k and top-p/nucleus)\n- large mini-batch training even on a single GPU via delayed updates\n- mixed precision training (trains faster with less GPU memory on [NVIDIA tensor cores](https://developer.nvidia.com/tensor-cores))\n- extensible: easily register new models, criterions, tasks, optimizers and learning rate schedulers\n\nWe also provide [pre-trained models](#pre-trained-models-and-examples) for several benchmark\ntranslation and language modeling datasets.\n\n![Model](fairseq.gif)\n\n# Requirements and Installation\n\n* [PyTorch](http://pytorch.org/) version >= 1.0.0\n* Python version >= 3.5\n* For training new models, you'll also need an NVIDIA GPU and [NCCL](https://github.com/NVIDIA/nccl)\n\nPlease follow the instructions here to install PyTorch: https://github.com/pytorch/pytorch#installation.\n\nIf you use Docker make sure to increase the shared memory size either with\n`--ipc=host` or `--shm-size` as command line options to `nvidia-docker run`.\n\nAfter PyTorch is installed, you can install fairseq with `pip`:\n```\npip install fairseq\n```\nOn MacOS,\n```\nCFLAGS=\"-stdlib=libc++\" pip install fairseq\n```\n**Installing from source**\n\nTo install fairseq from source and develop locally:\n```\ngit clone https://github.com/pytorch/fairseq\ncd fairseq\npip install --editable .\n```\n\n**Improved training speed**\n\nTraining speed can be further improved by installing NVIDIA's\n[apex](https://github.com/NVIDIA/apex) library with the `--cuda_ext` option.\nfairseq will automatically switch to the faster modules provided by apex.\n\n# Getting Started\n\nThe [full documentation](https://fairseq.readthedocs.io/) contains instructions\nfor getting started, training new models and extending fairseq with new model\ntypes and tasks.\n\n# Pre-trained models and examples\n\nWe provide pre-trained models and pre-processed, binarized test sets for several tasks listed below,\nas well as example training and evaluation commands.\n\n- [Translation](examples/translation/README.md): convolutional and transformer models are available\n- [Language Modeling](examples/language_model/README.md): convolutional models are available\n\nWe also have more detailed READMEs to reproduce results from specific papers:\n- [RoBERTa: A Robustly Optimized BERT Pretraining Approach (Liu et al., 2019)](examples/roberta/README.md)\n- [wav2vec: Unsupervised Pre-training for Speech Recognition (Schneider et al., 2019)](examples/wav2vec/README.md)\n- [Mixture Models for Diverse Machine Translation: Tricks of the Trade (Shen et al., 2019)](examples/translation_moe/README.md)\n- [Pay Less Attention with Lightweight and Dynamic Convolutions (Wu et al., 2019)](examples/pay_less_attention_paper/README.md)\n- [Understanding Back-Translation at Scale (Edunov et al., 2018)](examples/backtranslation/README.md)\n- [Classical Structured Prediction Losses for Sequence to Sequence Learning (Edunov et al., 2018)](https://github.com/pytorch/fairseq/tree/classic_seqlevel)\n- [Hierarchical Neural Story Generation (Fan et al., 2018)](examples/stories/README.md)\n- [Scaling Neural Machine Translation (Ott et al., 2018)](examples/scaling_nmt/README.md)\n- [Convolutional Sequence to Sequence Learning (Gehring et al., 2017)](examples/conv_seq2seq/README.md)\n- [Language Modeling with Gated Convolutional Networks (Dauphin et al., 2017)](examples/language_model/conv_lm/README.md)\n\n# Join the fairseq community\n\n* Facebook page: https://www.facebook.com/groups/fairseq.users\n* Google group: https://groups.google.com/forum/#!forum/fairseq-users\n\n# License\nfairseq(-py) is MIT-licensed.\nThe license applies to the pre-trained models as well.\n\n# Citation\n\nPlease cite as:\n\n```bibtex\n@inproceedings{ott2019fairseq,\n title = {fairseq: A Fast, Extensible Toolkit for Sequence Modeling},\n author = {Myle Ott and Sergey Edunov and Alexei Baevski and Angela Fan and Sam Gross and Nathan Ng and David Grangier and Michael Auli},\n booktitle = {Proceedings of NAACL-HLT 2019: Demonstrations},\n year = {2019},\n}\n```", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/pytorch/fairseq", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "fairseq", "package_url": "https://pypi.org/project/fairseq/", "platform": "", "project_url": "https://pypi.org/project/fairseq/", "project_urls": { "Homepage": "https://github.com/pytorch/fairseq" }, "release_url": "https://pypi.org/project/fairseq/0.8.0/", "requires_dist": null, "requires_python": "", "summary": "Facebook AI Research Sequence-to-Sequence Toolkit", "version": "0.8.0" }, "last_serial": 5676778, "releases": { "0.6.1": [ { "comment_text": "", "digests": { "md5": "42fa1c58ff11a36f9ff500e0f08909ce", "sha256": "7133d1b163805535e13e91068c03c4b9cddec327a8748df27d90560215f3f5a6" }, "downloads": -1, "filename": "fairseq-0.6.1.tar.gz", "has_sig": false, "md5_digest": "42fa1c58ff11a36f9ff500e0f08909ce", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 132864, "upload_time": "2019-02-09T05:46:51", "url": "https://files.pythonhosted.org/packages/cb/56/2032410e608c310418fbe6be6a89cd23d1b0e392872063d5464611b63267/fairseq-0.6.1.tar.gz" } ], "0.6.2": [ { "comment_text": "", "digests": { "md5": "a52ca46bd35d83e81a8e2e7bed0045d8", "sha256": "4ef01eb5f4e1731e16ac3ea06122ca1526f768ab46a218a20fd972687f46cbd7" }, "downloads": -1, "filename": "fairseq-0.6.2.tar.gz", "has_sig": false, "md5_digest": "a52ca46bd35d83e81a8e2e7bed0045d8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 139260, "upload_time": "2019-03-15T17:36:05", "url": "https://files.pythonhosted.org/packages/d7/b1/bb10d50b5fb0a030ddbeb049c8df66137f3a137e42d6d2474e73aa878a51/fairseq-0.6.2.tar.gz" } ], "0.7.1": [ { "comment_text": "", "digests": { "md5": "84f1fb0a46258991bfd0cf2cbeac0bf1", "sha256": "559116342b3c11f948ea29eea0d35a82668351c83c058d77800bb88aa6151842" }, "downloads": -1, "filename": "fairseq-0.7.1.tar.gz", "has_sig": false, "md5_digest": "84f1fb0a46258991bfd0cf2cbeac0bf1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 186342, "upload_time": "2019-06-20T15:21:16", "url": "https://files.pythonhosted.org/packages/97/b4/b37d9ef01891ed1884f7d969d6b2ba4ac3e1adcada34dda3e39c4caac9b9/fairseq-0.7.1.tar.gz" } ], "0.7.2": [ { "comment_text": "", "digests": { "md5": "4c3624eeb4a45027975d06b9a3bc59ee", "sha256": "66675264017ed345da5d085325189705f091a3da83896049d4f37508c6d33426" }, "downloads": -1, "filename": "fairseq-0.7.2.tar.gz", "has_sig": false, "md5_digest": "4c3624eeb4a45027975d06b9a3bc59ee", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 190759, "upload_time": "2019-07-19T13:43:26", "url": "https://files.pythonhosted.org/packages/1c/13/41fb03306f9e50581210d2fb24f2f056f700f9ffdddb0f7734c7bda5d715/fairseq-0.7.2.tar.gz" } ], "0.8.0": [ { "comment_text": "", "digests": { "md5": "b3fcd8aeb7a7420636040b54fd2c8d47", "sha256": "55dfedb630ba76ac6c25bf443fd624ce78e097b4820fdc3b1c6d2b153e9f212e" }, "downloads": -1, "filename": "fairseq-0.8.0.tar.gz", "has_sig": false, "md5_digest": "b3fcd8aeb7a7420636040b54fd2c8d47", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 216335, "upload_time": "2019-08-14T12:35:17", "url": "https://files.pythonhosted.org/packages/62/19/a71af3ea3bdf7c2fd66c8076a3be090147f700e3e513b0b3b11d80d97fe3/fairseq-0.8.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "b3fcd8aeb7a7420636040b54fd2c8d47", "sha256": "55dfedb630ba76ac6c25bf443fd624ce78e097b4820fdc3b1c6d2b153e9f212e" }, "downloads": -1, "filename": "fairseq-0.8.0.tar.gz", "has_sig": false, "md5_digest": "b3fcd8aeb7a7420636040b54fd2c8d47", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 216335, "upload_time": "2019-08-14T12:35:17", "url": "https://files.pythonhosted.org/packages/62/19/a71af3ea3bdf7c2fd66c8076a3be090147f700e3e513b0b3b11d80d97fe3/fairseq-0.8.0.tar.gz" } ] }