{ "info": { "author": "Thilina Rajapakse", "author_email": "chaturangarajapakshe@gmail.com", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Science/Research", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3", "Topic :: Scientific/Engineering :: Artificial Intelligence" ], "description": "[](https://opensource.org/licenses/Apache-2.0) [](https://pepy.tech/project/simpletransformers)\n\n[](#contributors-)\n\n\n# Simple Transformers\n\nThis library is based on the [Transformers](https://github.com/huggingface/transformers) library by HuggingFace. `Simple Transformers` lets you quickly train and evaluate Transformer models. Only 3 lines of code are needed to **initialize**, **train**, and **evaluate** a model.\n\n**Supported Tasks:**\n\n- Sequence Classification\n- Token Classification (NER)\n- Question Answering\n- Language Model Fine-Tuning\n- Language Model Training\n- Language Generation\n- T5 Model\n- Seq2Seq Tasks\n- Multi-Modal Classification\n- Conversational AI.\n- Text Representation Generation.\n\n# Table of contents\n\n\n\n- [Simple Transformers](#simple-transformers)\n- [Table of contents](#table-of-contents)\n - [Setup](#setup)\n - [With Conda](#with-conda)\n - [Optional](#optional)\n - [Usage](#usage)\n - [A quick example](#a-quick-example)\n - [Experiment Tracking with Weights and Biases](#experiment-tracking-with-weights-and-biases)\n - [Current Pretrained Models](#current-pretrained-models)\n - [Contributors \u2728](#contributors-)\n - [How to Contribute](#how-to-contribute)\n - [How to Update Docs](#how-to-update-docs)\n - [Acknowledgements](#acknowledgements)\n\n\n\n## Setup\n\n### With Conda\n\n1. Install `Anaconda` or `Miniconda` Package Manager from [here](https://www.anaconda.com/distribution/)\n2. Create a new virtual environment and install packages.\n\n```bash\n$ conda create -n st python pandas tqdm\n$ conda activate st\n```\n\nUsing Cuda:\n\n```bash\n$ conda install pytorch>=1.6 cudatoolkit=11.0 -c pytorch\n```\n\nWithout using Cuda\n\n```bash\n$ conda install pytorch cpuonly -c pytorch\n```\n\n3. Install `simpletransformers`.\n\n```bash\n$ pip install simpletransformers\n```\n\n#### Optional\n\n1. Install `Weights` and `Biases` (wandb) for tracking and visualizing training in a web browser.\n\n```bash\n$ pip install wandb\n```\n\n## Usage\n\n**All documentation is now live at [simpletransformers.ai](https://simpletransformers.ai/)**\n\n`Simple Transformer` models are built with a particular Natural Language Processing (NLP) task in mind. Each such model comes equipped with features and functionality designed to best fit the task that they are intended to perform. The high-level process of using Simple Transformers models follows the same pattern.\n\n1. Initialize a task-specific model\n2. Train the model with `train_model()`\n3. Evaluate the model with `eval_model()`\n4. Make predictions on (unlabelled) data with `predict()`\n\nHowever, there are necessary differences between the different models to ensure that they are well suited for their intended task. The key differences will typically be the differences in input/output data formats and any task specific features/configuration options. These can all be found in the documentation section for each task.\n\nThe currently implemented task-specific `Simple Transformer` models, along with their task, are given below.\n\n| Task | Model |\n| --------------------------------------------------------- | ------------------------------- |\n| Binary and multi-class text classification | `ClassificationModel` |\n| Conversational AI (chatbot training) | `ConvAIModel` |\n| Language generation | `LanguageGenerationModel` |\n| Language model training/fine-tuning | `LanguageModelingModel` |\n| Multi-label text classification | `MultiLabelClassificationModel` |\n| Multi-modal classification (text and image data combined) | `MultiModalClassificationModel` |\n| Named entity recognition | `NERModel` |\n| Question answering | `QuestionAnsweringModel` |\n| Regression | `ClassificationModel` |\n| Sentence-pair classification | `ClassificationModel` |\n| Text Representation Generation | `RepresentationModel` |\n| Document Retrieval | `RetrievalModel` |\n\n- **Please refer to the relevant section in the [docs](https://simpletransformers.ai/) for more information on how to use these models.**\n- Example scripts can be found in the [examples](https://github.com/ThilinaRajapakse/simpletransformers/tree/master/examples) directory.\n- See the [Changelog](https://github.com/ThilinaRajapakse/simpletransformers/blob/master/CHANGELOG.md) for up-to-date changes to the project.\n\n### A quick example\n\n```python\nfrom simpletransformers.classification import ClassificationModel, ClassificationArgs\nimport pandas as pd\nimport logging\n\n\nlogging.basicConfig(level=logging.INFO)\ntransformers_logger = logging.getLogger(\"transformers\")\ntransformers_logger.setLevel(logging.WARNING)\n\n# Preparing train data\ntrain_data = [\n [\"Aragorn was the heir of Isildur\", 1],\n [\"Frodo was the heir of Isildur\", 0],\n]\ntrain_df = pd.DataFrame(train_data)\ntrain_df.columns = [\"text\", \"labels\"]\n\n# Preparing eval data\neval_data = [\n [\"Theoden was the king of Rohan\", 1],\n [\"Merry was the king of Rohan\", 0],\n]\neval_df = pd.DataFrame(eval_data)\neval_df.columns = [\"text\", \"labels\"]\n\n# Optional model configuration\nmodel_args = ClassificationArgs(num_train_epochs=1)\n\n# Create a ClassificationModel\nmodel = ClassificationModel(\n \"roberta\", \"roberta-base\", args=model_args\n)\n\n# Train the model\nmodel.train_model(train_df)\n\n# Evaluate the model\nresult, model_outputs, wrong_predictions = model.eval_model(eval_df)\n\n# Make predictions with the model\npredictions, raw_outputs = model.predict([\"Sam was a Wizard\"])\n\n```\n\n### Experiment Tracking with Weights and Biases\n\n- Weights and Biases makes it incredibly easy to keep track of all your experiments. Check it out on Colab here: [](http://wandb.me/simpletformers-QA-colab)\n\n\n---\n\n## Current Pretrained Models\n\nFor a list of pretrained models, see [Hugging Face docs](https://huggingface.co/pytorch-transformers/pretrained_models.html).\n\nThe `model_types` available for each task can be found under their respective section. Any pretrained model of that type\nfound in the Hugging Face docs should work. To use any of them set the correct `model_type` and `model_name` in the `args`\ndictionary.\n\n---\n\n## Contributors \u2728\n\nThanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):\n\n\n\n\n