{ "info": { "author": "", "author_email": "", "bugtrack_url": null, "classifiers": [], "description": "Deepvoice3\\_pytorch\n===================\n\n|Build Status|\n\nPyTorch implementation of convolutional networks-based text-to-speech\nsynthesis models:\n\n1. `arXiv:1710.07654 `__: Deep Voice\n 3: 2000-Speaker Neural Text-to-Speech.\n2. `arXiv:1710.08969 `__: Efficiently\n Trainable Text-to-Speech System Based on Deep Convolutional Networks\n with Guided Attention.\n\nAudio sampels are available at\nhttps://r9y9.github.io/deepvoice3\\_pytorch/.\n\nHighlights\n----------\n\n- Convolutional sequence-to-sequence model with attention for\n text-to-speech synthesis\n- Multi-speaker and single speaker versions of DeepVoice3\n- Audio samples and pre-trained models\n- Preprocessor for `LJSpeech\n (en) `__, `JSUT\n (jp) `__\n and\n `VCTK `__\n datasets\n- Language-dependent frontend text processor for English and Japanese\n\nPretrained models\n-----------------\n\n+-----+----------+---------+----------------------------------+----------------+-------+\n| URL | Model | Data | Hyper paramters | Git commit | Steps |\n+=====+==========+=========+==================================+================+=======+\n| `li | DeepVoic | LJSpeec | ``builder=deepvoice3,preset=deep | `4357976 `__ | |\n| ox. | | | | | |\n| com | | | | | |\n| /s/ | | | | | |\n| cs6 | | | | | |\n| d07 | | | | | |\n| 0om | | | | | |\n| my2 | | | | | |\n| lmh | | | | | |\n| /20 | | | | | |\n| 171 | | | | | |\n| 213 | | | | | |\n| _de | | | | | |\n| epv | | | | | |\n| oic | | | | | |\n| e3_ | | | | | |\n| che | | | | | |\n| ckp | | | | | |\n| oin | | | | | |\n| t_s | | | | | |\n| tep | | | | | |\n| 000 | | | | | |\n| 210 | | | | | |\n| 000 | | | | | |\n| .pt | | | | | |\n| h?d | | | | | |\n| l=0 | | | | | |\n| >`_ | | | | | |\n| _ | | | | | |\n+-----+----------+---------+----------------------------------+----------------+-------+\n| `li | Nyanko | LJSpeec | ``builder=nyanko,preset=nyanko_l | `ba59dc7 `__ | |\n| ox. | | | | | |\n| com | | | | | |\n| /s/ | | | | | |\n| 1y8 | | | | | |\n| bt6 | | | | | |\n| bng | | | | | |\n| gbz | | | | | |\n| zlp | | | | | |\n| /20 | | | | | |\n| 171 | | | | | |\n| 129 | | | | | |\n| _ny | | | | | |\n| ank | | | | | |\n| o_c | | | | | |\n| hec | | | | | |\n| kpo | | | | | |\n| int | | | | | |\n| _st | | | | | |\n| ep0 | | | | | |\n| 005 | | | | | |\n| 850 | | | | | |\n| 00. | | | | | |\n| pth | | | | | |\n| ?dl | | | | | |\n| =0> | | | | | |\n| `__ | | | | | |\n+-----+----------+---------+----------------------------------+----------------+-------+\n| `li | Multi-sp | VCTK | ``builder=deepvoice3_vctk,preset | `0421749 `__ | |\n| ox. | | | | | |\n| com | | | | | |\n| /s/ | | | | | |\n| uzm | | | | | |\n| tzg | | | | | |\n| ced | | | | | |\n| yu5 | | | | | |\n| 31k | | | | | |\n| /20 | | | | | |\n| 171 | | | | | |\n| 222 | | | | | |\n| _de | | | | | |\n| epv | | | | | |\n| oic | | | | | |\n| e3_ | | | | | |\n| vct | | | | | |\n| k10 | | | | | |\n| 8_c | | | | | |\n| hec | | | | | |\n| kpo | | | | | |\n| int | | | | | |\n| _st | | | | | |\n| ep0 | | | | | |\n| 003 | | | | | |\n| 000 | | | | | |\n| 00. | | | | | |\n| pth | | | | | |\n| ?dl | | | | | |\n| =0> | | | | | |\n| `__ | | | | | |\n+-----+----------+---------+----------------------------------+----------------+-------+\n\nSee \"Synthesize from a checkpoint\" section in the README for how to\ngenerate speech samples. Please make sure that you are on the specific\ngit commit noted above.\n\nNotes on hyper parameters\n-------------------------\n\n- Default hyper parameters, used during\n preprocessing/training/synthesis stages, are turned for English TTS\n using LJSpeech dataset. You will have to change some of parameters if\n you want to try other datasets. See ``hparams.py`` for details.\n- ``builder`` specifies which model you want to use. ``deepvoice3``,\n ``deepvoice3_multispeaker`` [1] and ``nyanko`` [2] are surpprted.\n- ``presets`` represents hyper parameters known to work well for\n particular dataset/model from my experiments. Before you try to find\n your best parameters, I would recommend you to try those presets by\n setting ``preset=${name}``. e.g., for LJSpeech, you can try either\n\n ::\n\n python train.py --data-root=./data/ljspeech --checkpoint-dir=checkpoints_deepvoice3 \\\n --hparams=\"builder=deepvoice3,preset=deepvoice3_ljspeech\" \\\n --log-event-path=log/deepvoice3_preset\n\n or\n\n ::\n\n python train.py --data-root=./data/ljspeech --checkpoint-dir=checkpoints_nyanko \\\n --hparams=\"builder=nyanko,preset=nyanko_ljspeech\" \\\n --log-event-path=log/nyanko_preset\n\n- Hyper parameters described in DeepVoice3 paper for single speaker\n didn't work for LJSpeech dataset, so I changed a few things. Add\n dilated convolution, more channels, more layers and add guided\n attention loss, etc. See code for details. The changes are also\n applied for multi-speaker model.\n- Multiple attention layers are hard to learn. Empirically, one or two\n (first and last) attention layers seems enough.\n- With guided attention (see https://arxiv.org/abs/1710.08969),\n alignments get monotonic more quickly and reliably if we use multiple\n attention layers. With guided attention, I can confirm five attention\n layers get monotonic, though I cannot get speech quality\n improvements.\n- Binary divergence (described in https://arxiv.org/abs/1710.08969)\n seems stabilizes training particularly for deep (> 10 layers)\n networks.\n- Adam with step lr decay works. However, for deeper networks, I find\n Adam + noam's lr scheduler is more stable.\n\nRequirements\n------------\n\n- Python 3\n- PyTorch >= v0.3\n- TensorFlow >= v1.3\n- `tensorboard-pytorch `__\n (master)\n- `nnmnkwii `__ >= v0.0.11\n- `MeCab `__ (Japanese only)\n\nInstallation\n------------\n\nPlease install packages listed above first, and then\n\n::\n\n git clone https://github.com/r9y9/deepvoice3_pytorch\n pip install -e \".[train]\"\n\nIf you want Japanese text processing frontend, install additional\ndependencies by:\n\n::\n\n pip install -e \".[jp]\"\n\nGetting started\n---------------\n\n0. Download dataset\n~~~~~~~~~~~~~~~~~~~\n\n- LJSpeech (en): https://keithito.com/LJ-Speech-Dataset/\n- VCTK (en):\n http://homepages.inf.ed.ac.uk/jyamagis/page3/page58/page58.html\n- JSUT (jp):\n https://sites.google.com/site/shinnosuketakamichi/publication/jsut\n\n1. Preprocessing\n~~~~~~~~~~~~~~~~\n\nPreprocessing can be done by ``preprocess.py``. Usage is:\n\n::\n\n python preprocess.py ${dataset_name} ${dataset_path} ${out_dir}\n\nSupported ``${dataset_name}``\\ s for now are\n\n- ``ljspeech`` (en, single speaker)\n- ``vctk`` (en, multi-speaker)\n- ``jsut`` (jp, single speaker)\n\nSuppose you will want to preprocess LJSpeech dataset and have it in\n``~/data/LJSpeech-1.0``, then you can preprocess data by:\n\n::\n\n python preprocess.py ljspeech ~/data/LJSpeech-1.0/ ./data/ljspeech\n\nWhen this is done, you will see extracted features (mel-spectrograms and\nlinear spectrograms) in ``./data/ljspeech``.\n\n2. Training\n~~~~~~~~~~~\n\nBasic usage of ``train.py`` is:\n\n::\n\n python train.py --data-root=${data-root} --hparams=\"parameters you want to override\"\n\nSuppose you will want to build a DeepVoice3-style model using LJSpeech\ndataset with default hyper parameters, then you can train your model by:\n\n::\n\n python train.py --data-root=./data/ljspeech/ --hparams=\"builder=deepvoice3,preset=deepvoice3_ljspeech\"\n\nModel checkpoints (.pth) and alignments (.png) are saved in\n``./checkpoints`` directory per 5000 steps by default.\n\nIf you are building a Japaneses TTS model, then for example,\n\n::\n\n python train.py --data-root=./data/jsut --hparams=\"frontend=jp\" --hparams=\"builder=deepvoice3,preset=deepvoice3_ljspeech\"\n\n``frontend=jp`` tell the training script to use Japanese text processing\nfrontend. Default is ``en`` and uses English text processing frontend.\n\nNote that there are many hyper parameters and design choices. Some are\nconfigurable by ``hparams.py`` and some are hardcoded in the source\n(e.g., dilation factor for each convolution layer). If you find better\nhyper parameters, please let me know!\n\n4. Moniter with Tensorboard\n~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nLogs are dumped in ``./log`` directory by default. You can monitor logs\nby tensorboard:\n\n::\n\n tensorboard --logdir=log\n\n5. Synthesize from a checkpoint\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nGiven a list of text, ``synthesis.py`` synthesize audio signals from\ntrained model. Usage is:\n\n::\n\n python synthesis.py ${checkpoint_path} ${text_list.txt} ${output_dir}\n\nExample test\\_list.txt:\n\n::\n\n Generative adversarial network or variational auto-encoder.\n Once upon a time there was a dear little girl who was loved by every one who looked at her, but most of all by her grandmother, and there was nothing that she would not have given to the child.\n A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module.\n\nAdvanced usage\n--------------\n\nMulti-speaker model\n~~~~~~~~~~~~~~~~~~~\n\nCurrently VCTK is the only supported dataset for building a\nmulti-speaker model. Since some audio samples in VCTK have long silences\nthat affect performance, it's recommended to do phoneme alignment and\nremove silences according to `vctk\\_preprocess `__.\n\nOnce you have phoneme alignment for each utterance, you can extract\nfeatures by:\n\n::\n\n python preprocess.py vctk ${your_vctk_root_path} ./data/vctk\n\nNow that you have data prepared, then you can train a multi-speaker\nversion of DeepVoice3 by:\n\n::\n\n python train.py --data-root=./data/vctk --checkpoint-dir=checkpoints_vctk \\\n --hparams=\"preset=deepvoice3_vctk,builder=deepvoice3_multispeaker\" \\\n --log-event-path=log/deepvoice3_multispeaker_vctk_preset\n\nIf you want to reuse learned embedding from other dataset, then you can\ndo this instead by:\n\n::\n\n python train.py --data-root=./data/vctk --checkpoint-dir=checkpoints_vctk \\\n --hparams=\"preset=deepvoice3_vctk,builder=deepvoice3_multispeaker\" \\\n --log-event-path=log/deepvoice3_multispeaker_vctk_preset \\\n --load-embedding=20171213_deepvoice3_checkpoint_step000210000.pth\n\nThis may improve training speed a bit.\n\nSpeaker adaptation\n~~~~~~~~~~~~~~~~~~\n\nIf you have very limited data, then you can consider to try fine-turn\npre-trained model. For example, using pre-trained model on LJSpeech, you\ncan adapt it to data from VCTK speaker ``p225`` (30 mins) by the\nfollowing command:\n\n::\n\n python train.py --data-root=./data/vctk --checkpoint-dir=checkpoints_vctk_adaptation \\\n --hparams=\"builder=deepvoice3,preset=deepvoice3_ljspeech\" \\\n --log-event-path=log/deepvoice3_vctk_adaptation \\\n --restore-parts=\"20171213_deepvoice3_checkpoint_step000210000.pth\"\n --speaker-id=0\n\nFrom my experience, it can get reasonable speech quality very quickly\nrather than training the model from scratch.\n\nThere are two important options used above:\n\n- ``--restore-parts=``: It specifies where to load model parameters.\n The differences from the option ``--checkpoint=`` are 1)\n ``--restore-parts=`` ignores all invalid parameters, while\n ``--checkpoint=`` doesn't. 2) ``--restore-parts=`` tell trainer\n to start from 0-step, while ``--checkpoint=`` tell trainer to\n continue from last step. ``--checkpoint=`` should be ok if you are\n using exactly same model and continue to train, but it would be\n useful if you want to customize your model architecture and take\n advantages of pre-trained model.\n- ``--speaker-id=``: It specifies what speaker of data is used for\n training. This should only be specified if you are using\n multi-speaker dataset. As for VCTK, speaker id is automatically\n assigned incrementally (0, 1, ..., 107) according to the\n ``speaker_info.txt`` in the dataset.\n\nAcknowledgements\n----------------\n\nPart of code was adapted from the following projects:\n\n- https://github.com/keithito/tacotron\n- https://github.com/facebookresearch/fairseq-py\n\n.. |Build Status| image:: https://travis-ci.org/r9y9/deepvoice3_pytorch.svg?branch=master\n :target: https://travis-ci.org/r9y9/deepvoice3_pytorch", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "deepvoice3_pytorch", "package_url": "https://pypi.org/project/deepvoice3_pytorch/", "platform": "", "project_url": "https://pypi.org/project/deepvoice3_pytorch/", "project_urls": null, "release_url": "https://pypi.org/project/deepvoice3_pytorch/0.1.0/", "requires_dist": null, "requires_python": "", "summary": "PyTorch implementation of convolutional networks-based text-to-speech synthesis models.", "version": "0.1.0" }, "last_serial": 4421558, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "53a070dc4d7d1ccf36cc69fb053e0b0a", "sha256": "90710f3a7901bb698b8c1ed19305c40fa4f230d266595ef8fcb6305164a987fd" }, "downloads": -1, "filename": "deepvoice3_pytorch-0.0.1.tar.gz", "has_sig": false, "md5_digest": "53a070dc4d7d1ccf36cc69fb053e0b0a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22003, "upload_time": "2017-12-27T06:47:37", "url": "https://files.pythonhosted.org/packages/00/e9/efce908efc9ab128c20ec33598c6311df417525698dcd51fb1fe84435f72/deepvoice3_pytorch-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "f98a9933eae9710f0f8f649302af0fa4", "sha256": "5d52b60619bb14c2868d0545d3b70f5c903e13f36d5c726392f31fe699c9357c" }, "downloads": -1, "filename": "deepvoice3_pytorch-0.0.2.tar.gz", "has_sig": false, "md5_digest": "f98a9933eae9710f0f8f649302af0fa4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 28469, "upload_time": "2017-12-27T06:51:44", "url": "https://files.pythonhosted.org/packages/21/66/495c63885a39ee6d6a577df7e90ee961f51c521d128e29f86c403ae2cd33/deepvoice3_pytorch-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "3f21e6a50afc3680cf2260a7b748ec58", "sha256": "d3058e97be8e1453e38ebcb48212587376cf5826b3cfc024377f65c53970f4ce" }, "downloads": -1, "filename": "deepvoice3_pytorch-0.0.3.tar.gz", "has_sig": false, "md5_digest": "3f21e6a50afc3680cf2260a7b748ec58", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 28646, "upload_time": "2018-02-06T15:53:20", "url": "https://files.pythonhosted.org/packages/80/b3/c08030422cec5d3280dbc17cd16feddecf98f058678709e72ad8e37a34f5/deepvoice3_pytorch-0.0.3.tar.gz" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "da33b402ddf4328850e999dbfdc1adf3", "sha256": "0fa00e5811452827c4aeb31e5a14e2e22248f49cbad19eeb828aaad6b61a5f7f" }, "downloads": -1, "filename": "deepvoice3_pytorch-0.0.4.tar.gz", "has_sig": false, "md5_digest": "da33b402ddf4328850e999dbfdc1adf3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 27063, "upload_time": "2018-04-11T13:39:22", "url": "https://files.pythonhosted.org/packages/4d/a2/4d8ef36d191df3fc723f1ff4af5264bca0df710df40e7f0d0573c244ab43/deepvoice3_pytorch-0.0.4.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "c741c4800ee49c72909a6f5e4ac07848", "sha256": "18c0fb8f5f6d673483e812ca363228599d7c725801712252c538fee888e4690b" }, "downloads": -1, "filename": "deepvoice3_pytorch-0.0.5.tar.gz", "has_sig": false, "md5_digest": "c741c4800ee49c72909a6f5e4ac07848", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 30227, "upload_time": "2018-05-05T03:31:05", "url": "https://files.pythonhosted.org/packages/7b/88/555308bc45a734c41c5d921f046c42249c7dd51ef1095d76565ebcb428c8/deepvoice3_pytorch-0.0.5.tar.gz" } ], "0.1.0": [ { "comment_text": "", "digests": { "md5": "896f62d614a39f80a0d40e493cf3a903", "sha256": "d714268db05cb97a527f5ab6f60880a013d02074cc0c70599e402edbddd01af5" }, "downloads": -1, "filename": "deepvoice3_pytorch-0.1.0.tar.gz", "has_sig": false, "md5_digest": "896f62d614a39f80a0d40e493cf3a903", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 28990, "upload_time": "2018-10-27T07:06:01", "url": "https://files.pythonhosted.org/packages/61/08/a5b7404068c2adaa533a4079aeccd4a5cb2514eee07d6a5b82ded41ff381/deepvoice3_pytorch-0.1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "896f62d614a39f80a0d40e493cf3a903", "sha256": "d714268db05cb97a527f5ab6f60880a013d02074cc0c70599e402edbddd01af5" }, "downloads": -1, "filename": "deepvoice3_pytorch-0.1.0.tar.gz", "has_sig": false, "md5_digest": "896f62d614a39f80a0d40e493cf3a903", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 28990, "upload_time": "2018-10-27T07:06:01", "url": "https://files.pythonhosted.org/packages/61/08/a5b7404068c2adaa533a4079aeccd4a5cb2514eee07d6a5b82ded41ff381/deepvoice3_pytorch-0.1.0.tar.gz" } ] }