{ "info": { "author": "Anton Fedotov", "author_email": "anton.fedotov.af@gmail.com.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Scientific/Engineering", "Topic :: Scientific/Engineering :: Artificial Intelligence", "Topic :: Scientific/Engineering :: Image Recognition", "Topic :: Software Development :: Libraries", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "# Neural Piepline\n\nNeural networks training pipeline based on PyTorch. Designed to standardize training process and to increase coding preformance.\n\n[![Build Status](https://travis-ci.org/toodef/neural-pipeline.svg?branch=master)](https://travis-ci.org/toodef/neural-pipeline)\n[![Coverage Status](https://coveralls.io/repos/github/toodef/neural-pipeline/badge.svg?branch=master)](https://coveralls.io/github/toodef/neural-pipeline?branch=master)\n[![Maintainability](https://api.codeclimate.com/v1/badges/1feaafcc614adf27c30f/maintainability)](https://codeclimate.com/github/toodef/neural-pipeline/maintainability)\n\n* Core is about 2K lines, covered by tests, that you doesn't need to write again\n* Flexible and customizable training process\n* Checkpoints management and train process resuming (source and target device independent)\n* Metrics processing and visualization by builtin ([tensorboard](https://www.tensorflow.org/guide/summaries_and_tensorboard), [Matplotlib](https://matplotlib.org)) or custom monitors\n* Training best practices (e.g. learning rate decaying and hard negative mining)\n* Metrics logging and comparison (DVC compatible)\n\n# Train MNIST example:\nThis code run MNIST image classification with Tensorboard monitoring. Code based on PyTorch [example](https://github.com/pytorch/examples/blob/master/mnist/main.py).\n\nSee full example [there](https://github.com/toodef/neural-pipeline/blob/master/examples/files/img_classification.py).\n```python\nfrom neural_pipeline.builtin.monitors.tensorboard import TensorboardMonitor\nfrom neural_pipeline import DataProducer, AbstractDataset, TrainConfig, TrainStage,\\\n ValidationStage, Trainer, FileStructManager\n\nimport torch\nfrom torch import nn\nfrom torchvision import datasets, transforms\n\nclass Net(nn.Module):\n # Network implementation\n\nclass MNISTDataset(AbstractDataset):\n transforms = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])\n\n def __init__(self, data_dir: str, is_train: bool):\n self.dataset = datasets.MNIST(data_dir, train=is_train, download=True)\n\n def __len__(self):\n return len(self.dataset)\n\n def __getitem__(self, item):\n data, target = self.dataset[item]\n return {'data': self.transforms(data), 'target': target}\n\nfsm = FileStructManager(base_dir='data', is_continue=False)\nmodel = Net()\n\ntrain_dataset = DataProducer([MNISTDataset('data/dataset', True)], batch_size=4, num_workers=2)\nvalidation_dataset = DataProducer([MNISTDataset('data/dataset', False)], batch_size=4, num_workers=2)\n\ntrain_config = TrainConfig([TrainStage(train_dataset), ValidationStage(validation_dataset)], torch.nn.NLLLoss(),\n torch.optim.SGD(model.parameters(), lr=1e-4, momentum=0.5))\n\ntrainer = Trainer(model, train_config, fsm, torch.device('cuda:0')).set_epoch_num(50)\ntrainer.monitor_hub.add_monitor(TensorboardMonitor(fsm, is_continue=False))\ntrainer.train()\n```\n\n# Installation:\n[![PyPI version](https://badge.fury.io/py/neural-pipeline.svg)](https://badge.fury.io/py/neural-pipeline)\n[![PyPI Downloads/Month](https://pepy.tech/badge/neural-pipeline/month)](https://pepy.tech/project/neural-pipeline)\n[![PyPI Downloads](https://pepy.tech/badge/neural-pipeline)](https://pepy.tech/project/neural-pipeline)\n\n`pip install neural-pipeline`\n\n##### For `builtin` module using install:\n`pip install tensorboardX matplotlib`\n\n##### Install latest version before it's published on PyPi\n`pip install -U git+https://github.com/toodef/neural-pipeline`\n\n# Getting started:\n### Documentation\n[![Documentation Status](https://readthedocs.org/projects/neural-pipeline/badge/?version=master)](https://neural-pipeline.readthedocs.io/en/master/?badge=master)\n[See the full documentation there](https://neural-pipeline.readthedocs.io/en/master/)\n\nData flow scheme:\n![Data flow](https://github.com/toodef/neural-pipeline/blob/master/docs/img/data_flow.svg)\n\n### See the examples\n* MNIST classification - [notebook](https://github.com/toodef/neural-pipeline/blob/master/examples/notebooks/img_classification.ipynb), [file](https://github.com/toodef/neural-pipeline/blob/master/examples/files/img_classification.py)\n* Segmentation - [notebook](https://github.com/toodef/neural-pipeline/blob/master/examples/notebooks/img_segmentation.ipynb), [file](https://github.com/toodef/neural-pipeline/blob/master/examples/files/img_segmentation.py)\n* Resume training process - [file](https://github.com/toodef/neural-pipeline/blob/master/examples/files/resume_train.py)\n\n\n\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/toodef/neural-pipeline", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "neural-pipeline", "package_url": "https://pypi.org/project/neural-pipeline/", "platform": "", "project_url": "https://pypi.org/project/neural-pipeline/", "project_urls": { "Homepage": "https://github.com/toodef/neural-pipeline" }, "release_url": "https://pypi.org/project/neural-pipeline/0.1.0/", "requires_dist": [ "numpy", "tqdm", "torch (==0.4.1)" ], "requires_python": "", "summary": "Neural networks training pipeline based on PyTorch. Designed to standardize training process and to increase coding preformance", "version": "0.1.0" }, "last_serial": 4812712, "releases": { "0.0.10": [ { "comment_text": "", "digests": { "md5": "6fdc212953133a7535e4a58bc5be616f", "sha256": "d2a38224094529cd99fc310f26bfe6f1fde960591bc99b74bf99800cd38846a1" }, "downloads": -1, "filename": "neural_pipeline-0.0.10-py3-none-any.whl", "has_sig": false, "md5_digest": "6fdc212953133a7535e4a58bc5be616f", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 20402, "upload_time": "2019-01-17T20:46:49", "url": "https://files.pythonhosted.org/packages/17/57/9b1874cc9a9346db3e49948fb60cd9332f3a7c06a99ba4632bbabb3719db/neural_pipeline-0.0.10-py3-none-any.whl" } ], "0.0.12": [ { "comment_text": "", "digests": { "md5": "ecc58aeccd69d95457660aa0f313b97e", "sha256": "b3f7d700f803b5aceb7655a251ac65143f682e4ceb23e2611e3ca95b982906d5" }, "downloads": -1, "filename": "neural_pipeline-0.0.12-py3-none-any.whl", "has_sig": false, "md5_digest": "ecc58aeccd69d95457660aa0f313b97e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 20847, "upload_time": "2019-01-20T16:39:06", "url": "https://files.pythonhosted.org/packages/62/35/311fc300de1b670b45b1c6c2c258f47b79ecd7e65285b5a78c8918341c09/neural_pipeline-0.0.12-py3-none-any.whl" } ], "0.0.13": [ { "comment_text": "", "digests": { "md5": "5793724f5c7f6bb1ee189be302b131f4", "sha256": "0fe9708e1659944890a7a29f350f49c6686cc36a55645c2763c81953267683ff" }, "downloads": -1, "filename": "neural_pipeline-0.0.13-py3-none-any.whl", "has_sig": false, "md5_digest": "5793724f5c7f6bb1ee189be302b131f4", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 21419, "upload_time": "2019-01-20T21:09:23", "url": "https://files.pythonhosted.org/packages/5c/a3/41fb90a30c9b7d41a96b96229deb6806422f65e774ffe5c5506d3fe5d3e2/neural_pipeline-0.0.13-py3-none-any.whl" } ], "0.0.14": [ { "comment_text": "", "digests": { "md5": "8c2a8e5bc180825c9e3795210aaf6c94", "sha256": "fe12bb90869b31b2e3e3c5e8d15843f2bc2f0d0941172b2dde7bf9386761faef" }, "downloads": -1, "filename": "neural_pipeline-0.0.14-py3-none-any.whl", "has_sig": false, "md5_digest": "8c2a8e5bc180825c9e3795210aaf6c94", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 21448, "upload_time": "2019-01-20T21:47:02", "url": "https://files.pythonhosted.org/packages/c7/34/cecb9aba87dae19d2c288102db9b26a6240efc66d855d608c9cb2323da0b/neural_pipeline-0.0.14-py3-none-any.whl" } ], "0.0.15": [ { "comment_text": "", "digests": { "md5": "793c7ec0e6c0f5baafe859282d9a007a", "sha256": "b0cd51f14ce7a3190879319f84e97b4058a775fb9c281bcb7f7c23c10a74d11b" }, "downloads": -1, "filename": "neural_pipeline-0.0.15-py3-none-any.whl", "has_sig": false, "md5_digest": "793c7ec0e6c0f5baafe859282d9a007a", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 21917, "upload_time": "2019-01-21T20:04:35", "url": "https://files.pythonhosted.org/packages/64/3d/3c405062e8630439fd659fb273940228523a2f96f0cccc088a19133d9ba4/neural_pipeline-0.0.15-py3-none-any.whl" } ], "0.0.16": [ { "comment_text": "", "digests": { "md5": "f26e1812fd11b1915ab3a935a7b5601f", "sha256": "4254aa9b575fcfa9d50908ff56378d7205119cdc1398da7319a01cd8002a9470" }, "downloads": -1, "filename": "neural_pipeline-0.0.16-py3-none-any.whl", "has_sig": false, "md5_digest": "f26e1812fd11b1915ab3a935a7b5601f", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 23313, "upload_time": "2019-01-27T14:00:31", "url": "https://files.pythonhosted.org/packages/93/81/668c4b58587ae7edad2cd8ba28db852899dfc2d26a9178c353f5cefe3570/neural_pipeline-0.0.16-py3-none-any.whl" } ], "0.0.17": [ { "comment_text": "", "digests": { "md5": "9441c82e1bd8e494b66eec85fcfababf", "sha256": "360a4714e7ea5fa7ddce3d0a92cd34cb3f04fd0dd6787597aefea294200aa1b0" }, "downloads": -1, "filename": "neural_pipeline-0.0.17-py3-none-any.whl", "has_sig": false, "md5_digest": "9441c82e1bd8e494b66eec85fcfababf", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 24465, "upload_time": "2019-01-29T21:06:49", "url": "https://files.pythonhosted.org/packages/f6/9f/d682f2f91b941912f3ea0b5bb2e29dd5d489eba75d2ccd94e566b3ab1be8/neural_pipeline-0.0.17-py3-none-any.whl" } ], "0.0.18": [ { "comment_text": "", "digests": { "md5": "3cc20584244c7a3b8fdc32beac4aec55", "sha256": "44e98bfd83f3394929b6211b22a9df8ad73c2ceabe1a948a07562dba2380e52a" }, "downloads": -1, "filename": "neural_pipeline-0.0.18-py3-none-any.whl", "has_sig": false, "md5_digest": "3cc20584244c7a3b8fdc32beac4aec55", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 26041, "upload_time": "2019-02-03T23:08:35", "url": "https://files.pythonhosted.org/packages/e2/84/94f341f3d0c2c5c512d452684bf1d4bd5f40c3e8656cf0c0fdd9ab4a4fc4/neural_pipeline-0.0.18-py3-none-any.whl" } ], "0.0.19": [ { "comment_text": "", "digests": { "md5": "9b299976c498b3140274680ed89ff278", "sha256": "eacacbab9249b29853be3c25dd9629d23f0c94fa7c923f2838e582d0dfe30c64" }, "downloads": -1, "filename": "neural_pipeline-0.0.19-py3-none-any.whl", "has_sig": false, "md5_digest": "9b299976c498b3140274680ed89ff278", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 26295, "upload_time": "2019-02-05T21:42:55", "url": "https://files.pythonhosted.org/packages/0d/e9/23470f5c2647582588fd1f10b6f8a1295078b2e634478fcc130ac84081c6/neural_pipeline-0.0.19-py3-none-any.whl" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "8033a20ede22d48df27e1eeaa93c72ec", "sha256": "17945ef02afa6f34874b4c7919138b2acd0dec373413c305d1bbd28eb9dbe728" }, "downloads": -1, "filename": "neural_pipeline-0.0.5-py3-none-any.whl", "has_sig": false, "md5_digest": "8033a20ede22d48df27e1eeaa93c72ec", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 18444, "upload_time": "2018-12-15T19:13:51", "url": "https://files.pythonhosted.org/packages/53/8f/be12098a4aa5e2f2f65fcbe1dbd2568a75f59517bb3e09345cab735a870f/neural_pipeline-0.0.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "aac8bfdb9685d3dc183be6973cacdfd5", "sha256": "b09edc4fcebe1f4760c79f811e36a9e4028dd7c0ab869fb050c288a34aed9a7f" }, "downloads": -1, "filename": "neural-pipeline-0.0.5.tar.gz", "has_sig": false, "md5_digest": "aac8bfdb9685d3dc183be6973cacdfd5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12393, "upload_time": "2018-12-15T19:13:52", "url": "https://files.pythonhosted.org/packages/14/a4/2a745cf31a05fc2809bc71b80f0bb8f6b26e4584a9537653075e0571d888/neural-pipeline-0.0.5.tar.gz" } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "0b865d4bce90401cbfb49581ec3f74ab", "sha256": "ca119b75e1944629378bdd38f975a0d5d6c441ad01e9f6afa6d5c3e782bd3676" }, "downloads": -1, "filename": "neural_pipeline-0.0.6-py3-none-any.whl", "has_sig": false, "md5_digest": "0b865d4bce90401cbfb49581ec3f74ab", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 22578, "upload_time": "2018-12-17T19:51:33", "url": "https://files.pythonhosted.org/packages/10/e9/76dc0ccdd06100d4a5f60d5048feb9691d649ed971d518c217029e3e14d2/neural_pipeline-0.0.6-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "cde21e4307f8dc8e0f6fa3f6eb1e6f5e", "sha256": "70444880c2884eb9e20a8c8f310979d8bca53a9cb52156441349c98d7da5566c" }, "downloads": -1, "filename": "neural-pipeline-0.0.6.tar.gz", "has_sig": false, "md5_digest": "cde21e4307f8dc8e0f6fa3f6eb1e6f5e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12261, "upload_time": "2018-12-17T19:51:35", "url": "https://files.pythonhosted.org/packages/95/4f/f17e8efac4bc37a9f12a9e80a0a29f58c299c59503416934ce71dfef02b5/neural-pipeline-0.0.6.tar.gz" } ], "0.0.7": [ { "comment_text": "", "digests": { "md5": "808640109de8f4bb8dd3de21de5057b0", "sha256": "823e378283189d7dc8722167594071df85654b1cb354976f142d8de53850abdd" }, "downloads": -1, "filename": "neural_pipeline-0.0.7-py3-none-any.whl", "has_sig": false, "md5_digest": "808640109de8f4bb8dd3de21de5057b0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 19430, "upload_time": "2019-01-14T21:23:37", "url": "https://files.pythonhosted.org/packages/b2/9b/f11f74acde526d02fcbdf1576cb1dc2fb23486f7ed673172b4002cc8409f/neural_pipeline-0.0.7-py3-none-any.whl" } ], "0.0.8": [ { "comment_text": "", "digests": { "md5": "fd14c24f97865969c23b282448a29330", "sha256": "575d05b46dd9226df83919545b09831ca13ee08510c458d18b99dadb5eee9c07" }, "downloads": -1, "filename": "neural_pipeline-0.0.8-py3-none-any.whl", "has_sig": false, "md5_digest": "fd14c24f97865969c23b282448a29330", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 20598, "upload_time": "2019-01-16T19:13:42", "url": "https://files.pythonhosted.org/packages/29/be/a202be45a98ee345d90105a9a4fbd8cb7ce3a057d529ffc1c7a6e962ae31/neural_pipeline-0.0.8-py3-none-any.whl" } ], "0.0.9": [ { "comment_text": "", "digests": { "md5": "b5e625845e4d243e7d5d70e398a4d2cb", "sha256": "1af38ed1077757bf67326979f0c75ea3cfe6a9d73c4f13b8363c9fa4d86a8da3" }, "downloads": -1, "filename": "neural_pipeline-0.0.9-py3-none-any.whl", "has_sig": false, "md5_digest": "b5e625845e4d243e7d5d70e398a4d2cb", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 20681, "upload_time": "2019-01-16T21:26:18", "url": "https://files.pythonhosted.org/packages/bc/d8/a8b7c9365b786775a9105d98e7e0e6382a212a8625392d0630a5d5542640/neural_pipeline-0.0.9-py3-none-any.whl" } ], "0.1.0": [ { "comment_text": "", "digests": { "md5": "3da235fbbbdbcf079f4a439d0114bbbb", "sha256": "74a32a7fe0d33efb1ae36fc7a223a93139d9b1bd68ccb998ae3908357a02d8ac" }, "downloads": -1, "filename": "neural_pipeline-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "3da235fbbbdbcf079f4a439d0114bbbb", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 30267, "upload_time": "2019-02-12T20:42:55", "url": "https://files.pythonhosted.org/packages/f6/ae/440a1d20745d5de34c3a79605a63ad00669b93ffa0125f01535c9c72134c/neural_pipeline-0.1.0-py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "3da235fbbbdbcf079f4a439d0114bbbb", "sha256": "74a32a7fe0d33efb1ae36fc7a223a93139d9b1bd68ccb998ae3908357a02d8ac" }, "downloads": -1, "filename": "neural_pipeline-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "3da235fbbbdbcf079f4a439d0114bbbb", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 30267, "upload_time": "2019-02-12T20:42:55", "url": "https://files.pythonhosted.org/packages/f6/ae/440a1d20745d5de34c3a79605a63ad00669b93ffa0125f01535c9c72134c/neural_pipeline-0.1.0-py3-none-any.whl" } ] }