{
    "info": {
        "author": "Noah Golmant",
        "author_email": "noah.golmant@gmail.com",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "# Skeletor [![Build Status](https://travis-ci.org/noahgolmant/skeletor.svg?branch=master)](https://travis-ci.org/noahgolmant/skeletor)\n\nSkeletor is a lightweight wrapper for research code. It is meant to enable fast, parallelizable prototyping without sacrificing reproducibility or ease of experiment analysis.\n\nYou can install it with: `pip install skeletor-ml`\n\n## Why use skeletor?\n\nTracking and analyzing experiment results is easy. Skeletor uses [track](https://github.com/richardliaw/track), which provides a simple interface to log metrics throughout training and to view those metrics in a pandas DataFrame afterwards. It can log locally and to S3. Compared to other logging tools, track has minimal overhead and a very simple interface. No longer do you need to decorate every function or specify a convoluted experiment pipeline.\n\nOrchestrating many experiments in parallel is simple and robust. Almost every experiment tracking framework implements its own scheduling and hyperparameter search algorithms. Luckily, I don't trust myself to do this correctly. Instead, skeletor uses [ray](https://github.com/ray-project/ray), a high-performance distributed execution framework. In particular, it uses [ray tune](https://ray.readthedocs.io/en/latest/tune.html) for scalable hyperparameter search. \n\n## Setup\n\nNecessary packages are listed in `setup.py`.\nJust run `pip install skeletor-ml` to get started.\n\n## Basic Usage\n\nA basic example `train.py` might look like:\n\n```\nimport skeletor\nfrom skeletor.models import build_model\nfrom skeletor.datasets import build_dataset\nfrom skeletor.optimizers import build_optimizer\nimport track\n\ndef add_args(parser):\n    parser.add_argument('--arch', default='resnet50')\n    parser.add_argument('--lr', default=0.1, type=float)\n\ndef train(epoch, trainloader, model, optimizer):\n    ...\n    return avg_train_loss\n\ndef test(epoch, testloader, model):\n    ...\n    return avg_test_loss\n\ndef experiment(args):\n    trainloader, testloader = build_dataset('cifar10')\n    model = build_model(args.arch, num_classes=10)\n    opt = build_optimizer('SGD', lr=args.lr)\n    for epoch in range(200):\n        track.debug(\"Starting epoch %d\" % epoch)\n        train_loss = train(epoch, trainloader, model, opt)\n        test_loss = test(epoch, testloader, model)\n        track.metric(iteration=epoch,\n                     train_loss=train_loss,\n                     test_loss=test_loss)\n\nskeletor.supply_args(add_args)\nskeletor.execute(experiment)\n```\n\nYou just have to supply (1) a function that adds your desired arguments to an `ArgumentParser` object, and (2) a function that runs the experiment using the parsed arguments. You can then use `track` to log statistics during training.\n\nYou can supply a third function to run analysis after training. `skeletor.supply_postprocess(postprocess_fn)` takes in a user-defined function of the form `postprocess_fn(proj)`. `proj` is a `track.Project` object.\n\nInternally, the basic experiment flow is:\n\n`run add_args(parser) -> parse the args -> run experiment_fn(args) -> optionally run postprocess_fn(proj)`\n\n## Launching experiments\n\nTo launch an experiment in `train.py`, you just do `python train.py <my args> <experimentname>`. The results will go in `<logroot>/<experimentname>`. For example, you can do something like\n\n`CUDA_VISIBLE_DEVICES=0 python train.py --arch ResNet50 --lr .1 resnet_cifar`\n\n\nThe same code can be used to launch several experiments in parallel. Suppose I have a config called `config.yaml` that looks like:\n\n```\narch: ResNet50\nlr:\n  grid_search: [.001, .01, .1, 1.0]\n```\n\nI can test out all of these learning rates at the same time by running:\n\n`CUDA_VISIBLE_DEVICES=0,1 python train.py --config=config.yaml --self_host=2 resnet_cifar`\n\nRay will handle scheduling the jobs across all available resources.\n\nLogs (`track` records) will be stored in `<args.logroot>/<args.experimentname>`.\nSee the `track` docs for how to access these records as DataFrames.\n\n## Examples\n\nYou can find an example of running a grid search for training a residual network on CIFAR-10 in PyTorch in `examples/train.py`.\n\n\n## Getting experiment results\n\nI added a utility in `skeletor.proc` for converting all `track` trial records for an experiment into a single Pandas DataFrame. It can also pickle it.\n\nThat means if I run an experiment like above called `resnet_cifar`, I can access all of the results for all the trials as a single DataFrame by calling `skeletor.proc.proj('resnet_cifar', './logs')`.\n\n## Registering custom models, dataloaders, and optimizers\n\nRegistering custom classes allows you to construct an instance of the specified class by calling `build_model`, `build_dataset`, or `build_optimizer` with the class string name. This is useful for hyperparameter searching because you can search over these choices directly by class name.\n\nI try to provide a simple interface for registering custom implementations with skeletor. For example, I can register a custom `Model` class by calling `skeletor.models.add_model(Model)`. This allows me to create models through `skeletor.models.build_model('Model')`. You can also register entire modules full of definitions at once. There are analogous functions `add_dataset, add_optimizer` for datasets and optimizers.\n\n```\nclass MyNetwork(Module):\n    ...\n\nskeletor.models.add_model(MyNetwork)\n\narch_name = 'MyNetwork'\nmodel = skeletor.models.build_model(arch_name)\n```\n\n## Help me out / Things to Do\n\nWe have active [issues](https://github.com/noahgolmant/skeletor/issues)! Feel free to suggest new improvements or add PRs to contribute.\n\n[...](https://www.youtube.com/watch?v=g20_8-TPyTQ)\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/noahgolmant/skeletor",
        "keywords": "",
        "license": "MIT License",
        "maintainer": "",
        "maintainer_email": "",
        "name": "skeletor-ml",
        "package_url": "https://pypi.org/project/skeletor-ml/",
        "platform": "",
        "project_url": "https://pypi.org/project/skeletor-ml/",
        "project_urls": {
            "Homepage": "https://github.com/noahgolmant/skeletor"
        },
        "release_url": "https://pypi.org/project/skeletor-ml/0.1.4/",
        "requires_dist": [
            "pyyaml (>=3.12)",
            "ray (>=0.4.0)",
            "track-ml (>=0.1)",
            "torchvision",
            "awscli"
        ],
        "requires_python": "",
        "summary": "A lightweight module for research experiment reproducibility and analysis",
        "version": "0.1.4"
    },
    "last_serial": 4517935,
    "releases": {
        "0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a498f8c4e6105bb21ae0a0a3993cef70",
                    "sha256": "dac95a612ef5617354d52a5a902429c458139fdaf763c100468042c367c59326"
                },
                "downloads": -1,
                "filename": "skeletor_ml-0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a498f8c4e6105bb21ae0a0a3993cef70",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 9652,
                "upload_time": "2018-10-21T09:16:54",
                "url": "https://files.pythonhosted.org/packages/37/8b/bf1431cd3b892f2317110a4effda0f8090c3bd32d662fb778e719565a94f/skeletor_ml-0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "9e35fc3d95ac747587766f7f3787fe4f",
                    "sha256": "7b381c13d278f277314c9ccf7870351aa9a58115f6fba07775cc497c74bd4bc9"
                },
                "downloads": -1,
                "filename": "skeletor-ml-0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "9e35fc3d95ac747587766f7f3787fe4f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6880,
                "upload_time": "2018-10-21T09:16:55",
                "url": "https://files.pythonhosted.org/packages/ca/6e/184ef84ba9fc559befbb44d219f9eefdef094aae33dac4a3d036870fd3fd/skeletor-ml-0.1.tar.gz"
            }
        ],
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f1eb6a305f438fafbf05261eb168a458",
                    "sha256": "2712d88d3a72e065f5a6712d2b9b6da4e9cae5d77055d7d425382f103abd1f6c"
                },
                "downloads": -1,
                "filename": "skeletor_ml-0.1.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f1eb6a305f438fafbf05261eb168a458",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 12958,
                "upload_time": "2018-10-21T13:34:49",
                "url": "https://files.pythonhosted.org/packages/9d/57/daa0ee6f1502aa31d4ceacf313da3f909b61e43c27e6830ab97f0e2ad178/skeletor_ml-0.1.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "230e254c92c8478c8b958f6da07d82da",
                    "sha256": "b57c91569ef83d1f9b76bf9c11f615c9f169163bd9b3c1cabadc8974cf255e23"
                },
                "downloads": -1,
                "filename": "skeletor-ml-0.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "230e254c92c8478c8b958f6da07d82da",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10950,
                "upload_time": "2018-10-21T13:34:50",
                "url": "https://files.pythonhosted.org/packages/e0/d2/2f49153782c687545f78d04b0f8922a289c637e772368193eee6438d4819/skeletor-ml-0.1.1.tar.gz"
            }
        ],
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a6aa41ddddb12d56c39ad2c2273d8b75",
                    "sha256": "2169058f6fc2647a23e3ffb28fbffc71614ff852619feadefb6daae65adbcb7d"
                },
                "downloads": -1,
                "filename": "skeletor_ml-0.1.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a6aa41ddddb12d56c39ad2c2273d8b75",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 22803,
                "upload_time": "2018-10-23T06:50:39",
                "url": "https://files.pythonhosted.org/packages/40/c3/34441dbae0876d6bc54ce9ebd29fdbea0c32c8d7c6cbfa2c81734ec1e77c/skeletor_ml-0.1.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "a8ac946862c876dec01d7d551adfbda6",
                    "sha256": "b06f6d68275f39d9d228e36bd9fc08ad7f13837cdbc982c4fd390cf44c92a697"
                },
                "downloads": -1,
                "filename": "skeletor-ml-0.1.2.tar.gz",
                "has_sig": false,
                "md5_digest": "a8ac946862c876dec01d7d551adfbda6",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14971,
                "upload_time": "2018-10-23T06:50:41",
                "url": "https://files.pythonhosted.org/packages/e7/89/cee32cd4522c8d6fd11bd88f095e47a97e7ae81bdd8be442daa64f3d2730/skeletor-ml-0.1.2.tar.gz"
            }
        ],
        "0.1.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4adfecf8a5d747939e331b78a37c4cf4",
                    "sha256": "c93bb2ab7fe4fc1845e325f7b9cd828f21a3883fb158eb35edb80cf435775ff9"
                },
                "downloads": -1,
                "filename": "skeletor_ml-0.1.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "4adfecf8a5d747939e331b78a37c4cf4",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 26388,
                "upload_time": "2018-11-05T06:21:27",
                "url": "https://files.pythonhosted.org/packages/6a/ad/eb27febca1670ed6037d9a0b558a3aece08bec7baaa393d7fce648082635/skeletor_ml-0.1.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "ce40b7214dbae2701297f907661505a1",
                    "sha256": "fff9682c10a40d83341bcc42c2fadc232069ef64ab660d4c8f530b1e2a685767"
                },
                "downloads": -1,
                "filename": "skeletor-ml-0.1.3.tar.gz",
                "has_sig": false,
                "md5_digest": "ce40b7214dbae2701297f907661505a1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 19176,
                "upload_time": "2018-11-05T06:21:29",
                "url": "https://files.pythonhosted.org/packages/9e/79/f02741830f2ffa22b15bce8c22141a72450478dbdc83f765f215b7a0087e/skeletor-ml-0.1.3.tar.gz"
            }
        ],
        "0.1.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1767d9275f234209d52d7b4cf0914cff",
                    "sha256": "c76a701e8ced98203a90c478c521ace572d92f225eb91bb943c629e3fc49bb03"
                },
                "downloads": -1,
                "filename": "skeletor_ml-0.1.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1767d9275f234209d52d7b4cf0914cff",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 26630,
                "upload_time": "2018-11-22T17:55:09",
                "url": "https://files.pythonhosted.org/packages/dc/e5/b613ec02b0cce5254722f841cac4657fb59b8823abb2b3003c011c4c31bc/skeletor_ml-0.1.4-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "ad0205c0393260acf7d2f08390926f0b",
                    "sha256": "f16503afb7cb090c03b2f91b13e1fb7752944bb3a581b93abb0ce646d9acedf1"
                },
                "downloads": -1,
                "filename": "skeletor-ml-0.1.4.tar.gz",
                "has_sig": false,
                "md5_digest": "ad0205c0393260acf7d2f08390926f0b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 19701,
                "upload_time": "2018-11-22T17:55:11",
                "url": "https://files.pythonhosted.org/packages/09/60/9d0d81637cee24d88bfc258b88a2fba44d36cdaa541e17090bce306fcd57/skeletor-ml-0.1.4.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "1767d9275f234209d52d7b4cf0914cff",
                "sha256": "c76a701e8ced98203a90c478c521ace572d92f225eb91bb943c629e3fc49bb03"
            },
            "downloads": -1,
            "filename": "skeletor_ml-0.1.4-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "1767d9275f234209d52d7b4cf0914cff",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 26630,
            "upload_time": "2018-11-22T17:55:09",
            "url": "https://files.pythonhosted.org/packages/dc/e5/b613ec02b0cce5254722f841cac4657fb59b8823abb2b3003c011c4c31bc/skeletor_ml-0.1.4-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "ad0205c0393260acf7d2f08390926f0b",
                "sha256": "f16503afb7cb090c03b2f91b13e1fb7752944bb3a581b93abb0ce646d9acedf1"
            },
            "downloads": -1,
            "filename": "skeletor-ml-0.1.4.tar.gz",
            "has_sig": false,
            "md5_digest": "ad0205c0393260acf7d2f08390926f0b",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 19701,
            "upload_time": "2018-11-22T17:55:11",
            "url": "https://files.pythonhosted.org/packages/09/60/9d0d81637cee24d88bfc258b88a2fba44d36cdaa541e17090bce306fcd57/skeletor-ml-0.1.4.tar.gz"
        }
    ]
}