{
    "info": {
        "author": "Intel Nervana",
        "author_email": "intelnervana@intel.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Environment :: Console",
            "Intended Audience :: Developers",
            "Intended Audience :: End Users/Desktop",
            "Intended Audience :: Science/Research",
            "Operating System :: MacOS :: MacOS X",
            "Operating System :: POSIX",
            "Programming Language :: Python :: 3",
            "Topic :: Scientific/Engineering :: Artificial Intelligence",
            "Topic :: Scientific/Engineering :: Information Analysis",
            "Topic :: System :: Distributed Computing"
        ],
        "description": "# mlt\nMachine Learning Container Templates\n\n> MLT: it's like the keras of kubernetes\n>\n> \\- @mas-dse-greina\n\n[![CircleCI](https://circleci.com/gh/IntelAI/mlt.svg?style=svg&circle-token=239cc4305ca25063bf9a40cd332c822f5e64663f)](https://circleci.com/gh/IntelAI/mlt)\n\n`mlt` aids in the creation of containers for machine learning jobs.\nIt does so by making it easy to use container and kubernetes object templates.\n\n![MLT flow diagram](docs/mlt.png)\n\nWe have found it useful to share project templates for various machine learning frameworks. Some have native support from Kubernetes operators, such as mxnet and TensorFlow. Others do not, but still have best practices for how to run on a Kubernetes cluster.\n\nOn top of getting boiler plate code at the beginning of a project to work, the best practices may change over time. `mlt` allows existing projects to adapt to these without having to reset and start over.\n\n![MLT watch](docs/watch.png)\n\n`mlt` addresses another aspect of the application development: _iterative_ container creation. Storage and container creation is supposed to be fast - so why not rebuild containers automatically?\n`mlt` has a `--watch` option, which lets you write code and have an IDE-like experience.\nWhen changes are detected, a timer starts and triggers container rebuilds.\nlint and unit tests can be run in this step, as an early indicator of whether the code will run in the cluster.\nWhen the container is built, it is pushed to the cluster container registry.\nFrom here, it is a quick step to redeploy the Kubernetes objects, through `mlt deploy`\n\n\n## Build\n\nPrerequisites:\n- Python (2.7 or newer) development environment. This usually consists of `build-essential`, `libssl-dev`, `libffi-dev` and `python-dev`\n- [Docker](https://docs.docker.com/install/)\n- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)\n- [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)\n- [python](https://www.python.org/downloads/)\n- [pip](https://pip.pypa.io/en/stable/installing/)\n\nOptional prerequisites:\n- [kubetail](https://github.com/johanhaleby/kubetail) (for `mlt logs` command)\n- [Ksync](https://github.com/vapor-ware/ksync) (for `mlt sync` commands)\n- [ks](https://github.com/IntelAI/mlt/blob/master/scripts/ksonnet_install_linux.sh) (Needed `0.11.0` version for volume support to work in `horovod` template)\n- [TFJob operator](https://github.com/kubeflow/tf-operator#installing-the-tfjob-crd-and-operator-on-your-k8s-cluster) (for the `distributed tensorflow` templates)\n- [PyTorch operator](https://github.com/kubeflow/pytorch-operator) (for the `pytorch-distributed` template)\n- [jq](http://macappstore.org/jq/) (for the `tensorboard` templates)\n- [openssh-server](https://www.openssh.com/) (for the `horovod` template)\n\nLinux Debian-based distro users:\n- Run `make debian_prereq_install` to install every dependency required for `mlt` to work. We assume users already have `kubectl` and `docker` set up though.\n\n## Installation\n\n### Install from [PyPI](https://pypi.org/)\n```bash\n$ pip install mlt\n```\n\n### Installation from Source\n\n```bash\n$ git clone git@github.com:IntelAI/mlt.git\nCloning into 'mlt'...\nremote: Counting objects: 1981, done.\nremote: Compressing objects: 100% (202/202), done.\nremote: Total 1981 (delta 202), reused 280 (delta 121), pack-reused 1599\nReceiving objects: 100% (1981/1981), 438.10 KiB | 6.54 MiB/s, done.\nResolving deltas: 100% (1078/1078), done.\n\n$ cd mlt\n\n$ pip install .\n```\n\n## Create local Python distributions\n```bash\n$ make dist\n$ cd dist\n$ ls mlt*\nmlt-0.1.0a1+12.gf49c412.dirty-py2.py3-none-any.whl\n```\n\n## Usage summary\n\n### Sample mlt deployment\n[![asciicast](https://asciinema.org/a/171353.png)](https://asciinema.org/a/171353)\n\n```bash\n\n$ mlt template list\n\nTemplate             Description\n-------------------  --------------------------------------------------------------------------------------------------\nhello-world          A TensorFlow python HelloWorld example run through Kubernetes Jobs.\npytorch              Sample distributed application taken from http://pytorch.org/tutorials/intermediate/dist_tuto.html\npytorch-distributed  A distributed PyTorch MNIST example run using the pytorch-operator.\ntf-dist-mnist        A distributed TensorFlow MNIST model which designates worker 0 as the chief.\ntf-distributed       A distributed TensorFlow matrix multiplication run through the TensorFlow Kubernetes Operator.\n\n$ mlt init my-app --template=hello-world\n[master (root-commit) 40239a2] Initial commit.\n 7 files changed, 191 insertions(+)\n create mode 100644 mlt.json\n create mode 100644 Dockerfile\n create mode 100644 Makefile\n create mode 100644 k8s-templates/tfjob.yaml\n create mode 100644 k8s/README.md\n create mode 100644 main.py\n create mode 100644 requirements.txt\n\n$ cd my-app\n\n# NOTE: `mlt config` has been renamed to `mlt template_config` to avoid confusion regarding developer config settings.\n\n# List template-specific config parameters\n$ mlt template_config list\nParameter Name                Value\n----------------------------  ----------------------\nregistry                      my-project-12345\nnamespace                     my-app\nname                          my-app\ntemplate_parameters.greeting  Hello\n\n# Update the greeting parameter\n$ mlt template_config set template_parameters.greeting Hi\n\n# Check the template_config list to see the updated parameter value\n$ mlt template_config list\nParameter Name                Value\n----------------------------  ----------------------\nregistry                      constant-cubist-173123\nnamespace                     dmsuehir\nname                          dmsuehir\ntemplate_parameters.greeting  Hi\n\n$ mlt build\nStarting build my-app:71fb176d-28a9-46c2-ab51-fe3d4a88b02c\nBuilding |######################################################| (ETA:  0:00:00)\nPushing  |######################################################| (ETA:  0:00:00)\nBuilt and pushed to gcr.io/my-project-12345/my-app:71fb176d-28a9-46c2-ab51-fe3d4a88b02c\n\n$ mlt deploy\nDeploying gcr.io/my-project-12345/my-app:71fb176d-28a9-46c2-ab51-fe3d4a88b02c\n\nInspect created objects by running:\n  $ kubectl get --namespace=my-app all\n\n### Provide -l flag to tail logs immediately after deploying.\n$ mlt deploy --no-push -l\nSkipping image push\nDeploying gcr.io/my-project-12345/my-app:b9f124d2-ef34-4d66-b137-b8a6026bf782\n\nInspect created objects by running:\n$ kubectl get --namespace=my-app all\n\nChecking for pod(s) readiness\nWill tail 1 logs...\nmy-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg\n[my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg] 2018-05-17 22:28:34.578791: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA\n[my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg] b'Hello, TensorFlow!'\n\n$ mlt status\nNAME                                                  READY     STATUS    RESTARTS   AGE       IP            NODE\nmy-app-897cb68f-e91f-42a0-968e-3e8073334450-vvpqj     1/1       Running   0          14s       10.23.45.67   gke-my-cluster-highmem-8-skylake-1\n\n### To deploy in interactive mode (using no-push as an example)\n### NOTE: only basic functionality is supported at this time. Only one container and one pod in a deployment for now.\n#### If more than one container in a deployment, we'll pick the first one we find and deploy that.\n\n$ mlt deploy -i --no-push\nSkipping image push\nDeploying localhost:5000/test:d6c9c06b-2b64-4038-a6a9-434bf90d6acc\n\n$ mlt logs\nChecking for pod(s) readiness\nWill tail 1 logs...\nmy-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg\n[my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg] 2018-05-17 22:28:34.578791: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA\n[my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg] b'Hello, TensorFlow!'\n\nInspect created objects by running:\n$ kubectl get --namespace=my-app all\n\nConnecting to pod...\nroot@test-9e035719-1d8b-4e0c-adcb-f706429ffeac-wl42v:/src/app# ls\nDockerfile  Makefile  README.md  k8s  k8s-templates  main.py  mlt.json\trequirements.txt\n\n# Displays events for the current job\n$ mlt events\nLAST SEEN   FIRST SEEN   COUNT     NAME                                                                            KIND      SUBOBJECT                     TYPE      REASON                  SOURCE                                                   MESSAGE\n\n6m          6m           1         my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg.152f8f13466696b4              Pod                                     Normal    Scheduled               default-scheduler                                        Successfully assigned my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg to gke-dls-us-n1-highmem-8-skylake-82af83b4-8nvh\n6m          6m           1         my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg.152f8f134ff373d7              Pod                                     Normal    SuccessfulMountVolume   kubelet, gke-dls-us-n1-highmem-8-skylake-82af83b4-8nvh   MountVolume.SetUp succeeded for volume \"default-token-grq2c\"\n6m          6m           1         my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg.152f8f1399b33ba0              Pod       spec.containers{my-app}       Normal    Pulled                  kubelet, gke-dls-us-n1-highmem-8-skylake-82af83b4-8nvh   Container image \"gcr.io/my-project-12345/my-app:b9f124d2-ef34-4d66-b137-b8a6026bf782\" already present on machine\n6m          6m           1         my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg.152f8f139dec0dc3              Pod       spec.containers{my-app}       Normal    Created                 kubelet, gke-dls-us-n1-highmem-8-skylake-82af83b4-8nvh   Created container\n6m          6m           1         my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg.152f8f13a2ea0ff6              Pod       spec.containers{my-app}       Normal    Started                 kubelet, gke-dls-us-n1-highmem-8-skylake-82af83b4-8nvh   Started container\n6m          6m           1         my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33.152f8f13461279e4                    Job                                     Normal    SuccessfulCreate        job-controller                                           Created pod: my-app-09aa35f4-bdf8-4da8-8400-8728bf7afa33-sqzqg\n\n\n```\n\n### Examples\n\n* [Distributed U-Net model training using KVC and MLT](examples/distributed_unet)\n* [Distributed U-Net model training using horovod and openmpi on Kubernetes](examples/horovod_unet)\n\n### Template Development\n\nTo add new templates, see the [Template Developers Manual](docs/template_developers.md).\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/IntelAI/mlt",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "mlt",
        "package_url": "https://pypi.org/project/mlt/",
        "platform": "",
        "project_url": "https://pypi.org/project/mlt/",
        "project_urls": {
            "Homepage": "https://github.com/IntelAI/mlt"
        },
        "release_url": "https://pypi.org/project/mlt/0.3.0/",
        "requires_dist": [
            "pip (>=10.0.1)",
            "conditional (>=1.2)",
            "docopt (>=0.6.2)",
            "progressbar2 (>=3.36.0)",
            "tabulate (>=0.8.2)",
            "termcolor (>=1.1.0)",
            "PyYAML (>=3.12)",
            "watchdog (>=0.8.3)",
            "jsonschema (==2.6.0)",
            "pytz (==2018.5)"
        ],
        "requires_python": "",
        "summary": "Machine Learning Container Templates",
        "version": "0.3.0"
    },
    "last_serial": 4258357,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3e407c9c8afbf4dbf376b9cb33e6ada9",
                    "sha256": "9abe196e19f3c9bdbd28059d64c9e678464f202f47a0038e35282b85a187a008"
                },
                "downloads": -1,
                "filename": "mlt-0.1.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "3e407c9c8afbf4dbf376b9cb33e6ada9",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 52331,
                "upload_time": "2018-04-28T01:55:02",
                "url": "https://files.pythonhosted.org/packages/71/1e/d9a6c016aa8560e53a73b30a91f61f2da52382f1723f3ac08e6760ba116d/mlt-0.1.0-py2.py3-none-any.whl"
            }
        ],
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "50c9464e029b181b750bb2148078a739",
                    "sha256": "e6ba45e0bddd47d80aff904ad90e6d3e0f891ced371abbccd821887946c2135f"
                },
                "downloads": -1,
                "filename": "mlt-0.1.1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "50c9464e029b181b750bb2148078a739",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 52405,
                "upload_time": "2018-05-09T20:48:55",
                "url": "https://files.pythonhosted.org/packages/44/e1/90579b660bf0b523d12b28bb2c860521066b2a3f2dcf0ac145f87898e2b5/mlt-0.1.1-py2.py3-none-any.whl"
            }
        ],
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "72b8b814cf2bad0c67ee4322d4a355e5",
                    "sha256": "6fe57b1ba8d414321e93b5258d020fb431e2da6c7ca5b5e9e141cbce96d04052"
                },
                "downloads": -1,
                "filename": "mlt-0.1.2-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "72b8b814cf2bad0c67ee4322d4a355e5",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 67338,
                "upload_time": "2018-05-29T18:15:14",
                "url": "https://files.pythonhosted.org/packages/43/87/fc44b33f39638ab1e802e4f2600d6337caf9616db842ccd39be0a3b4a568/mlt-0.1.2-py2.py3-none-any.whl"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "44f1a0d45a763c26c51b0a342574925b",
                    "sha256": "a4aa3283620051b64735df62dbbd9ffea6bd31489ef18ff58b56f030de569b21"
                },
                "downloads": -1,
                "filename": "mlt-0.2.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "44f1a0d45a763c26c51b0a342574925b",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 103930,
                "upload_time": "2018-07-18T16:35:58",
                "url": "https://files.pythonhosted.org/packages/92/57/b3f20ed0a6cf156c089bbba7751790120102e90ef2589e83db9954f1a078/mlt-0.2.0-py2.py3-none-any.whl"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "131393e9dc0ee5a6218165a1f89b89a0",
                    "sha256": "0c4693bf583d629862b0d9263a39dccbc8072c00aa9f67d7b8f2cc02a4efeb8a"
                },
                "downloads": -1,
                "filename": "mlt-0.2.1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "131393e9dc0ee5a6218165a1f89b89a0",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 105172,
                "upload_time": "2018-07-26T06:26:46",
                "url": "https://files.pythonhosted.org/packages/aa/ab/df964a07d2cb6629283b136021a0f52aa22172b6576cb822ba8a2d65f06d/mlt-0.2.1-py2.py3-none-any.whl"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "446d40e7c5818f4b77db9fd9cb26cb0c",
                    "sha256": "30aed0728b63496041c23028d8b33b9e2ab0c30ef687ce9a87add5eaa469d2b0"
                },
                "downloads": -1,
                "filename": "mlt-0.3.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "446d40e7c5818f4b77db9fd9cb26cb0c",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 126680,
                "upload_time": "2018-09-10T16:42:54",
                "url": "https://files.pythonhosted.org/packages/dd/02/11633ee7d830b70746b819175d29f9131d088dbc54e660cae0b78c8d51d6/mlt-0.3.0-py2.py3-none-any.whl"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "446d40e7c5818f4b77db9fd9cb26cb0c",
                "sha256": "30aed0728b63496041c23028d8b33b9e2ab0c30ef687ce9a87add5eaa469d2b0"
            },
            "downloads": -1,
            "filename": "mlt-0.3.0-py2.py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "446d40e7c5818f4b77db9fd9cb26cb0c",
            "packagetype": "bdist_wheel",
            "python_version": "py2.py3",
            "requires_python": null,
            "size": 126680,
            "upload_time": "2018-09-10T16:42:54",
            "url": "https://files.pythonhosted.org/packages/dd/02/11633ee7d830b70746b819175d29f9131d088dbc54e660cae0b78c8d51d6/mlt-0.3.0-py2.py3-none-any.whl"
        }
    ]
}