{
    "info": {
        "author": "Ge Yang",
        "author_email": "yangge1987@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Intended Audience :: Science/Research",
            "Programming Language :: Python :: 3"
        ],
        "description": "Escher (``escher-cli``), a command line utility that runs your script with arguments load from a Yaml file\n==========================================================================================================\n\n\ud83d\udd25 \ud83d\udca5\\ **Now ``Escher`` supports launching workers on AWS!** \ud83c\udf1f \u2728 \u26a1\ufe0f\n\nWhere Am I? (for the developer himself)\n---------------------------------------\n\n1. [ ] **Need to add ``escher init``** so that you have a standard way\n   to setup the scripts for a project.\n\n**For Usage, jump to the next section already :)**\n\nMotivations\n-----------\n\nWhen publishing deep-learning models, there are a lot of boilerplate\nscripts for the training and inference. When there are twenty-plus\nmodels in your work folder it gets increasingly difficult to maintain\ncopies of run scripts that has tight bindings with the particular server\nyou were using.\n\nIt makes sense to use a single deep-learning ``cli`` to minimalize these\ncopy-and-pasted scripts\n\nWhen you work on a lot of deep-learning project, some of them playing\nwith other people's code-block, some of then developing new ones, it really\nmakes sense to standard the structure of project and reduce boiler\nplate. Here with ``escher-cli``, we learn from the best packaging tools\non this planet, Ruby ``gem``, Javascript ``npm``, and rust ``crate``. We\nare particularly inspired by ``create-react-app`` and the good work that\nyou-know-who has offered the ``reactjs`` community, and some of the\nlessons people have learned over-the-years from tools like ``gulp``,\n``bower``, ``webpack`` and numerous others.\n\nYou can think of ``escher-cli`` as the ``create-react-app`` for\ndeep-learning. The reason why you want use this is because it offers a\nboilerplate-free way to **setup new project**, **launch and manage aws,\nazure, google compute engine servers**, **local and remote training**.\nInstead of writing ``Makefiles`` that you have to copy paste in every\nproject, you install ``escher-cli`` globally *or* in your particular\nproject virtual-env.\n\nIn summary, our (**what**) contributions (**you**) are (**get**):\n\n1. A community-driven standard in deep-ML training, development and\n   (later) serving.\n2. cross-platform ``cli`` tools that reduces the cognitive load of\n   running training locally and globally\n3. No copy-and-paste scripts in each project.\n\nIDE Support\n~~~~~~~~~~~\n\nWe will build a cli tool first. If this catches on, I'm sure JetBrains\nwill add integration in PyCharm:) And if you are not using PyCharm, you\nshould :-P\n\nInstall and Usage\n-----------------\n\n**PyPI Page**: https://pypi.python.org/pypi/escher-cli\n\nTodo and Milestones\n-------------------\n\n-  [ ] run locally\n-  [ ] build a worker daemon that ``start``, ``halt``, ``resume``,\n   ``reran`` jobs\n\n   -  [x] finish graphQL schema\n\n-  [ ] build a master controller that\n\n   -  [ ] does worker discovery (worker report on startup)\n   -  [ ] maintains a list of jobs\n\n      -  [ ] new\n      -  [ ] running\n      -  [ ] ran\n\n   -  [ ] finish graphQL schema\n\nDone\n~~~~\n\n-  [x] define job file and schema (.escher.run)\n\nRun Locally\n~~~~~~~~~~~\n\n.. code-block:: bash\n\n    \u2713 pip install escher-cli\n    \u2717 escher init  # this one creates the `.escher` \n    \u2713 escher run  # runs a default script\n    # outputs >> `escher run` <default> script works!\n    \u2713 escher run scripts/load_electron.escher  # escher script that \"looks like\" this.\n\nTo run experiments on Remote Workers\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n-  [ ] add remote worker support\n\n.. code-block:: bash\n\n    \u2713 pip install escher-cli\n    \u2717 escher init --aws-example\n    escherd  # this starts the local daemon\n        \u2717 escher master\n        \u2717 escher worker\n    # now sit back and enjoy the show\n    esher run --worker=gpu-worker scripts/load_electrons.escher\n\nConfiguration for the ``gpu-worker`` sits inside the ``.escher`` file.\nIt specifies the ``aws`` image id, security group etc.\n\nOverview\n--------\n\n``Escher`` is a command line utility for your experiments. It manages\nconcurrent runs, command line arguments and other stuff nicely for you!\n\nThe things it manages include:\n\n-  generating run-specific variables, like ``{time}`` and ``{cwd}`` that\n   you can access as template keys in your string.\n-  maximum numbers of concurrent runs (via python ``multiprocess.Pool``)\n-  environment variables (env) such as ``DISPLAY=:1``. You can also\n   passing a file\n-  **default arguments**\n-  **batch arguments** for multiple experiments\n-  **pre-launch** scripts that is ran only once for each batch\n-  **at-launch** script that is run at the launch time of each\n   experiment\n-  **post-launch** script that is as soon as each experiment launches\n-  **post-run** scripts that happens after each experiment is done.\n\n``Escher`` works with all of your current work flow and it doesn't lock\nyou into a specific way of doing things. You still get to use bash\nscripts or Makefile's.\n\nWhy do we use YAML (with JSON planned as well)? My bash script is just fine!\n----------------------------------------------------------------------------\n\nBecause YAML(and json) are human **and** machine readable whereas bash\narguments are **not**. When is the last time you looked at a bash script\nlooking for hyper-parameters, only to realized that some key parameters\nare missing? With ``Escher`` you won't have this problem. You can save\nall of the past and current runs as nicely readable YAML files(or json,\nHjson Cson if you speak coffee script, whatever).\n\n-  if you are writing a report, you can include these YAML/json/cson\n   files directly into markdown or latex.\n-  To see what you ran last night, you can ``glob`` all of the\n   configurations with ``runs/**/experiment.yml`` and make nice tables\n   in an instant.\n-  To run a full batch of experiments, you can type ``default_args``\n   just ones, and then only repeat the fields that is different between\n   each run.\n\nMost importantly, ``Escher`` wants to establish a standard command line\ntool for machine learning work, and having a standard ``run.config.yml``\nis just the first step!\n\nWhy is ``Escher`` named \"Escher\"?\n---------------------------------\n\nIt used to be called ``Dave``, after my adviser. I thought it would be\nfunny to ``dave --config-file \"load_electrons.yml\"``. Now it is named to\n``Escher`` (as for the module), and ``escher-cli`` as the package name.\n\nThere already exists a package called ``escher`` which is why this\npackage is ``escher-cli``. However I like the ``-cli`` postfix b/c it is\nmore clear.\n\nHow shall I use ``Escher``?\n---------------------------\n\nFirst thing first, you could ``alias`` ``Escher`` with whoever you want\nto call to run your experiments \ud83d\ude00.\n\nAfter aliasing this after your favorite person, you can follow the\nexamples bellow :)\n\nNote that `**batteries are\nincluded** <https://www.facebook.com/episodeyang/videos/10101189402110434/>`__,\nand your mileage will vary.\n\nExample Usage\n-------------\n\nFirst install via ``pip`` (it's that simple!!)\n\n.. code-block:: bash\n\n    pip install dave\n\nSuppose you have the following folder structure\n\n::\n\n    \u251c\u2500\u2500 your_thesis\n    \u2514\u2500\u2500 MAML_tensorflow\n     \u00a0\u00a0 \u251c\u2500\u2500 README.md\n     \u00a0\u00a0 \u251c\u2500\u2500 experiment.yml\n     \u00a0\u00a0 \u251c\u2500\u2500 maml.py\n     \u00a0\u00a0 \u2514\u2500\u2500 models\n     \u00a0\u00a0  \u00a0\u00a0 \u251c\u2500\u2500 __init__.py\n     \u00a0\u00a0  \u00a0\u00a0 \u2514\u2500\u2500 mlp.py\n\nwhere the experiment.yml file looks like this:\n\n.. code-block:: yaml\n\n    %YAML 1.2\n    ---\n    config:\n      max_concurrent: 10\n    env:\n      PYTHONPATH: test_directory\n    run: |\n      {env} python maml_bradly.py {args}\n    default_args:\n      npts: 100\n      num_epochs: 70000\n      num_tasks: 10\n      num_grad_steps: 1\n      num_points_sampled: 10\n      fix_amp: False\n    batch_args: # use good typing convention here\n      - num_tasks: 10\n        num_grad_steps: 1\n        num_points_sampled: 10\n      - num_tasks: 10\n        num_grad_steps: 4\n        num_points_sampled: 20\n    tmp:\n      - last_run: 10\n\nNow under the project root, you can just run\n\n.. code-block:: bash\n\n    dave --config-file \"escher.py\"\n\nand it will automatically run the experiment twice, using the arguments\nin the ``batch_args`` field of the Yaml configuration file. ## To\nDevelop\n\nfirst download from github. Then under project folder, run (you also\nneed to install the packages).\n\n.. code-block:: bash\n\n    make dev test\n\nBucket List\n~~~~~~~~~~~\n\n-  [ ] Allow extensions\n-  [ ] work on windows\n-  [ ] allow env files\n-  [ ] daemon mode\n\nHappy Doing Science!\n--------------------\n\n.. figure:: https://github.com/episodeyang/escher-cli/blob/master/figures/phd092316s.gif?raw=true\n   :alt: one-more-experiment\n\n   one-more-experiment\n\n\n",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/episodeyang/escher-cli",
        "keywords": "escher,escher-cli,experiment,experimentation,script runner,deep learning,machine learning",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "escher-cli",
        "package_url": "https://pypi.org/project/escher-cli/",
        "platform": "",
        "project_url": "https://pypi.org/project/escher-cli/",
        "project_urls": {
            "Homepage": "https://github.com/episodeyang/escher-cli"
        },
        "release_url": "https://pypi.org/project/escher-cli/0.0.3/",
        "requires_dist": [
            "Click",
            "click-default-group",
            "params-proto",
            "pathos",
            "pathlib",
            "munch",
            "ruamel.yaml"
        ],
        "requires_python": "",
        "summary": "A command line utility that runs your command line scripts from a yaml script",
        "version": "0.0.3"
    },
    "last_serial": 3549044,
    "releases": {
        "0.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7bc337d3baf8c2850c79719712258ee3",
                    "sha256": "0d82efeee636596b1927de5889be6b63ed89f1caf82974a55a290c116f3f1c72"
                },
                "downloads": -1,
                "filename": "escher_cli-0.0.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "7bc337d3baf8c2850c79719712258ee3",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 8066,
                "upload_time": "2018-01-15T23:16:46",
                "url": "https://files.pythonhosted.org/packages/ce/10/6dd4e6c6674af81792508ef396e1db6f10a7483442c7b90c2e64d82b4e7a/escher_cli-0.0.0-py3-none-any.whl"
            }
        ],
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a313fbc955896169f5acac20539e65f2",
                    "sha256": "d8d586e825188c8776dc7e4fd6f5955018ca0900fbcf93df99e60a10f8432585"
                },
                "downloads": -1,
                "filename": "escher_cli-0.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a313fbc955896169f5acac20539e65f2",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 10936,
                "upload_time": "2018-02-03T19:31:05",
                "url": "https://files.pythonhosted.org/packages/1f/6c/d6b4807ac50d33ad01af1c3266a5252412e438f150a2a53159b1232069df/escher_cli-0.0.1-py3-none-any.whl"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "8a29e603726952b1ba91536538176fca",
                    "sha256": "916d80f9a417562c9fc8ca7be2ab866ef66811fb7af76344841763d6c5465174"
                },
                "downloads": -1,
                "filename": "escher_cli-0.0.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "8a29e603726952b1ba91536538176fca",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 18011,
                "upload_time": "2018-02-03T19:34:19",
                "url": "https://files.pythonhosted.org/packages/08/2c/5ede0eba85a5891fc40f1717a56caae1d8a6b017c4dbeb78537867d59a0e/escher_cli-0.0.2-py3-none-any.whl"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "8ee8bdac77e0d4fc02a438da1e9e8679",
                    "sha256": "9acf908dbdb76f365ac21ce745e4b32326b7c09f35d95bd54412ac2611047453"
                },
                "downloads": -1,
                "filename": "escher_cli-0.0.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "8ee8bdac77e0d4fc02a438da1e9e8679",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 17926,
                "upload_time": "2018-02-03T19:42:35",
                "url": "https://files.pythonhosted.org/packages/45/da/b286277458fecf59b8a19757168b0a2bcf2eef457beccab99fb4d80e4207/escher_cli-0.0.3-py3-none-any.whl"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "8ee8bdac77e0d4fc02a438da1e9e8679",
                "sha256": "9acf908dbdb76f365ac21ce745e4b32326b7c09f35d95bd54412ac2611047453"
            },
            "downloads": -1,
            "filename": "escher_cli-0.0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "8ee8bdac77e0d4fc02a438da1e9e8679",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 17926,
            "upload_time": "2018-02-03T19:42:35",
            "url": "https://files.pythonhosted.org/packages/45/da/b286277458fecf59b8a19757168b0a2bcf2eef457beccab99fb4d80e4207/escher_cli-0.0.3-py3-none-any.whl"
        }
    ]
}