{
    "info": {
        "author": "Tjelvar Olsson",
        "author_email": "tjelvar.olsson@jic.ac.uk",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "Architect jobs for running analyses\n===================================\n\n.. image:: https://badge.fury.io/py/jobarchitect.svg\n   :target: http://badge.fury.io/py/jobarchitect\n   :alt: PyPi package\n\n.. image:: https://readthedocs.org/projects/jobarchitect/badge/?version=latest\n   :target: http://jobarchitect.readthedocs.io/en/latest/?badge=latest\n   :alt: Documentation Status\n\n- Documentation: http://jobarchitect.readthedocs.io\n- GitHub: https://github.com/JIC-CSB/jobarchitect\n- PyPI: https://pypi.python.org/pypi/jobarchitect\n- Free software: MIT License\n\n\nOverview\n--------\n\nThis tool is intended to automate generation of scripts to run analysis on data\nsets. To use it, you will need a data set that has been created (or annotated)\nwith `dtool <https://github.com/JIC-CSB/dtool>`_.\nIt aims to help by:\n\n1. Removing the need to know where specific data items are stored in a data set\n2. Providing a means to split an analyses into several chunks (file based\n   parallelization)\n3. Providing a framework for seamlessly running an analyses inside a container\n\n\nDesign\n------\n\nThis project has two main components. The first is a command line tool named\n``sketchjob`` intended to be used by the end user. It is used to generate\nscripts defining jobs to be run. The second (``_analyse_by_ids``) is a command\nline tool that is used by the scripts generated by ``sketchjob``. The end user\nis not meant to make use of this second script directly.\n\n\nInstallation\n------------\n\nTo install the jobarchitect package.\n\n::\n\n    $ cd jobarchitect\n    $ python setup.py install\n\n\nUse\n---\n\nThe ``jobarchitect`` tool only works with \"smart\" tools.\nA \"smart\" tool is a tool that understands `dtoolcore <https://github.com/JIC-CSB/dtoolcore>`_\ndatasets, has no positional command line arguments and supports the\nnamed arguments ``--dataset-path``, ``--identifier``, ``--output-directory``.\nThe tool should only process the dataset item specified by the identifier\nand write all output to the specified output directory.\n\nA dtool dataset can be created using `dtool <https://github.com/JIC-CSB/dtool>`_.\nBelow is some sample::\n\n    $ dtool new dataset\n    project_name [project_name]:\n    dataset_name [dataset_name]: example_dataset\n    ...\n\n    $ echo \"My example data\" > example_dataset/data/my_file.txt\n    $ datatool manifest update example_dataset/\n\nCreate an output directory::\n\n    $ mkdir output\n\nThen you can generate analysis run scripts with::\n\n    $ sketchjob my_smart_tool.py exmaple_dataset output/\n    #!/bin/bash\n\n    _analyse_by_ids \\\n      --tool_path=my_smart_tool.py \\\n      --input_dataset_path=example_dataset/ \\\n      --output_root=output/ \\\n      290d3f1a902c452ce1c184ed793b1d6b83b59164\n\nTry the script with::\n\n    $ sketchjob my_smart_tool.py exmaple_dataset output/ > run.sh\n    $ bash run.sh\n    $ cat output/first_image.png\n    290d3f1a902c452ce1c184ed793b1d6b83b59164  /private/var/folders/hn/crprzwh12kj95plc9jjtxmq82nl2v3/T/tmp_pTfc6/stg02d730c7-17a2-4d06-a017-e59e14cb8885/first_image.png\n\nUse with split\n--------------\n\nThe unix command ``split`` is a good way to divide the single large output (that concatenates many command invocations) produced by sketchjob into individual files. For example::\n\n    $ split -n 60 many_slurm_scripts.slurm all_slurm_scripts/submit_segment\n\nWorking with Docker\n-------------------\n\nBuilding a Docker image\n^^^^^^^^^^^^^^^^^^^^^^^\n\nFor the tests to pass, you will need to build an example Docker image, which\nyou do with the provided script::\n\n    $ bash build_docker_image.sh\n\nRunning code with the Docker backend\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nBy inspecting the script and associcated Docker file, you can get an idea of\nhow to build Docker images that can be used with the jobarchitect Docker\nbackend, e.g::\n\n    $ sketchjob scripts/my_smart_tool.py ~/junk/cotyledon_images ~/junk/output --backend=docker --image-name=jicscicomp/jobarchitect\n    #!/bin/bash\n\n    IMAGE_NAME=jicscicomp/jobarchitect\n    docker run  \\\n      --rm  \\\n      -v /Users/olssont/junk/cotyledon_images:/input_dataset:ro  \\\n      -v /Users/olssont/junk/output:/output  \\\n      -v /Users/olssont/sandbox/scripts:/scripts:ro \\\n      $IMAGE_NAME  \\\n      _analyse_by_ids  \\\n        --tool_path=/scripts/my_smart_tool.py \\\n        --input_dataset_path=/input_dataset  \\\n        --output_root=/output  \\\n        290d3f1a902c452ce1c184ed793b1d6b83b59164 09648d19e11f0b20e5473594fc278afbede3c9a4",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "https://github.com/JIC-CSB/jobarchitect/tarball/0.7.0",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/JIC-CSB/jobarchitect",
        "keywords": null,
        "license": "MIT",
        "maintainer": null,
        "maintainer_email": null,
        "name": "jobarchitect",
        "package_url": "https://pypi.org/project/jobarchitect/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/jobarchitect/",
        "project_urls": {
            "Download": "https://github.com/JIC-CSB/jobarchitect/tarball/0.7.0",
            "Homepage": "https://github.com/JIC-CSB/jobarchitect"
        },
        "release_url": "https://pypi.org/project/jobarchitect/0.7.0/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Tools for batching jobs and dealing with file paths",
        "version": "0.7.0"
    },
    "last_serial": 2959966,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6e1105618bb8c88da9a9636a6ddea192",
                    "sha256": "460c820318a7dde9ef8799aeb88e6fd43a3636d0c3308870ec12d3629ea5408e"
                },
                "downloads": -1,
                "filename": "jobarchitect-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "6e1105618bb8c88da9a9636a6ddea192",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4857,
                "upload_time": "2017-02-03T13:51:40",
                "url": "https://files.pythonhosted.org/packages/68/7a/e7b6554c9518e88cfaaffa5ef7e2b8b3d24034af91b049f65c13ca92f516/jobarchitect-0.1.0.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ca4775787268dbe43cb2233e4b825a16",
                    "sha256": "24771e3667bbb787148a108227b2d118c894a929bcf385799803f95e6ab912b5"
                },
                "downloads": -1,
                "filename": "jobarchitect-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ca4775787268dbe43cb2233e4b825a16",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6534,
                "upload_time": "2017-02-06T16:57:15",
                "url": "https://files.pythonhosted.org/packages/ef/6b/ef60666bb33541e91232dee55cd22f1c0ed89956c9041e279797bc5ded28/jobarchitect-0.2.0.tar.gz"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "683ea46b38ed5ab3d6e3744c248ed6fc",
                    "sha256": "0cef48d5935490390fe67dc35e85b54a44c50cd799f697b8950fc00cfad58469"
                },
                "downloads": -1,
                "filename": "jobarchitect-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "683ea46b38ed5ab3d6e3744c248ed6fc",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6557,
                "upload_time": "2017-02-07T17:32:41",
                "url": "https://files.pythonhosted.org/packages/51/50/59a13432d777e96fb642f73ed8bac0003dd2b1fd4b716653df474a34e6f3/jobarchitect-0.3.0.tar.gz"
            }
        ],
        "0.4.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1f4626d9910d8862a84ccad918b074a1",
                    "sha256": "beb169e61a86b172e633ed4cc32deb84727978c8b8e436e9e28512398e2fe636"
                },
                "downloads": -1,
                "filename": "jobarchitect-0.4.0.tar.gz",
                "has_sig": false,
                "md5_digest": "1f4626d9910d8862a84ccad918b074a1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7100,
                "upload_time": "2017-06-08T17:32:27",
                "url": "https://files.pythonhosted.org/packages/90/7b/471e3c950e42886667effc5bd760e7655c6a03a9d86302919e18c0667190/jobarchitect-0.4.0.tar.gz"
            }
        ],
        "0.5.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "32c13111fe2a8a5eade46a4f1af89f08",
                    "sha256": "c8c4427c89822ab7ab63d8a18967e3d200eb2ac0fa9c79838c377fbd84d544a1"
                },
                "downloads": -1,
                "filename": "jobarchitect-0.5.0.tar.gz",
                "has_sig": false,
                "md5_digest": "32c13111fe2a8a5eade46a4f1af89f08",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6785,
                "upload_time": "2017-06-15T10:05:46",
                "url": "https://files.pythonhosted.org/packages/e4/ba/13685cbb8448c2b8913113920cc902989ca9b0515ec56a5273f6f3010965/jobarchitect-0.5.0.tar.gz"
            }
        ],
        "0.6.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9f895f48b6a66545c754c00e1929b0f7",
                    "sha256": "ec0ed69a6c2dd97087f18a68d86797c1f8d0f22ee401e20eeee723fd08d4a94d"
                },
                "downloads": -1,
                "filename": "jobarchitect-0.6.0.tar.gz",
                "has_sig": false,
                "md5_digest": "9f895f48b6a66545c754c00e1929b0f7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6914,
                "upload_time": "2017-06-15T15:23:54",
                "url": "https://files.pythonhosted.org/packages/20/26/c760534546c5d3f231614888faeeabc56f24a6469d0dc00f561c65dc3fef/jobarchitect-0.6.0.tar.gz"
            }
        ],
        "0.7.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "cceae256726674693cb0dcc58bdb377a",
                    "sha256": "0c79ee97789804a45f0bd83e3914a90427602b23145daff244e6b417bbf2a662"
                },
                "downloads": -1,
                "filename": "jobarchitect-0.7.0.tar.gz",
                "has_sig": false,
                "md5_digest": "cceae256726674693cb0dcc58bdb377a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7395,
                "upload_time": "2017-06-19T16:16:04",
                "url": "https://files.pythonhosted.org/packages/e5/5e/07669c950287008cec4bb799ee01e726fa316a28053c024982b8d2a32b9e/jobarchitect-0.7.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "cceae256726674693cb0dcc58bdb377a",
                "sha256": "0c79ee97789804a45f0bd83e3914a90427602b23145daff244e6b417bbf2a662"
            },
            "downloads": -1,
            "filename": "jobarchitect-0.7.0.tar.gz",
            "has_sig": false,
            "md5_digest": "cceae256726674693cb0dcc58bdb377a",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 7395,
            "upload_time": "2017-06-19T16:16:04",
            "url": "https://files.pythonhosted.org/packages/e5/5e/07669c950287008cec4bb799ee01e726fa316a28053c024982b8d2a32b9e/jobarchitect-0.7.0.tar.gz"
        }
    ]
}