{
    "info": {
        "author": "David R. MacIver",
        "author_email": "david@drmaciver.com",
        "bugtrack_url": null,
        "classifiers": [
            "Operating System :: POSIX",
            "Operating System :: Unix",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7"
        ],
        "description": "====\nEach\n====\n\nEach is a shell command for running robust, parallel, for loops with good feedback mechanisms.\n\nIt's optimised for \"medium scale\" experiments, which are a bit too small and\nad hoc to run on a proper cluster (or maybe you just don't *have* a proper cluster. I don't),\nbut are slow enough that restarting them from scratch would be painful and, ideally,\nparallel enough that if you happen to have a sixteen core server lying around to run them on you really want to be using all sixteen of those cores.\n\nPrimary usage of each is a more robust version of the following bash for loop:\n\n.. code-block:: bash\n\n    for f in \"$source\"/* ; do\n        DEST=\"$destination\"/$(basename $f)\n        mkdir -p \"$DEST\"\n        \"$command\" < \"$f\" > \"$DEST/out\" 2> \"$DEST/err\"\n        echo $? > \"$DEST/status\"\n    done\n\nThe same could be written using each as follows:\n\n.. code-block:: bash\n\n\teach \"$source\" \"$command\" --destination=\"$destination\"\n\nAs well as being shorter and more readable, writing this with each gets you:\n\n1. Automatic parallelism. You can control how many processes are run with ``--processes=n``,\n   but it defaults to using all but one of the cores available (or one on a single core machine).\n2. Automatic resume - if each dies, when it next starts up it will resume from where it left off.\n3. Feedback on progress, with good predictive analytics about when the process will finish (still a work in progress but the basics are there).\n\nLater you will also get good logic for retrying errors, but I haven't written that bit yet.\n\nEach is still a bit early days, so it likely has some rough edges, but it's well tested and has been making my life vastly better already.\n\n-----\nUsage\n-----\n\nUsage is:\n\n.. code-block:: bash\n\n    each some-input-directory 'some command to run' --destination=\"output directory\"\n\nCommands can be arbitrary shell commands (and will be run by ``$SHELL -c 'some command to run'`` by default).\n\nBy default, the file's contents will be passed to the child process's stdin. If you want to pass the file by name, you can use the special string `{}`.\nIf you do, the file to be processed will be substituted for it (with its absolute path name) and stdin will be empty.\n\nMore advanced usage options are available from ``each --help``.\n\n--------------------------------\nFrequently Anticipated Questions\n--------------------------------\n\n~~~~\nWhy?\n~~~~\n\nI have a bunch of experiments that are basically \"run this long running task on\neach of these files\" with the tasks having varying degrees of flakiness, and I\nkept finding myself writing bad versions of this, so I thought I would solve\nthe problem once and for all.\n\nMain features over the bash loop version:\n\n1. You don't risk learning how to write more bash than you want to.\n2. It resumes from where it left off if you kill it.\n3. Automatic parallelism\n4. You get a cool progress bar.\n5. When I get around to writing better retry features you'll get those for free.\n\n~~~~~~~~~~~~~~~~~~~~\nHow do I install it?\n~~~~~~~~~~~~~~~~~~~~\n\n.. code-block:: bash\n\n  pip install each\n\n~~~~~~~~~~~~~~~~~~~~\nWhat does it run on?\n~~~~~~~~~~~~~~~~~~~~\n\nProbably anything unixy. I've developed and tested it on Linux (including WSL, the Windows\n10 Linux Subsystem), but it's likely to work unmodified on OSX. It's unlikely to work on\nWindows. I'm not against Windows support if someone wants to contribute it, but I won't\nbe writing it myself and it's a fairly unixy sort of tool.\n\n\n~~~~~~~~~~~~~~~~~~\nShould I use this?\n~~~~~~~~~~~~~~~~~~\n\nEh, maybe. I'm finding it pretty helpful but it may be very idiosyncratic to my\nusage.\n\nIf you try it and it doesn't work for you, file an issue or make a PR.\nI'm happy for it to be generally useful but I don't plan to sink a huge amount\nof time into supporting it.\n\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nWill you make it work on Python 2?\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nNo.\n\n\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nWill you release it under a more permissive license?\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nAlso no.\n\n\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nI don't like these answers. What should I use instead?\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nI dunno. Maybe `bashreduce <https://github.com/erikfrey/bashreduce>`_?",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/DRMacIver/each/",
        "keywords": "",
        "license": "GPL v3",
        "maintainer": "",
        "maintainer_email": "",
        "name": "each",
        "package_url": "https://pypi.org/project/each/",
        "platform": "",
        "project_url": "https://pypi.org/project/each/",
        "project_urls": {
            "Homepage": "https://github.com/DRMacIver/each/"
        },
        "release_url": "https://pypi.org/project/each/0.0.6/",
        "requires_dist": null,
        "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*",
        "summary": "A tool for running programs on many inputs",
        "version": "0.0.6"
    },
    "last_serial": 4805835,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "55a20296b9051e38e41e4d9440e3177f",
                    "sha256": "1d4f55ede61b62280a1e653cc52c2ad7828a248138ecd79af53ecf62e6b30d76"
                },
                "downloads": -1,
                "filename": "each-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "55a20296b9051e38e41e4d9440e3177f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3131,
                "upload_time": "2018-08-11T10:59:39",
                "url": "https://files.pythonhosted.org/packages/a4/29/5f1f1bb199bcdcc6ba0206e4d6ff52fd148519c3dfe8314138a1751bb4fd/each-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "65fc782d164bb70d272047beccd78e92",
                    "sha256": "f3bbd00d964fa179f674d614a64d70d921a459fe09aa7290267eb91170213514"
                },
                "downloads": -1,
                "filename": "each-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "65fc782d164bb70d272047beccd78e92",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3133,
                "upload_time": "2018-08-11T11:24:35",
                "url": "https://files.pythonhosted.org/packages/5c/c8/72909d075cdd0076690975d17709eb30f5941e57bdf0aa0d2032be38d24d/each-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a28985b313dcd3728fa2273b8ae7f2de",
                    "sha256": "a79df359be7768fe042b68af9d390695d06630d43d109681a8a7e5b5cf3e6afd"
                },
                "downloads": -1,
                "filename": "each-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "a28985b313dcd3728fa2273b8ae7f2de",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3186,
                "upload_time": "2018-08-11T11:36:50",
                "url": "https://files.pythonhosted.org/packages/65/7f/c3d6022e04a5b25e0010ec5aee95d15a5bad114d44940494486915d422a5/each-0.0.3.tar.gz"
            }
        ],
        "0.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "19267493478fffab319124f8dd1c7537",
                    "sha256": "9fc228a77ad77b85c9b4fe95cfa0e35d2f2275781c1bd4d837a9b3155a384917"
                },
                "downloads": -1,
                "filename": "each-0.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "19267493478fffab319124f8dd1c7537",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3199,
                "upload_time": "2018-08-11T11:47:41",
                "url": "https://files.pythonhosted.org/packages/35/7c/67e94cbc964931716a645ce79216c9927fa46647b8ae656760e8e856a130/each-0.0.4.tar.gz"
            }
        ],
        "0.0.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3afdd0a8111ff9a931c93d02dd24a75b",
                    "sha256": "afb3433b3fe8df7220a7487e70c04c68926b9670d5475cfabd9e339228a0ee6e"
                },
                "downloads": -1,
                "filename": "each-0.0.5.tar.gz",
                "has_sig": false,
                "md5_digest": "3afdd0a8111ff9a931c93d02dd24a75b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5094,
                "upload_time": "2018-08-11T16:44:28",
                "url": "https://files.pythonhosted.org/packages/bc/c3/aa9efcda952b6f39e73d3ba834ef0534e8f058971ef00f9d9be63a1cc0a2/each-0.0.5.tar.gz"
            }
        ],
        "0.0.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "8a905d7c86237974a1f8ae04a46738b2",
                    "sha256": "52acb937c53e99131ee4aeb93b41a1c0266dd930bce9ab8e1acdd111ee7ea761"
                },
                "downloads": -1,
                "filename": "each-0.0.6.tar.gz",
                "has_sig": false,
                "md5_digest": "8a905d7c86237974a1f8ae04a46738b2",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*",
                "size": 10926,
                "upload_time": "2019-02-11T12:44:34",
                "url": "https://files.pythonhosted.org/packages/dc/e5/394d1cae938919f8d1da353383c54d5e77d8ba364a0b90ad0fd03e70991a/each-0.0.6.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "8a905d7c86237974a1f8ae04a46738b2",
                "sha256": "52acb937c53e99131ee4aeb93b41a1c0266dd930bce9ab8e1acdd111ee7ea761"
            },
            "downloads": -1,
            "filename": "each-0.0.6.tar.gz",
            "has_sig": false,
            "md5_digest": "8a905d7c86237974a1f8ae04a46738b2",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*",
            "size": 10926,
            "upload_time": "2019-02-11T12:44:34",
            "url": "https://files.pythonhosted.org/packages/dc/e5/394d1cae938919f8d1da353383c54d5e77d8ba364a0b90ad0fd03e70991a/each-0.0.6.tar.gz"
        }
    ]
}