{
    "info": {
        "author": "Jesse Galley",
        "author_email": "jesse@jessegalley.net",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 2 - Pre-Alpha",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Natural Language :: English",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.3",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5"
        ],
        "description": "===============================\ncmdfor\n===============================\n\n.. image:: https://img.shields.io/travis/jwgalley/cmdfor.svg\n        :target: https://travis-ci.org/jwgalley/cmdfor\n\n.. image:: https://img.shields.io/pypi/v/cmdfor.svg\n        :target: https://pypi.python.org/pypi/cmdfor\n\n\nnd for every line of input\n\n* Free software: MIT license\n* Documentation: (COMING SOON!) https://cmdfor.readthedocs.org.\n\nFeatures\n--------\n\nA shell utility (and package) which runs a command for every line of input.\n\nIt allows for spawning an arbitrary number of concurrent threads, and control\nover where to keep each commands output.\n\nIn my daily work, I have to run all manner of commands on huge batches of\nitems. These things are usually not CPU bound, so it makes sense to\nmultithread these tasks.\n\nThus, I find myself doing bash commands such as the following, which takes an\ninput file of items, splits it into equal(ish) parts, and then spawns a worker\nfor each part, all the while keeping granular logs and return codes::\n\n    lines=`wc -l domains.txt | awk '{print $1}'`; threads=10; split=$(((lines/threads)+1)); mkdir -p in out; split -d -l ${split} domains.txt in/part. ; ls in/ | while read -r f; do cat in/${f} | while read -r d; do host -t a \"${d}\" > out/${d} 2>&1; echo -e \"${d}\\t$?\"; done > log.${f} & echo ${!}; done > pids\n\nThat gets pretty tiring to type all the time. Why not use ``xargs -P`` you say?\nWell that works perfectly fine for cases where I don't need to make very\ncomplicated commands, and don't need to log all return codes. Maybe I can do\nall of that with ``xargs``, but I wanted to make this anyway as a learning\nexperience.\n\nHow-To\n------\n\nThe program can take input from STDIN or from a file passed with the -i option.\n\nAll arguments that aren't options are considered the subcommand to run. All\nwildcards ``{}`` are replaced with the corresponding positional field from the\ninput data.\n\nTo delete a list of files, basically the same behaviour as xargs::\n\n    cat files.txt | cmdfor rm {}\n\nTo run the fictional command ``imaplogin`` for every line of a csv that\ncontains <email>,<password> fields, logging each individual command's output\nto an file in the directory ./out::\n\n    cat email_users.csv | cmdfor -d, -o ./out -- imaplogin -u {} -p {}\n\nTo look up the IP addresses of a huge amount of hostnames, using 10 concurrent\nthreads, and storing each individual commands stdout and stderr in seperate\nfiles in the directory ./results, with each file being named after the hostname\non which the query was performed::\n\n    cat hostnames.txt | cmdfor -t 10 -Eo ./results -l 1 -- host -t a {}\n\n\nTo-Do\n-----\n\n1. Come up with a real test case. Since this is a shell utility and really only\ndeals with shell subcommands, I don't know what will work and what won't on\ntravis.ci (can I run a shell command there?)\n2. By default, it suppresses all output from subprocesses, and writes a message\nto STDOUT for each process spawn and reap. This output is too verbose for the\ndefault behaviour, and so it should be toggled with -v. The default should be\nquitier and simpler. Perhaps just the returncodes of each task.\n3. Refactoring some stuff to be a little less messy. The function signatures\nare huge, and there are messages generated in odd places. I think it would be\nbetter to pass a context object.\n\n\n\n\n\n\nHistory\n-------\n\n2018-04-09  v0.1.0  initial release, still need to do tests and docs\n2018-04-10  v0.1.5  fixing import issues and removing <2.7 support\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/jwgalley/cmdfor",
        "keywords": "",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "cmdfor",
        "package_url": "https://pypi.org/project/cmdfor/",
        "platform": "",
        "project_url": "https://pypi.org/project/cmdfor/",
        "project_urls": {
            "Homepage": "https://github.com/jwgalley/cmdfor"
        },
        "release_url": "https://pypi.org/project/cmdfor/0.1.5/",
        "requires_dist": [
            "click"
        ],
        "requires_python": "",
        "summary": "Run cmd for every line of input",
        "version": "0.1.5"
    },
    "last_serial": 3753095,
    "releases": {
        "0.1.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b193058b8987e90d47e27bdc052388f3",
                    "sha256": "c25c8dcce2a15983baaa2179c51f3b07226c9830eef4c940788cf7ef12a76ab5"
                },
                "downloads": -1,
                "filename": "cmdfor-0.1.5-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b193058b8987e90d47e27bdc052388f3",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 10438,
                "upload_time": "2018-04-10T17:50:48",
                "url": "https://files.pythonhosted.org/packages/e4/56/0098400caed87faf87d23be67269d4c4e4a00d1a2a193c7ab91d32c76318/cmdfor-0.1.5-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "54a9e158a4cdd828b5c3a434c60ef9d0",
                    "sha256": "cd73631799e73e30513c046337e3d401be7bd0488f96332e21b77f5738ab0b9c"
                },
                "downloads": -1,
                "filename": "cmdfor-0.1.5.tar.gz",
                "has_sig": false,
                "md5_digest": "54a9e158a4cdd828b5c3a434c60ef9d0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 33511,
                "upload_time": "2018-04-10T17:50:49",
                "url": "https://files.pythonhosted.org/packages/e5/a6/c5a6d066fbfb0dcaca385d7041a33f8a84383ab8f542d9b672036223003c/cmdfor-0.1.5.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "b193058b8987e90d47e27bdc052388f3",
                "sha256": "c25c8dcce2a15983baaa2179c51f3b07226c9830eef4c940788cf7ef12a76ab5"
            },
            "downloads": -1,
            "filename": "cmdfor-0.1.5-py2.py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "b193058b8987e90d47e27bdc052388f3",
            "packagetype": "bdist_wheel",
            "python_version": "py2.py3",
            "requires_python": null,
            "size": 10438,
            "upload_time": "2018-04-10T17:50:48",
            "url": "https://files.pythonhosted.org/packages/e4/56/0098400caed87faf87d23be67269d4c4e4a00d1a2a193c7ab91d32c76318/cmdfor-0.1.5-py2.py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "54a9e158a4cdd828b5c3a434c60ef9d0",
                "sha256": "cd73631799e73e30513c046337e3d401be7bd0488f96332e21b77f5738ab0b9c"
            },
            "downloads": -1,
            "filename": "cmdfor-0.1.5.tar.gz",
            "has_sig": false,
            "md5_digest": "54a9e158a4cdd828b5c3a434c60ef9d0",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 33511,
            "upload_time": "2018-04-10T17:50:49",
            "url": "https://files.pythonhosted.org/packages/e5/a6/c5a6d066fbfb0dcaca385d7041a33f8a84383ab8f542d9b672036223003c/cmdfor-0.1.5.tar.gz"
        }
    ]
}