{
    "info": {
        "author": "(the author)",
        "author_email": "mdilligaf@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: BSD License",
            "Natural Language :: English",
            "Operating System :: OS Independent",
            "Programming Language :: Python",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Topic :: Software Development :: Libraries :: Python Modules"
        ],
        "description": "\nFenpei\n===============================\n\nThis little tool helps in scheduling, tracking and aggregating calculations and their results. It forms the step that brings you from 'a directory with working code for a job' to 'running dozens of jobs and getting results easily'.\n\n    pip install fenpei\n\nThis is intended to be used to run multiple intensive computations on a (linux) cluster. At present, it assumes a shared file system on the cluster.\n\nIt takes a bit of work to integrate with your situation but it is very flexible and should make your life easier after setting it up. Some features:\n\n* Jobs are created in Python files, making it short and extremely flexible.\n* It uses a command line interface (some shell experience required) to easily start, stop or monitor jobs.\n* Easy to use with existing code and easily reproducible, since it works by creating isolated job directories.\n* Can replaces scheduling queue functionality and start jobs through ssh, or can work with existing systems (slurm and qsum included, others implementable).\n* Flexibility for caching, preparation and result extraction.\n* Uses multi-processing and can easily use caching for greater performance, and symlinks to save space.\n\nNote that:\n\n* You will have to write Python code for your specific job, as well as any analysis or visualization for the extracted data.\n* Except for status monitoring mode, it derives the state on each run, it doesn't keep a database that can get outdated or corrupted.\n\nOne example to run reproducible jobs with Fenpei (there are many ways):\n\n* Make a script that runs your code from source to completion for one set of parameters.\n* Subclass the ShJobSingle job and add all the files that you need in `get_nosub_files`.\n* Replace all the parameters in the run script and other config files by `{{ some_param_name }}`. Add these files to `get_sub_files`.\n* Make a Python file (example below) for each analysis you want to run, and fill in all the `some_param_name` with the appropriate values.\n* From a shell, use `python your_jobfile.py -s` to see the status, then use other flags for more functionality (see below).\n* Implement `is_complete` and `result` in your job (and `crash_reason` if you want `-t`) (others can be overridden too, if you require special behaviour).\n* Add analysis code to your job file if you want to visualize the results.\n\nExample file to generate jobs::\n\n    def generate_jobs():\n        for alpha in [0.01, 0.10, 1.00]:\n            for beta in range(0, 41):\n                dict(name='a{0:.2f}_b{1:d}'.format(alpha, beta), subs=dict(\n                    alpha=alpha,\n                    beta=beta,\n                    gamma=5,\n                    delta='yes'\n                ), use_symlink=True)\n\n    def analyze(queue):\n        results = queue.compare_results(('J', 'init_vib', 'init_rot',))\n        # You now have the results for all jobs, indexed by the above three parameters.\n        # Visualization is up to you, and will be run when the user adds -x\n\n    if __name__ == '__main__':\n        jobs = create_jobs(JobCls=ShefJob, generator=generate_jobs(), default_batch=splitext(basename(__file__))[0])\n        queue = SlurmQueue(partition='example', jobs=jobs, summary_func=analyze)\n        queue.run_argv()\n\nThis file registers many jobs for combinations of alpha and beta parameters. You can now use the command line::\n\n    usage: results.py [-h] [-v] [-f] [-e] [-a] [-d] [-l] [-p] [-c] [-w WEIGHT]\n                      [-q LIMIT] [-k] [-r] [-g] [-s] [-m] [-x] [-t] [-j]\n                      [--jobs JOBS] [--cmd ACTIONS]\n\n    distribute jobs over available nodes\n\n    optional arguments:\n      -h, --help            show this help message and exit\n      -v, --verbose         more information (can be used multiple times, -vv)\n      -f, --force           force certain mistake-sensitive steps instead of\n                            failing with a warning\n      -e, --restart         with this, start and cleanup ignore complete\n                            (/running) jobs\n      -a, --availability    list all available nodes and their load (cache reload)\n      -d, --distribute      distribute the jobs over available nodes\n      -l, --list            show a list of added jobs\n      -p, --prepare         prepare all the jobs\n      -c, --calc            start calculating one jobs, or see -z/-w/-q\n      -w WEIGHT, --weight WEIGHT\n                            -c will start jobs with total WEIGHT running\n      -q LIMIT, --limit LIMIT\n                            -c will add jobs until a total LIMIT running\n      -k, --kill            terminate the calculation of all the running jobs\n      -r, --remove          clean up all the job files\n      -g, --fix             fix jobs, check cache etc (e.g. after update)\n      -s, --status          show job status\n      -m, --monitor         show job status every few seconds\n      -x, --result          run analysis code to summarize results\n      -t, --whyfail         print a list of failed jobs with the reason why they\n                            failed\n      -j, --serial          job commands (start, fix, etc) may NOT be run in\n                            parallel (parallel is faster but order of jobs and\n                            output is inconsistent)\n      --jobs JOBS           specify by name the jobs to (re)start, separated by\n                            whitespace\n      --cmd ACTIONS         run a shell command in the directories of each job\n                            that has a dir ($NAME/$BATCH/$STATUS if --s)\n\n    actions are executed (largely) in the order they are supplied; some actions\n    may call others where necessary\n\nPull requests, extra documentation and bug reports are welcome! It's Revised BSD-licensed so you can do many things.\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/mverleg/fenpei",
        "keywords": "",
        "license": "Revised BSD License (LICENSE.txt)",
        "maintainer": "",
        "maintainer_email": "",
        "name": "fenpei",
        "package_url": "https://pypi.org/project/fenpei/",
        "platform": "",
        "project_url": "https://pypi.org/project/fenpei/",
        "project_urls": {
            "Homepage": "https://github.com/mverleg/fenpei"
        },
        "release_url": "https://pypi.org/project/fenpei/2.7.2/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Distribution of tasks.",
        "version": "2.7.2"
    },
    "last_serial": 5508789,
    "releases": {
        "2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9f783308eca1ed836905ea5dabb9a8a9",
                    "sha256": "053a5eafaecced441d2e2186e470106b54fec4703c5793d8d749f946972504d1"
                },
                "downloads": -1,
                "filename": "fenpei-2.1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "9f783308eca1ed836905ea5dabb9a8a9",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 35101,
                "upload_time": "2016-10-03T21:34:57",
                "url": "https://files.pythonhosted.org/packages/46/26/ddc24c64d122bea9af7233c81caac55af5b9224a095fd93402aecd3ac9d1/fenpei-2.1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "5ff958c029f2f34857fdf0a360b250b8",
                    "sha256": "8f09f5917d692ec092bea2ceff6dfe4b003869e33934f2b4f9288288d1487d2b"
                },
                "downloads": -1,
                "filename": "fenpei-2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "5ff958c029f2f34857fdf0a360b250b8",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 28834,
                "upload_time": "2016-10-03T21:34:54",
                "url": "https://files.pythonhosted.org/packages/1c/7f/c198c706e52bdc6b22a84d1008b76f2f3ae8f22c7c2e76a70c5981ad9af1/fenpei-2.1.tar.gz"
            }
        ],
        "2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b5aab6eaceb04ac174808cf9d67ac735",
                    "sha256": "0ac93f676ea7db4cb491d4d7332ba23a9cf4d808ca502a64e28bdb79a3561f20"
                },
                "downloads": -1,
                "filename": "fenpei-2.2-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b5aab6eaceb04ac174808cf9d67ac735",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 36813,
                "upload_time": "2016-11-11T12:05:15",
                "url": "https://files.pythonhosted.org/packages/7a/fb/648f83c5e370636d1c6d15b25a114191ddbe96df6cf879798db62bba6a96/fenpei-2.2-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "658352f5c8aa830f0892ab69aa1390c4",
                    "sha256": "0c814f2d5cd610b7eb2cce5948d95b5f92671194e11b378efa252890f70e5ef8"
                },
                "downloads": -1,
                "filename": "fenpei-2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "658352f5c8aa830f0892ab69aa1390c4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 27724,
                "upload_time": "2016-11-11T12:05:12",
                "url": "https://files.pythonhosted.org/packages/08/9e/60d217acbae29a201fdf7048795f0ac9464353006a8ccd042423d3c1574a/fenpei-2.2.tar.gz"
            }
        ],
        "2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "104dffc6b93bb60b447fa8b911614ebb",
                    "sha256": "dad1805bcb40ca75d193293ed2f923bb1b42f87df9ca3e80bf07cdad3a7f93e7"
                },
                "downloads": -1,
                "filename": "fenpei-2.3-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "104dffc6b93bb60b447fa8b911614ebb",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 37170,
                "upload_time": "2016-11-17T12:06:52",
                "url": "https://files.pythonhosted.org/packages/26/c5/f1574f1f2f2490eb1482bea1f368f290d8c927cb7561d6ece5c6edf8b38c/fenpei-2.3-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "6d74345b237bd9af603ea33c0d0d2037",
                    "sha256": "547ee2ac36e0cf222fdf9061236f4ac45b7a51f8ed2796bfe88aaf549fef6245"
                },
                "downloads": -1,
                "filename": "fenpei-2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "6d74345b237bd9af603ea33c0d0d2037",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 28179,
                "upload_time": "2016-11-17T12:06:49",
                "url": "https://files.pythonhosted.org/packages/63/52/6901423e5b030dbc2d136b0ef727f385ab629e1ad2a244e5aed65723b6d3/fenpei-2.3.tar.gz"
            }
        ],
        "2.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3c013b47e3f097b7c472a66b9e464496",
                    "sha256": "d0d1c53fd0d7964d32e2e5de9c5786fd8b9293b6d1a77f680efeb42054752426"
                },
                "downloads": -1,
                "filename": "fenpei-2.4-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "3c013b47e3f097b7c472a66b9e464496",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 37812,
                "upload_time": "2017-01-21T11:29:39",
                "url": "https://files.pythonhosted.org/packages/a1/9b/73253da25c1cd65887218a579d9392e28e55773413b4744b18ce346e3584/fenpei-2.4-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "006a94481df53449bc5bf89e201dd2d1",
                    "sha256": "648f4cbed777534cec6d4e11a290020ad8bf3b9cb9b0b83b123aae86d8115745"
                },
                "downloads": -1,
                "filename": "fenpei-2.4.tar.gz",
                "has_sig": false,
                "md5_digest": "006a94481df53449bc5bf89e201dd2d1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 28564,
                "upload_time": "2017-01-21T11:29:37",
                "url": "https://files.pythonhosted.org/packages/50/d1/1c7eb46edc864216c05bc2cbad8365696edbe667c4cc293b8457836c457e/fenpei-2.4.tar.gz"
            }
        ],
        "2.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "846a54a05db6cad5d713fe8dcc927fca",
                    "sha256": "b6c7d046b8ef577ecf60d6c48a010013ae55029a8705d7853489653750e5da06"
                },
                "downloads": -1,
                "filename": "fenpei-2.5.tar.gz",
                "has_sig": false,
                "md5_digest": "846a54a05db6cad5d713fe8dcc927fca",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 30087,
                "upload_time": "2018-04-01T18:32:25",
                "url": "https://files.pythonhosted.org/packages/7f/88/88ac9c9f6cebf11c952cd9f173441b007ff15c2d14a16ceb5e2bb346564f/fenpei-2.5.tar.gz"
            }
        ],
        "2.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ab1abe348f0777b53463a5e0fb794fde",
                    "sha256": "2e75cec3f2ca2426a5b57318ac8c8b2b94c16b6257c771f717530f929a3f3570"
                },
                "downloads": -1,
                "filename": "fenpei-2.6.tar.gz",
                "has_sig": false,
                "md5_digest": "ab1abe348f0777b53463a5e0fb794fde",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 32378,
                "upload_time": "2018-05-20T09:48:35",
                "url": "https://files.pythonhosted.org/packages/a2/60/60c6ae9e7c6d2d8ad0c4a6adac3edb3258531ccef9c4ebb44fb210aeda9c/fenpei-2.6.tar.gz"
            }
        ],
        "2.6.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "25c543543025fd3bc19653925e42364e",
                    "sha256": "f00378c44f178c64aa2ccd4a6eeef3d122443cd1d4ceb89fdad9675489190132"
                },
                "downloads": -1,
                "filename": "fenpei-2.6.1.tar.gz",
                "has_sig": false,
                "md5_digest": "25c543543025fd3bc19653925e42364e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 32389,
                "upload_time": "2018-05-22T20:21:39",
                "url": "https://files.pythonhosted.org/packages/5e/16/2baf37d64bcfdf4c6b0b056a06a5eeb8bc0ca20ef0154b57b802878ae9b9/fenpei-2.6.1.tar.gz"
            }
        ],
        "2.6.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d08d36ab13cfa4d970008176710cc629",
                    "sha256": "6610baf9b2ef1c3af4f2309404da1d1e6e7292721536a275f155345ad04b4c83"
                },
                "downloads": -1,
                "filename": "fenpei-2.6.2.tar.gz",
                "has_sig": false,
                "md5_digest": "d08d36ab13cfa4d970008176710cc629",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 32364,
                "upload_time": "2018-05-26T15:24:02",
                "url": "https://files.pythonhosted.org/packages/b8/e8/7b0d1c37ac0ce8e025143d3badc018343799f4176398932374dbe57208e6/fenpei-2.6.2.tar.gz"
            }
        ],
        "2.6.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d798df5a3924baabcc01e8dffc8db35a",
                    "sha256": "21fb76d68a11cc529de9912a48522250da44624b4e388a67536bd87a8e3da14d"
                },
                "downloads": -1,
                "filename": "fenpei-2.6.3.tar.gz",
                "has_sig": false,
                "md5_digest": "d798df5a3924baabcc01e8dffc8db35a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 32681,
                "upload_time": "2018-05-26T15:30:10",
                "url": "https://files.pythonhosted.org/packages/fe/01/81b13d87a6493051cb4576e08d8deb4f63acb353b703e17150b1b12d694b/fenpei-2.6.3.tar.gz"
            }
        ],
        "2.7.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c7297c006783389f82394920b04586bb",
                    "sha256": "e056382e7702eb3fa77c83f9fc439fbbe35b16463df5b0bbe1f9a38d59ed4f45"
                },
                "downloads": -1,
                "filename": "fenpei-2.7.0.tar.gz",
                "has_sig": false,
                "md5_digest": "c7297c006783389f82394920b04586bb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 32855,
                "upload_time": "2018-06-09T14:13:16",
                "url": "https://files.pythonhosted.org/packages/bc/dd/46ee2d29ab4ea3477f7c25e23f4d293f0030b30ceb69e6b5d54ea7804ce9/fenpei-2.7.0.tar.gz"
            }
        ],
        "2.7.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "959057c5718de36e5cf6a51cb5690044",
                    "sha256": "67a8d9105766e5ee868c63db74ff5122565c45c3737df05cd71b7a33c35ae2d6"
                },
                "downloads": -1,
                "filename": "fenpei-2.7.1.tar.gz",
                "has_sig": false,
                "md5_digest": "959057c5718de36e5cf6a51cb5690044",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 32903,
                "upload_time": "2018-06-09T14:22:54",
                "url": "https://files.pythonhosted.org/packages/56/dd/9dac95282cb8fa35abd63b7c64bf7d0988751210c23df8b61b49b604da25/fenpei-2.7.1.tar.gz"
            }
        ],
        "2.7.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "00602a9cf54fdfeb96c5ca278d54b0de",
                    "sha256": "1f51064d6c4aeda0f4ab0d223df0774dfac03850af47ef8843898899c0a394b6"
                },
                "downloads": -1,
                "filename": "fenpei-2.7.2.tar.gz",
                "has_sig": false,
                "md5_digest": "00602a9cf54fdfeb96c5ca278d54b0de",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 32382,
                "upload_time": "2019-07-09T19:54:30",
                "url": "https://files.pythonhosted.org/packages/b6/c8/a0a25678435d83944a9e903dda7941a52c2b52c3c14dd4f9198fd3e739f7/fenpei-2.7.2.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "00602a9cf54fdfeb96c5ca278d54b0de",
                "sha256": "1f51064d6c4aeda0f4ab0d223df0774dfac03850af47ef8843898899c0a394b6"
            },
            "downloads": -1,
            "filename": "fenpei-2.7.2.tar.gz",
            "has_sig": false,
            "md5_digest": "00602a9cf54fdfeb96c5ca278d54b0de",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 32382,
            "upload_time": "2019-07-09T19:54:30",
            "url": "https://files.pythonhosted.org/packages/b6/c8/a0a25678435d83944a9e903dda7941a52c2b52c3c14dd4f9198fd3e739f7/fenpei-2.7.2.tar.gz"
        }
    ]
}