{
    "info": {
        "author": "Barcelona Biomedical Genomics Lab",
        "author_email": "bbglab@irbbarcelona.org",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: Apache Software License",
            "Programming Language :: Python :: 3"
        ],
        "description": "QMap\n====\n\n**QMap** is a tool aimed to run a collection of similar jobs quickly and\neasily in parallel.\nIt can run standalone or using different HPC schedulers (Slurm, Sun Grid\nEngine and LFS).\n\n**QMap** contains 5 different tools:\n\n- *run*: execute commands with extended resources\n- *template*: create a jobs map file\n- *submit*: submit jobs from a map file\n- *reattach*: reattach to a previous QMap execution\n- *info*: explore the metadata of your jobs\n\n\n.. warning:: **QMap** makes use of the shell using the *subprocess* module\n   of the Python standard library. The shell is invoked making use of\n   ``shell=True`` which can lead to shell injections vulnerabilities.\n   More details in https://docs.python.org/3/library/subprocess.html#security-considerations\n\n\nDocumentation in: https://qmap.readthedocs.io/en/latest/\n\nTools\n-----\n\n**qmap run**\n   Execute a command with more resources maintaining your working environment\n\n   .. code:: bash\n\n      qmap run -m <memory> -c <cores>  \"<command>\"\n\n**qmap template**\n   Create a `jobs map file`_ that works with **qmap submit**.\n\n   .. code:: bash\n\n      qmap template \"<command with wildcards>\" -f <jobs map file>\n\n   The file created uses the current loaded Easy Build modules\n   and the current conda environment as jobs pre-commands [#precmd]_\n   if not explicitly provided.\n\n   The job commands are all the combinations that result of the expansion of:\n\n   :{{list,of,items}}: comma separated list of items\n   :{{file}}: all lines in file\n   :`*`, ?, [x-y]: wildcards in Python's glob module\n\n   Wildcards of the format ``{{...}}`` are expanded in a first phase\n   and glob wildcards are expanded later on.\n\n   As additional feature, any of the above mentioned groups can be named\n   ``{{?<name>:...}}`` and replaced anywhere using ``{{?=<name>}}``.\n\n   .. note::\n\n      To name glob wildcards they should be solely in the group.\n      E.g. ``{{?myfiles:*}}``\n\n\n**qmap submit**\n   Execute all jobs from a `jobs map file`_\n\n   .. code:: bash\n\n      qmap submit -m <memory> -c <cores>  <jobs map file> --logs <logs folder> --max-running <#>\n\n   ``qmap submit`` has been implemented to submit a set of jobs to a cluster for execution\n   and control them.\n   It acts as a layer between the workload manager and the user preventing she/he\n   from submitting a huge number of jobs at once (potentially blocking future users).\n   The number of jobs that can be submitted to the workload manager is controlled by the\n   *--max-running* flag.\n\n   .. warning::\n\n      If ``qmap submit`` is closed, jobs that have not been submitted to the\n      workload manager will never be.\n      Thus, it is recommended to run it inside a **screen**.\n\n   In addition, in the folder indicated to store the logs with the *--logs* flag\n   the user can find important information about each job execution as well as\n   the logs from STDOUT and STDERR.\n\n   Another feature of this tool is the possibility to group your jobs with the *--grouping*\n   option. This option uses the value passed as the number of commands that fit in each job.\n   Thus, several commands can be executed as part of the same job, one after another.\n   This option can be interesting for \"small\" jobs as they use the same allocation.\n   If any of the commands fail, the associated job will fail.\n\n   Finally, any job command can include several values that\n   are substituted before execution. Those values represent the job parameters\n   and additionally, a unique identifier for each job.\n\n   :${QMAP_LINE}: identifier of the line the job command has in the input file\n   :${QMAP_CORES}: cores for the execution\n\n\n\n\n**qmap reattach**\n   Once a ``qmap submit`` execution is closed, you can\n   reconnect from its logs directory\n\n   .. code:: bash\n\n      qmap reattach --logs <logs folder>\n\n   .. note::\n\n      If in the previous execution there were jobs that have not been submitted to the workload manager\n      ``qmap reattach`` can submit them, but the execution is halted except for the ``no-console`` interface.\n\n\n**qmap info**\n   ``qmap submit`` generates a file for each job with metadata information.\n   ``qmap info`` is designed to explore them and retrieve the\n   requested data. Information is stored in json format and\n   the user can request any fields:\n\n   .. code:: bash\n\n      qmap info --logs <logs folder> <field 1> <field 2>.<subfield 1> ...\n\n   In addition, the *--status* option can be used to filter the jobs\n   by their status (completed|failed|other|pending|running|unsubmitted|all).\n\n   If you do not pass any field, then the return value\n   is the input commands of the jobs.\n   This feature can be used to generate a new jobs file a subset of the original one.\n\n\n\n.. _jobs map file:\n\nJobs map file\n-------------\n\nThis file contains a list of the commands to be executed as well as\ncommands to be executed before and after each job (e.g. loading Easy Build modules or conda environments).\nThe format of the file is::\n\n  [pre]\n  # command to be executed before any job\n\n  [post]\n  # command to be executed after any job\n\n  [params]\n  # parameters for all the jobs\n  cores = 7\n  memory = 16G\n\n  [jobs]\n  job command\n  job command\n\n\nInstallation\n------------\n\n**QMap** depends on Python >3.5 and some external libraries.\n\nYou can install it directly from our github repository::\n\n    pip install git+https://github.com/bbglab/qmap.git\n\n\nLicense\n-------\n\n`Apache Software License 2.0 <LICENSE.txt>`_.\n\n\n.. [#precmd] Commands executed before any actual job",
        "description_content_type": "text/x-rst",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/bbglab/qmap",
        "keywords": "",
        "license": "Apache Software License 2.0",
        "maintainer": "",
        "maintainer_email": "",
        "name": "qmap",
        "package_url": "https://pypi.org/project/qmap/",
        "platform": "",
        "project_url": "https://pypi.org/project/qmap/",
        "project_urls": {
            "Homepage": "https://github.com/bbglab/qmap"
        },
        "release_url": "https://pypi.org/project/qmap/0.4/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Manage job executions in a cluster",
        "version": "0.4"
    },
    "last_serial": 5523314,
    "releases": {
        "0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "bed5504c885869985488e95b06ad2011",
                    "sha256": "74fb7ea70bed3f9580d13d467fef9bbb7a23d512818bf035aa2bf9db9e11811d"
                },
                "downloads": -1,
                "filename": "qmap-0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "bed5504c885869985488e95b06ad2011",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 42668,
                "upload_time": "2019-01-14T10:51:26",
                "url": "https://files.pythonhosted.org/packages/fa/5f/fde43ba5a0bdcb8c5b0dc39b148bd4b06d228a0291c511485f799edce115/qmap-0.1.tar.gz"
            }
        ],
        "0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c389c800c9b4ad2af11bbf7d4485a726",
                    "sha256": "678f4e95385e17dea011ce6acc22913cf84cfa61f84b6356dcd4f78c69662b55"
                },
                "downloads": -1,
                "filename": "qmap-0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "c389c800c9b4ad2af11bbf7d4485a726",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 42803,
                "upload_time": "2019-02-18T09:00:18",
                "url": "https://files.pythonhosted.org/packages/d1/3b/865c14c17dd8f67cc4359e12c111aefb91d8563f853e3ae42a4de651e46e/qmap-0.2.tar.gz"
            }
        ],
        "0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e3f2889390f20bd7c530d100788f025b",
                    "sha256": "971fb5192b6799bd8e8558d45848d128cd923a74c2c128deca47231dae4abb77"
                },
                "downloads": -1,
                "filename": "qmap-0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "e3f2889390f20bd7c530d100788f025b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 42852,
                "upload_time": "2019-04-01T14:06:18",
                "url": "https://files.pythonhosted.org/packages/aa/67/00dbdc2b922f65a2e00c4d5187c5e761fac67c8cadbaaef8f8ed47d84b22/qmap-0.3.tar.gz"
            }
        ],
        "0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2bef38dd06ea9fdd82888d07aaa98179",
                    "sha256": "082c2747006802d407ae297e6e74976762abeb0fcff25914f4b8f84a11adcda2"
                },
                "downloads": -1,
                "filename": "qmap-0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "2bef38dd06ea9fdd82888d07aaa98179",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 43194,
                "upload_time": "2019-07-12T14:03:57",
                "url": "https://files.pythonhosted.org/packages/5d/37/cc5ce66f2742041ff5f9ecced0eb00527a9539d796fc4e312e44d9dda505/qmap-0.4.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "2bef38dd06ea9fdd82888d07aaa98179",
                "sha256": "082c2747006802d407ae297e6e74976762abeb0fcff25914f4b8f84a11adcda2"
            },
            "downloads": -1,
            "filename": "qmap-0.4.tar.gz",
            "has_sig": false,
            "md5_digest": "2bef38dd06ea9fdd82888d07aaa98179",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 43194,
            "upload_time": "2019-07-12T14:03:57",
            "url": "https://files.pythonhosted.org/packages/5d/37/cc5ce66f2742041ff5f9ecced0eb00527a9539d796fc4e312e44d9dda505/qmap-0.4.tar.gz"
        }
    ]
}