{
"info": {
"author": "Stian Soiland-Reyes",
"author_email": "stain@apache.org",
"bugtrack_url": null,
"classifiers": [
"Development Status :: 2 - Pre-Alpha",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
"Topic :: Internet",
"Topic :: Software Development :: Build Tools",
"Topic :: System :: Archiving :: Packaging"
],
"description": "# CWLProv Python tool\n\nThe `cwlprov` Python tool is a command line interface to validate and inspect \n[CWLProv](https://w3id.org/cwl/prov) Research Objects that capture workflow runs,\ntypically executed in a [Common Workflow Language](https://www.commonwl.org/)\nimplementation.\n\n## Installation\n\nYou'll need [Python 3](https://www.python.org/downloads/).\n\nTo install from [pip](https://pypi.org/project/cwlprov/) try:\n\n pip3 install cwlprov\n\nIf you would rather install from the checkout of this source code:\n\n pip3 install .\n\nIf you would like to use the `cwltool rerun` feature you may also need:\n\n pip3 install cwlref-runner \n\n\n## Development\n\nTo develop cwlprov-py it is recommended to set up a new [virtualenv](https://docs.python.org/3/library/venv.html):\n\n virtualenv -p python3 venv\n\nTo activate the environment and install your development version of cwlprov:\n\n . venv3/bin/activate\n pip3 install .\n\n\n## Usage\n\nUse `cwlprov --help` to see all options. For instance `cwlprov validate` will validate the folder is valid according to CWLProv.\n\n $ cwlprov --help\n usage: cwlprov [-h] [--version] [--directory DIRECTORY] [--relative]\n [--absolute] [--output OUTPUT] [--verbose] [--quiet] [--hints]\n [--no-hints]\n {validate,info,who,prov,inputs,outputs,run,runs,rerun,derived,runtimes}\n ...\n\n cwlprov explores Research Objects containing provenance of Common Workflow\n Language executions. \n\n optional arguments:\n -h, --help show this help message and exit\n --version show program's version number and exit\n --directory DIRECTORY, -d DIRECTORY\n Path to CWLProv Research Object (default: .)\n --relative Output paths relative to current directory (default if\n -d is missing or relative)\n --absolute Output absolute paths (default if -d is absolute)\n --output OUTPUT, -o OUTPUT\n File to write output to (default: stdout)\n --verbose, -v Verbose logging (repeat for more verbose)\n --quiet, -q No logging or hints\n --hints Show hints on cwlprov usage\n --no-hints Do not show hints\n\n commands:\n {validate,info,who,prov,inputs,outputs,run,runs,rerun,derived,runtimes}\n validate validate the CWLProv Research Object\n info show research object metadata\n who show who ran the workflow\n prov export workflow execution provenance in PROV format\n inputs list workflow/step input files/values\n outputs list workflow/step output files/values\n run show workflow execution log\n runs List all workflow executions in RO\n rerun Rerun a workflow or step\n derived List what was derived from a data item, based on\n activity usage/generation\n runtimes Calculate average step execution runtimes\n\nThe [test/](test/) folder contains some examples of workflow runs for different CWLProv profiles.\n\nAll commands for `cwlprov` will attempt to detect the CWLProv research object from the current directory, alternatively take the `--directory` option to specify the root folder.\n\nThe `--quiet` option may be used in scripts for less verbose outputs. The `--verbose` option has the opposite affect to enable logging. For debug logging, use `-vv` or `--verbose --verbose`.\n\nNote that the general arguments listed above must be provided *before* the _command_, e.g. \n\n cwlprov --quiet --directory /tmp/1 validate\n\nMany of the commands accept additional arguments, which can be accessed by `cwlprov COMMAND --help`, e.g.:\n\n $ cwlprov run --help\n usage: cwlprov run [-h] [--step STEP] [--steps] [--no-steps] [--start]\n [--no-start] [--end] [--no-end] [--duration]\n [--no-duration] [--labels] [--no-labels] [--inputs]\n [--outputs]\n [id]\n\n positional arguments:\n id workflow run UUID\n\n optional arguments:\n -h, --help show this help message and exit\n --step STEP, -s STEP Show only step with given UUID\n --steps List steps of workflow\n --no-steps Do not list steps\n --start Show start timestamps (default)\n --no-start, -S Do not show start timestamps\n --end, -e Show end timestamps\n --no-end Do not show end timestamps\n --duration Show step duration (default)\n --no-duration, -D Do not show step duration\n --labels Show activity labels\n --no-labels, -L Do not show activity labels\n --inputs, -i Show inputs\n --outputs, -o Show outputs\n\n\n### Validation\n\nRunning `cwlprov` with no commands will return with status 0 if a CWLProv folder structure is detected:\n\n $ cd test/revsort-cwlprov-0.4.0\n test/revsort-cwlprov-0.4.0$ cwlprov \n Detected CWLProv Research Object: /home/stain/src/cwlprov-py/test/revsort-cwlprov-0.4.0\n\n $ cd /tmp\n /tmp$ cwlprov\n ERROR:cwlprov.tool:Could not find bagit.txt, try cwlprov -d mybag/\n\nIf a cwlprov is not detected or invalid, an error code is raised.\n\n cwlprov && echo Do cwlprov-stuff\n ERROR:cwlprov.tool:Could not find bagit.txt, try cwlprov -d mybag/\n\nCombined with the `--quiet` option `cwlprov` can be useful to find the root of a CWLProv folder:\n\n test/revsort-cwlprov-0.4.0/metadata/provenance$ cwlprov -q\n /home/stain/src/cwlprov-py/test/revsort-cwlprov-0.4.0\n\nAll commands of `cwlprov` will by default perform a _quick validation_, which conforms all files are present in the correct file size. For instance, if we remove a file:\n\n test/revsort-cwlprov-0.4.0$ rm data/32/327fc7aedf4f6b69a42a7c8b808dc5a7aff61376 \n\n test/revsort-cwlprov-0.4.0$ cwlprov \n ERROR:cwlprov.tool:BagIt validation failed for: /home/stain/src/cwlprov-py/test/revsort-cwlprov-0.4.0: Payload-Oxum validation failed. Expected 3 files and 3333 bytes but found 2 files and 2222 bytes\n\nTo perform full validation, use `cwlprov validate`:\n\n test/revsort-cwlprov-0.4.0$ cwlprov validate \n WARNING:bdbag.bdbagit:data/32/327fc7aedf4f6b69a42a7c8b808dc5a7aff61376 exists in manifest but was not found on filesystem\n ERROR:cwlprov.tool:BagIt validation failed for: /home/stain/src/cwlprov-py/test/revsort-cwlprov-0.4.0: Bag validation failed: data/32/327fc7aedf4f6b69a42a7c8b808dc5a7aff61376 exists in manifest but was not found on filesystem\n\n test/revsort-cwlprov-0.4.0$ git checkout . \n\n test/revsort-cwlprov-0.4.0$ cwlprov validate\n Valid CWLProv RO: .\n\nUnlike the quick validation, `cwlprov validate` will confirm checksums on all files, and thus detect byte-level changes. For instance, let's pretend `I` has been replaced with lower case `i` in a data file:\n\n test/revsort-cwlprov-0.4.0$ sed -i 's/I/i/g' data/32/327fc7aedf4f6b69a42a7c8b808dc5a7aff61376\n test/revsort-cwlprov-0.4.0$ cwlprov \n Detected CWLProv Research Object: /home/stain/src/cwlprov-py/test/revsort-cwlprov-0.4.0\n\n test/revsort-cwlprov-0.4.0$ cwlprov validate\n WARNING:bdbag.bdbagit:data/32/327fc7aedf4f6b69a42a7c8b808dc5a7aff61376 sha1 validation failed: expected=\"327fc7aedf4f6b69a42a7c8b808dc5a7aff61376\" found=\"60c41d3758bc8b03e78db07bc0f17d1804d2662d\"\n ERROR:cwlprov.tool:BagIt validation failed for: /home/stain/src/cwlprov-py/test/revsort-cwlprov-0.4.0: Bag validation failed: data/32/327fc7aedf4f6b69a42a7c8b808dc5a7aff61376 sha1 validation failed: expected=\"327fc7aedf4f6b69a42a7c8b808dc5a7aff61376\" found=\"60c41d3758bc8b03e78db07bc0f17d1804d2662d\"\n\n### Research Object information\n\nThe `cwlprov info` command gives high-level information about the research object and its identifiers.\n\n test/revsort-cwlprov-0.4.0$ cwlprov info\n Research Object of CWL workflow run\n Research Object ID: arcp://uuid,d47d3d43-4830-44f0-aa32-4cda74849c63/\n Profile: https://w3id.org/cwl/prov/0.4.0\n Workflow run ID: urn:uuid:d47d3d43-4830-44f0-aa32-4cda74849c63\n Packaged: 2018-08-21\n\nThe `Profile` indicates the version of the CWLProv the research object implements, \nwhich determine which features of a workflow run is represented.\n\nNote that a warning will be printed if an unknown CWLProv version is detected:\n\n $ cwlprov\n WARNING:cwlprov.tool:Unsupported CWLProv version: {'https://w3id.org/cwl/prov/0.8.0'}\n Supported profiles:\n https://w3id.org/cwl/prov/0.6.0\n https://w3id.org/cwl/prov/0.5.0\n https://w3id.org/cwl/prov/0.4.0\n https://w3id.org/cwl/prov/0.3.0\n\nThis typically means that cwlprov-py is outdated, although that is normally harmless. Try `pip install --upgrade cwlprov` \n\nThe `cwlprov who` command will try to determine the user that ran the workflow.\n\n $ cwlprov who\n Packaged By: cwltool 1.0.20180925133620 \n Executed By: Stian Soiland-Reyes \n\n_Note that for privacy concerns, CWL executors like [cwltool](https://github.com/common-workflow-language/cwltool)\nwould not log such user information unless this has been enabled with options like `--orcid` `--full-name` or `--enable-user-provenance`._\n\n### Workflow run\n\nTo list the step executions of a workflow use `cwlprov run`:\n\n test/revsort-cwlprov-0.4.0$ cwlprov run\n 2018-08-21 17:26:24.467844 Flow d47d3d43-4830-44f0-aa32-4cda74849c63 [ Run of workflow/packed.cwl#main \n 2018-08-21 17:26:24.530884 Step 6f501717-0c97-492e-b18a-10bc096f1797 Run of workflow/packed.cwl#main/rev (0:00:01.122498)\n 2018-08-21 17:26:25.656084 Step e7c8b2c0-dee6-4c61-b674-f0807cb47344 Run of workflow/packed.cwl#main/sorted (0:00:01.087999)\n 2018-08-21 17:26:26.752493 Flow d47d3d43-4830-44f0-aa32-4cda74849c63 ] Run of workflow/packed.cwl#main (0:00:02.284649)\n Legend:\n [ Workflow start\n ] Workflow end\n\nThe listing can be customized, see `cwlprov run --help` for details. For example:\n\n test/revsort-cwlprov-0.4.0$ cwlprov --no-hints run --no-labels --start --end --no-duration \n 2018-08-21 17:26:24.467844 Flow d47d3d43-4830-44f0-aa32-4cda74849c63 [\n 2018-08-21 17:26:24.530884 2018-08-21 17:26:25.653382 Step 6f501717-0c97-492e-b18a-10bc096f1797 \n 2018-08-21 17:26:25.656084 2018-08-21 17:26:26.744083 Step e7c8b2c0-dee6-4c61-b674-f0807cb47344 \n 2018-08-21 17:26:26.752493 Flow d47d3d43-4830-44f0-aa32-4cda74849c63 ]\n\n### Nested workflows\n\nNested workflows, steps that themselves are workflows, are indicated in `cwlprov run` with a `*`:\n\n (venv3) stain@biggie:~/src/cwlprov-py/test/nested-cwlprov-0.3.0$ cwlprov run\n 2018-08-08 22:44:06.573330 Flow 39408a40-c1c8-4852-9747-87249425be1e [ Run of workflow/packed.cwl#main \n 2018-08-08 22:44:06.691722 Step 4f082fb6-3e4d-4a21-82e3-c685ce3deb58 Run of workflow/packed.cwl#main/create-tar (0:00:00.010133)\n 2018-08-08 22:44:06.702976 Step 0cceeaf6-4109-4f08-940b-f06ac959944a * Run of workflow/packed.cwl#main/compile (unknown duration)\n 2018-08-08 22:44:12.680097 Flow 39408a40-c1c8-4852-9747-87249425be1e ] Run of workflow/packed.cwl#main (0:00:06.106767)\n Legend:\n [ Workflow start\n * Nested provenance, use UUID to explore: cwlprov run 0cceeaf6-4109-4f08-940b-f06ac959944a\n ] Workflow end\n\n (venv3) stain@biggie:~/src/cwlprov-py/test/nested-cwlprov-0.3.0$ cwlprov run 0cceeaf6-4109-4f08-940b-f06ac959944a\n 2018-08-08 22:44:06.607210 Flow 0cceeaf6-4109-4f08-940b-f06ac959944a [ Run of workflow/packed.cwl#main \n 2018-08-08 22:44:06.707070 Step 83752ab4-8227-4d4a-8baa-78376df34aed Run of workflow/packed.cwl#main/untar (0:00:00.008149)\n 2018-08-08 22:44:06.718554 Step f56d8478-a190-4251-84d9-7f69fe0f6f8b Run of workflow/packed.cwl#main/argument (0:00:00.532052)\n 2018-08-08 22:44:07.251588 Flow 0cceeaf6-4109-4f08-940b-f06ac959944a ] Run of workflow/packed.cwl#main (0:00:00.644378)\n Legend:\n [ Workflow start\n ] Workflow end\n\n_Note that there is a bug in CWLProv 0.3.0 logging shown above; steps of nested workflows are misleadingly labeled under `#main`_\n\nYou can list all workflow runs (including nested workflow runs) with `cwlprov runs`:\n\n test/nested-cwlprov-0.3.0$ cwlprov runs\n 39408a40-c1c8-4852-9747-87249425be1e * Run of workflow/packed.cwl#main\n 0cceeaf6-4109-4f08-940b-f06ac959944a Run of workflow/packed.cwl#main\n Legend:\n * master workflow\n\nTo explore the nested workflow run with other commands you may have to provide the run UUID with `--run` argument, e.g. \n\n test/nested-cwlprov-0.3.0$ cwlprov outputs --format=files --run 0cceeaf6-4109-4f08-940b-f06ac959944a 83752ab4-8227-4d4a-8baa-78376df34aed\n Output example_out:\n data/93/93035905e94e150874f5a881d39f3c5c6378dd38\n\n\n\n## License\n\nThis repository is distributed under [Apache License, version 2.0](https://www.apache.org/licenses/LICENSE-2.0) \n\nSee the file [LICENSE.txt](LICENSE.txt) for details, and [NOTICE](NOTICE) for required notices.\n\nSPDX-License-Identifier: Apache-2.0\n\n## Contributing\n\ncwlprov.py is maintained at https://github.com/common-workflow-language/cwlprov-py/ by the [Common Workflow Language](https://www.commonwl.org/) project.\n\nFeel free to raise an\n[issue](https://github.com/common-workflow-language/cwlprov-py/issues) or a\n[pull request](https://github.com/common-workflow-language/cwlprov-py/pulls) to\ncontribute to CWLProv. Contributions are assumed to be covered by \n[section 5 of the Apache License](https://www.apache.org/licenses/LICENSE-2.0#contributions).\n\nFor an informal CWLProv discussion with other developers, join the (relatively\nquiet) Gitter room\n[common-workflow-language/cwlprov](https://gitter.im/common-workflow-language/cwlprov),\nor the (more busy)\n[common-workflow-language/common-workflow-language](https://gitter.im/common-workflow-language/common-workflow-language).\n\n\n### Code of Conduct\n\nThe CWL Project is dedicated to providing a harassment-free experience for\neveryone, regardless of gender, gender identity and expression, sexual\norientation, disability, physical appearance, body size, age, race, or\nreligion. We do not tolerate harassment of participants in any form. This code\nof conduct applies to all CWL Project spaces, including the Google Group, the\nGitter chat room, the Google Hangouts chats, both online and off. Anyone who\nviolates this code of conduct may be sanctioned or expelled from these spaces\nat the discretion of the leadership team.\n\nFor more details, see our \n[Code of Conduct](https://github.com/common-workflow-language/common-workflow-language/blob/master/CODE_OF_CONDUCT.md).\n\n\n\n",
"description_content_type": "text/markdown",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/common-workflow-language/cwlprov-py",
"keywords": "cwl prov cwlprov provenance",
"license": "Apache License, Version 2.0",
"maintainer": "",
"maintainer_email": "",
"name": "cwlprov",
"package_url": "https://pypi.org/project/cwlprov/",
"platform": "",
"project_url": "https://pypi.org/project/cwlprov/",
"project_urls": {
"Homepage": "https://github.com/common-workflow-language/cwlprov-py"
},
"release_url": "https://pypi.org/project/cwlprov/0.1.1/",
"requires_dist": [
"prov (>=1.5.1)",
"bdbag (>=1.4.1)",
"arcp (>=0.2.0)",
"rdflib-jsonld (>=0.4.0)",
"rdflib (>=4.2.2)"
],
"requires_python": ">=3.6, <4",
"summary": "cwlprov API for Python",
"version": "0.1.1"
},
"last_serial": 4414625,
"releases": {
"0.1.1": [
{
"comment_text": "",
"digests": {
"md5": "695db4b0a9ae89f67f4c0f4364429f87",
"sha256": "c0bca8a038f130dc67f02d4a1c08757a3d7dd33b2219599e71267211a4e484b1"
},
"downloads": -1,
"filename": "cwlprov-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "695db4b0a9ae89f67f4c0f4364429f87",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6, <4",
"size": 31869,
"upload_time": "2018-10-25T10:31:24",
"url": "https://files.pythonhosted.org/packages/a0/e7/3b0bc23a4a657859479ed62df07a6f7df4316ab7ccd0a2bd0c2458b0071e/cwlprov-0.1.1-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "0c1e5ca1d6c744452df722139e567cde",
"sha256": "aa12a6fd99875fd1adc40a12fe3e75812318bb2edb9e087b228d73920cfbe7ab"
},
"downloads": -1,
"filename": "cwlprov-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "0c1e5ca1d6c744452df722139e567cde",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6, <4",
"size": 25206,
"upload_time": "2018-10-25T10:31:26",
"url": "https://files.pythonhosted.org/packages/4f/fb/c927d55dc3b8f14d9a54af9f8b0e190f871a0916d5ffb52a91f1eaa48036/cwlprov-0.1.1.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "695db4b0a9ae89f67f4c0f4364429f87",
"sha256": "c0bca8a038f130dc67f02d4a1c08757a3d7dd33b2219599e71267211a4e484b1"
},
"downloads": -1,
"filename": "cwlprov-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "695db4b0a9ae89f67f4c0f4364429f87",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6, <4",
"size": 31869,
"upload_time": "2018-10-25T10:31:24",
"url": "https://files.pythonhosted.org/packages/a0/e7/3b0bc23a4a657859479ed62df07a6f7df4316ab7ccd0a2bd0c2458b0071e/cwlprov-0.1.1-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "0c1e5ca1d6c744452df722139e567cde",
"sha256": "aa12a6fd99875fd1adc40a12fe3e75812318bb2edb9e087b228d73920cfbe7ab"
},
"downloads": -1,
"filename": "cwlprov-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "0c1e5ca1d6c744452df722139e567cde",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6, <4",
"size": 25206,
"upload_time": "2018-10-25T10:31:26",
"url": "https://files.pythonhosted.org/packages/4f/fb/c927d55dc3b8f14d9a54af9f8b0e190f871a0916d5ffb52a91f1eaa48036/cwlprov-0.1.1.tar.gz"
}
]
}