{ "info": { "author": "Jonathan R. Madsen", "author_email": "jonrobm.programming@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: Implementation :: CPython", "Topic :: Utilities" ], "description": "# PyCTest\n\n| Travis | AppVeyor |\n| ------ | -------- |\n| [![Build Status](https://travis-ci.org/jrmadsen/pyctest.svg?branch=master)](https://travis-ci.org/jrmadsen/pyctest) | [![Build status](https://ci.appveyor.com/api/projects/status/p7m76ovx7sg781pf/branch/master?svg=true)](https://ci.appveyor.com/project/jrmadsen/pyctest/branch/master) |\n\n[PyCTest Documentation (readthedocs)](https://pyctest.readthedocs.io/en/latest/)\n================================================================================\n\n## GitHub\n\n- `git clone https://github.com/jrmadsen/pyctest.git`\n\n## PyPi\n\n- `pip install pyctest`\n\n## Anaconda Cloud\n\n- `conda install -c jrmadsen pyctest`\n\n| Name | Version | Platforms | Downloads |\n| --- | --- | --- | --- |\n| [![Conda Recipe](https://img.shields.io/badge/recipe-pyctest-green.svg)](https://anaconda.org/jrmadsen/pyctest) | [![Anaconda-Server Badge](https://anaconda.org/jrmadsen/pyctest/badges/version.svg)](https://anaconda.org/jrmadsen/pyctest) | [![Anaconda-Server Badge](https://anaconda.org/jrmadsen/pyctest/badges/platforms.svg)](https://anaconda.org/jrmadsen/pyctest) | [![Anaconda-Server Badge](https://anaconda.org/jrmadsen/pyctest/badges/downloads.svg)](https://anaconda.org/jrmadsen/pyctest) |\n\n## conda-forge\n\n- `conda install -c conda-forge pyctest`\n\n| Name | Version | Platforms | Downloads |\n| --- | --- | --- | --- |\n| [![Conda Recipe](https://img.shields.io/badge/recipe-pyctest-green.svg)](https://anaconda.org/conda-forge/pyctest) | [![Conda Version](https://img.shields.io/conda/vn/conda-forge/pyctest.svg)](https://anaconda.org/conda-forge/pyctest) | [![Conda Platforms](https://img.shields.io/conda/pn/conda-forge/pyctest.svg)](https://anaconda.org/conda-forge/pyctest) | [![Conda Downloads](https://img.shields.io/conda/dn/conda-forge/pyctest.svg)](https://anaconda.org/conda-forge/pyctest) |\n\nBenefits\n========\n\n- Integration into continuous integration systems (e.g. Travis, AppVeyor, CircleCI, etc.) and pushing to CDash dashboard will combine all the results in one place\n - The warnings and errors are enumerated in CDash (no more parsing stdout logs for errors)\n- Easily create platform-independent testing\n- No need to migrate build system to CMake -- just specify `pyctest.BUILD_COMMAND`\n\nGeneral Setup\n=============\n\n- Add PyCTest to conda environment: `conda install pyctest`\n- Write a driver Python script, e.g. `pyctest-runner.py`, in top of the source code tree\n\n## Example for Python project\n\nThe following is an example for a Python code with a compiled C extension that uses `nosetests` for unit-testing:\n\n```python\n#!/usr/bin/env python\n\nimport os, sys, platform\nimport pyctest.pyctest as pyctest\nimport pyctest.helpers as helpers\n\nparser = helpers.ArgumentParser(\"ProjectName\", source_dir=os.getcwd(),binary_dir=os.getcwd(), vcs_type=\"git\")\nargs = parser.parse_args()\n\npyctest.BUILD_NAME = \"{}\".format(args.build)\npyctest.BUILD_COMMAND = \"python setup.py build_ext --inplace\"\n\ntest = pyctest.test()\ntest.SetName(\"unittest\")\n# insert the command to run the tests for project\ntest.SetCommand([\"nosetests\"])\n\npyctest.run()\n```\n\n## Example for autotools project\n\n```python\n#!/usr/bin/env python\n\nimport os, sys, platform\nimport multiprocessing as mp\nimport pyctest.pyctest as pyctest\nimport pyctest.helpers as helpers\n\nparser = helpers.ArgumentParser(\"ProjectName\", source_dir=os.getcwd(), binary_dir=os.getcwd(),\n vcs_type=\"git\")\nparser.add_argument(\"-n\", \"--build\", type=str, required=True, help=\"Build name for identification\")\nargs = parser.parse_args()\n\n# CONFIGURE_COMMAND can only run one command so if autogen is required, just execute it here\ncmd = pyctest.command([\"./autogen.sh\"])\ncmd.SetWorkingDirectory(pyctest.SOURCE_DIRECTORY)\ncmd.SetErrorQuiet(False)\ncmd.Execute()\n\npyctest.BUILD_NAME = \"{}\".format(args.build)\npyctest.CONFIGURE_COMMAND = \"./configure\"\npyctest.BUILD_COMMAND = \"make -j{}\".format(mp.cpu_count())\n\ntest = pyctest.test()\ntest.SetName(\"unittest\")\n# insert the command to run the tests for project\ntest.SetCommand([\"./run-testing.sh\"])\n\npyctest.run()\n```\n\n## Example for CMake project\n\n```python\n#!/usr/bin/env python\n\nimport os\nimport sys\nimport platform\nimport multiprocessing as mp\nimport pyctest.pyctest as pyctest\nimport pyctest.helpers as helpers\n\nbinary_dir = os.path.join(os.getcwd(), \"build-ProjectName\")\nparser = helpers.ArgumentParser(\"ProjectName\", os.getcwd(), binary_dir)\nparser.add_argument(\"-n\", \"--build\", type=str, required=True, help=\"Build name for identification\")\nargs = parser.parse_args()\n\npyctest.BUILD_NAME = \"{}\".format(args.build)\npyctest.UPDATE_COMMAND = \"git\"\npyctest.CONFIGURE_COMMAND = \"cmake {}\".format(pyctest.SOURCE_DIRECTORY)\npyctest.BUILD_COMMAND = \"cmake --build {} --target all -- -j{}\".format(pyctest.BINARY_DIRECTORY, mp.cpu_count())\n\ntest = pyctest.test()\ntest.SetName(\"unittest\")\n# insert the command to run the tests for project\ntest.SetCommand([\"./run-testing.sh\"])\n\npyctest.run()\n```\n\n### Python Modules\n\n- `import pyctest` -- global package\n- `import pyctest.pyctest` -- CTest module\n- `import pyctest.pycmake` -- CMake module\n- `import pyctest.helpers` -- Helpers module\n - includes command line arguments (`argparse`) for PyCTest\n\n### Direct Access to CMake/CTest/CPack Executables\n\n- `python -m pyctest.cmake ` == `cmake `\n- `python -m pyctest.ctest ` == `ctest `\n- `python -m pyctest.cpack ` == `cpack `\n\nFollowing Python code:\n```python\nfrom pyctest.ctest import CTest\nfrom pyctest.cmake import CMake\nfrom pyctest.cpack import CPack\n\nCMake({\"CMAKE_BUILD_TYPE\":\"Release\"}, os.getcwd(), \"-G\", \"Ninja\")\nCTest(\"--build-and-test\", os.getcwd(), \"-VV\")\nCPack(\"-G\", \"TGZ\")\n```\n\nis equivalent to the following shell commands:\n\n```shell\n$ cmake -DCMAKE_BUILD_TYPE=Release ${PWD} -G Ninja\n$ ctest --build-and-test ${PWD} -VV\n$ cpack -G TGZ\n```\n\n## Standard Configuration Variables\n\n- `pyctest.PROJECT_NAME`\n- `pyctest.SOURCE_DIRECTORY`\n- `pyctest.BINARY_DIRECTORY`\n- `pyctest.SITE`\n- `pyctest.BUILD_NAME`\n- `pyctest.TRIGGER`\n- `pyctest.CHECKOUT_COMMAND`\n- `pyctest.BUILD_COMMAND`\n- `pyctest.MODEL`\n- `pyctest.CUSTOM_COVERAGE_EXCLUDE`\n- `pyctest.CUSTOM_MAXIMUM_NUMBER_OF_ERRORS`\n- `pyctest.CUSTOM_MAXIMUM_NUMBER_OF_WARNINGS`\n- `pyctest.CUSTOM_MAXIMUM_PASSED_TEST_OUTPUT_SIZE`\n\n### Setting Arbitrary Variables\n\n```python\n pyctest.set(\"CTEST_TOKEN_FILE\", \"${CMAKE_CURRENT_LIST_DIR}/.ctest-token\")\n```\n\n### Generating a Test\n\n```python\n test = pyctest.test()\n test.SetName(\"nosetests\")\n test.SetCommand([\"nosetests\", \"test\", \"--cover-xml\", \"--cover-xml-file=coverage.xml\"])\n # set directory to run test\n test.SetProperty(\"WORKING_DIRECTORY\", pyctest.BINARY_DIRECTORY)\n test.SetProperty(\"RUN_SERIAL\", \"ON\")\n test.SetProperty(\"ENVIRONMENT\", \"OMP_NUM_THREADS=1\")\n```\n\n### Examples\n\n- [Basic example](https://github.com/jrmadsen/pyctest/blob/master/examples/Basic/README.md)\n- [Advanced example](https://github.com/jrmadsen/pyctest/blob/master/examples/TomoPy/README.md)\n - includes submission to CDash dashboard\n\n### CDash Integration Example\n\nResults from running the TomoPy example can be found at the [TomoPy CDash Testing Dashboard @ NERSC](https://cdash.nersc.gov/index.php?project=TomoPy)\n\n- Python code with C extensions without CMake build system\n- The build logs from \"python setup.py install\" are registered in the \"Build\" section\n- The `nosetests test` command + other are wrapped into CTests\n\n## Testing Example\n\nPyCTest can be used to simple execute tests and submit to a dashboard without any configuration, build, etc. steps\n\n```python\n#!/usr/bin/env python\n\nimport os\nimport sys\nimport shutil\nimport argparse\nimport platform\nimport traceback\n\nimport pyctest.pyctest as pyctest\nimport pyctest.pycmake as pycmake\nimport pyctest.helpers as helpers\n\nif __name__ == \"__main__\":\n\n directory = os.path.join(os.getcwd(), \"pycm-test\")\n\n # these are required\n pyctest.PROJECT_NAME = \"PyCTest\"\n pyctest.SOURCE_DIRECTORY = directory\n pyctest.BINARY_DIRECTORY = directory\n\n args = helpers.ArgumentParser(pyctest.PROJECT_NAME,\n pyctest.SOURCE_DIRECTORY,\n pyctest.BINARY_DIRECTORY).parse_args()\n\n # set explicitly\n pyctest.MODEL = \"Continuous\"\n pyctest.SITE = platform.node()\n\n # create a Test object\n test = pyctest.test()\n test.SetName(\"list_directory\")\n test.SetCommand([\"ls\", directory])\n test.SetProperty(\"WORKING_DIRECTORY\", os.getcwd())\n\n # create a second test\n # previous test is already stored by PyCTest\n test = pyctest.test()\n test.SetName(\"hostname\")\n test.SetCommand([\"hostname\"])\n test.SetProperty(\"TIMEOUT\", \"10\")\n\n # generate the CTestConfig.cmake and CTestCustom.cmake\n pyctest.generate_config(directory)\n\n # generate the CTestTestfile.cmake file\n pyctest.generate_test_file(directory)\n\n # run CTest -- e.g. ctest -VV ${PWD}/pycm-test\n pyctest.run(pyctest.ARGUMENTS, directory)\n```\n\n```bash\nCTest arguments (default): '-V -DSTAGES=Start;Update;Configure;Build;Test;Coverage;MemCheck -S Stages.cmake -j1'\nWriting CTest test file: \"/Users/jrmadsen/devel/c++/pyctest-master/pycm-test/CTestTestfile.cmake\"...\nGenerating test \"list_directory\"...\nGenerating test \"hostname\"...\n-- STAGES = Start;Update;Configure;Build;Test;Coverage;MemCheck\n-- [[Darwin macOS 10.13.6 x86_64] [Python 3.6.7]] Running CTEST_START stage...\nRun dashboard with model Continuous\n Source directory: /Users/jrmadsen/devel/c++/pyctest-master/pycm-test\n Build directory: /Users/jrmadsen/devel/c++/pyctest-master/pycm-test\n Track: Continuous\n Reading ctest configuration file: /Users/jrmadsen/devel/c++/pyctest-master/pycm-test/CTestConfig.cmake\n Site: JRM-macOS-DOE.local\n Build name: [Darwin macOS 10.13.6 x86_64] [Python 3.6.7]\n Use Continuous tag: 20181129-2118\n-- [[Darwin macOS 10.13.6 x86_64] [Python 3.6.7]] Skipping CTEST_UPDATE stage...\n-- [[Darwin macOS 10.13.6 x86_64] [Python 3.6.7]] Skipping CTEST_CONFIGURE stage...\n-- [[Darwin macOS 10.13.6 x86_64] [Python 3.6.7]] Skipping CTEST_BUILD stage...\n-- [[Darwin macOS 10.13.6 x86_64] [Python 3.6.7]] Running CTEST_TEST stage...\nTest project /Users/jrmadsen/devel/c++/pyctest-master/pycm-test\n Start 1: list_directory\n1/2 Test #1: list_directory ................... Passed 0.00 sec\n Start 2: hostname\n2/2 Test #2: hostname ......................... Passed 0.00 sec\n\n100% tests passed, 0 tests failed out of 2\n\nTotal Test time (real) = 0.01 sec\n-- [[Darwin macOS 10.13.6 x86_64] [Python 3.6.7]] Skipping CTEST_COVERAGE stage...\n-- [[Darwin macOS 10.13.6 x86_64] [Python 3.6.7]] Skipping CTEST_MEMCHECK stage...\n-- [[Darwin macOS 10.13.6 x86_64] [Python 3.6.7]] Skipping CTEST_SUBMIT stage...\n-- [[Darwin macOS 10.13.6 x86_64] [Python 3.6.7]] Finished Continuous Stages (Start;Update;Configure;Build;Test;Coverage;MemCheck)\n```\n\n## Command Line Interface\n\n```shell\nPyCTest args: ['--help']\n CTest args: []\n CMake args: []\nusage: pyctest-runner.py [-h] [-F] [-S] [-A] [-j ] [-m ] [-b ] [-i ] [-o ] [-M ] [-H ] [-P ] [-N ]\n [-C [ ...]] [-L