PySys 0.9.0 Release Notes
=========================

PySys has been written and tested against Python 2.7 and higher.


What's new in this release
--------------------------
- The PySys framework has been updated to be compliant with conversion to 
  Python 3.x with the 2to3.py conversion script. Installation on Python 3.x 
  is now supported via the source distribution bundle, where the 2to3.py 
  script is run automatically at install time. See details below for 
  installing the source distribution. A binary distribution installer for 
  windows will be included in a later release. 
- There are now separate 32 and 64 bit binary distribution installers for 
  windows. 
- On failure of the assertLineCount method, the log output now contains the 
  returned number and requested condition (tracker #3045931)  
- Each assert method now takes an "assertMessage" parameter to be written to 
  the log output on execution (tracker #3045924). See test PySys_internal_053 
  in the example testcases for example usage.
- Added the JUnitXMLResultsWriter to log test results in Apache Ant JUnit XML 
  format (one output file per test per cycle). This is useful for integration 
  into Continuous Integration build systems, e.g. TeamCity. The 
  TextResultsWriter and XMLResultsWriter now support the outputDir property 
  to specify the location to write the output files. See the pysys-examples 
  pysysproject.xml file for more details.
- Added the ability to run suites of pyunit tests wrapped up as a single 
  PySys test. This capability is exposed through the PyUnitTest class 
  contained in the pysys.unit.pyunit module. See the pysys-examples pyunit
  tests for example usage.
- Fix to the unix process helper to correctly set the working directory of 
  child processes in the fork and exec. 
- When running tests in parallel, a value of zero given for the -n|--threads 
  option to the run task of the pysys.py launcher, will set the number of 
  threads to the number of available CPUs.
  
 
Dependencies
------------
Running on windows requires installation of the pywin32 extensions written by 
Mark Hammond (http://sourceforge.net/projects/pywin32). Running the manual 
tester on unix systems requires the tcl/tk libraries to be installed on the 
host machine and the Python version to be compiled with tcl/tk support.


Windows Binary Installation
---------------------------
Installation on windows via the binary distribution bundle is performed by 
downloading the PySys-X.Y.Z.win32.exe installer executable and running. Note 
that a common error on windows is in the execution of the post-install 
script, where an error of the form below is reported;

  close failed in file object destructor:
  sys.excepthook is missing
  lost sys.stderr

If obtained, right click the installer executable in an explorer window, and 
select "Troubleshoot Compatability". Select "Try recommended settings", and 
then "Start the program ...".  


Windows/Unix Source Installation
--------------------------------
To install from source on unix or windows systems you should download the 
source archive and perform the following (use winrar or winzip to unpack 
on windows);

 $ tar zxvpf PySys-X.Y.Z.tar.gz
 $ cd PySys-X.Y.Z
 $ python setup.py build
 $ python setup.py install
 
To install on both windows and unix systems you may need to have root 
privileges on the machine. 


The 'pysys.py' launcher 
-----------------------
PySys installs a launcher script 'pysys.py' as part of the installation 
process to facilitate the management and execution of testcases. On unix 
systems the script is installed into the Python binary directory, e.g. 
/usr/local/bin, and is hence on the default user's path. On windows systems 
the script is installed into the Scripts directory of the Python 
installation, e.g. c:\Python24\Scripts\pysys.py, which is not by default on 
the user's path. To run on windows systems the Scripts directory of the 
Python installation should be added to the user's path to allow direct 
execution of the script. 

After installation, to see the available options to the pysys.py script use 

  $ pysys.py --help
  
The script takes four main top level command line options to it, namely 
'run', 'print', 'make' and 'clean', which are used to run a set of testcases, 
print the meta data for a set of testcases, make a new testcase directory 
structure, or clean all testcase output. For more information on the further 
options available to each add --help after the top level option, e.g. 

  $ pysys.py run --help


Running the samples
-------------------
PySys has a set of basic examples to demonstrate it's use for running 
automated and manual testcases. The samples are distributed in a unix line 
ending friendly tar.gz archive, and a windows line ending friendly zip file. 
To unpack the tests on unix systems use

 $ tar zxvpf PySys-examples.X.Y.Z.tar.gz
 $ cd pysys-examples

To run the testcases, after changing directory to the testcases location, 
perform 
 
 $ pysys.py run  


Release History
---------------
0.7.6 to 0.8.1
--------------
- Updated the pysys.process.plat-win32.helper.ProcessWrapper module to eliminate 
  the use of threads to collect the stdout and stderr from the process via pipes. 
  The module now directly uses win32file.CreateFile to create file objects to 
  pass to the call to win32process.CreateProcess. Many thanks to Mark Horsburgh
  for this recommended change. 
- Added the <formatters/> element to the pysysproject file. This allows setting 
  the format of the test output to stdout and the runlog in accordance to the
  format specifiers in the python logging and time modules. For examples of the 
  use of this element, see the pysysproject file included in the PySys examples. 
- Logging of exceptions and failed asserts has been changed from info to warn 
  level (tracker #2784251).
- Added extra debug logging in pysys.utils.filegrep, and pysys.basetest for when 
  performing asserts against a line count in an input file (tracker #2824758).
- The testcase output summary is now printed on termination of the test run via
  a keyboard interrupt (tracker #2816212).
- The PySys project file now allows assignment of the project root location to 
  a variable which can then be used for later expansion within the file. This 
  allows the definition of project variables to include the full path where this 
  is required, e.g. XSL stylesheets which must use the full path to the file 
  rather than a relative path etc. Note that modules within PySys can reference 
  the project root location directly using PROJECT.root (tracker #2795316). 
- The pysys.baserunner class now passes the -X arguments into the test summary 
  writer setup action to allow logging of the user supplied extra arguments
  (tracker #2814499). The pysys-log.xsl stylesheet used by the XMLResultsWriter 
  has been updated to display this information in the test summary display. 
- Fixed an issue where when the pysysproject file was missing, defaults for the 
  runner module and the test output summary writer were not being set.

0.7.5 to 0.7.6
--------------
- Fixed a defect in the unix process helper module to correct a file handle leak in 
  the write end of the stdin pipe.

0.6.1 to 0.7.5
--------------
- Added the ability to run tests concurrently through the -n | --threads option 
  to the pysys launcher run target. Tests to be run are placed on a request queue
  and processed by the designated number of worker threads. The results of each test
  are then placed on a result queue, collated and displayed in the order in which they 
  would run serially. Depending on the nature of the application under test, the 
  recommended number of threads to designate when using this option is no more than
  two times the number of CPUs. Note also that care needs to be made when running 
  tests in parallel, so as to ensure no shared resources are accessed in a non-atomic 
  way, e.g using direct references to os.environ() in one test when another test 
  modifies the environment directly etc.
- The constructor to the pysys.baserunner.BaseRunner class was changed to include 
  the threads parameter, i.e.
  
  	def __init__(self, record, purge, cycle, mode, threads, outsubdir, descriptors, xargs)
  	
  This parameter is required for the runner to create the required threadpool before 
  running a set of tests in parallel. Any custom runner classes extending the base runner 
  will need to be updated to incorporate this change. 
- Removed module specific loggers from pysys in order to support running tests in 
  parallel. There is now a single logger used within the framework, and which can 
  be referenced directly from the pysys.constants module. Attached to this logger are
  two handler types; one for logging to stdout, and one for logging to the run log 
  file saved in the output subdirectory of each test. The sdtout handler is set to only 
  log to stdout from the main thread, whilst the run log file handlers are set to log to 
  the output subdirectory of a test only on the creating thread.
- Added exception handling to the pysys.process.user module when trying to stop all 
  processes on destruction. When a background process takes longer to stop than the 
  default timeout period, the thrown pysys.exceptions.ProcessTimeout exception was 
  uncaught causing abnormal exit from the test run.  

0.6.0 to 0.6.1
--------------
- The clean target has been updated to accept the -a | --all command line option to 
  allow deleting all derived files produced when running a set of testcases, i.e. 
  both the testcase output subdirectory and any compiled testclass modules.
- The waitForSignal method of the ProcessUser class, subclassed by both the BaseTest
  and BaseRunner classes, has been updated to return a list of match objects on 
  invocation. By using tagged regular expressions in the expr parameter of the 
  method call, this allows retrieval of portions of the matched data e.g. to extract
  expressions in the file to use later in the validation routines. 
- All references to pysys.constants.TRUE and pysys.constants.FALSE have been replaced 
  by the native Python True and False literals. The values of the constants have been 
  set to True and False respectively so as to maintain backwards compatibility.

0.5.2 to 0.6.0
--------------
- The PySys test and PySys project files have been renamed by default from  
  .pysystest to pysystest.xm, and .pysysproject to pysysproject.xml respectively. 
  Backwards compatability is maintained for the previous file naming convention, 
  though this will be deprecated in a later release; it is strongly advised that 
  the new naming convention is adopted. New tests made using the PySys launcher 
  will by default use the new naming convention. This change was made due to issues 
  on Windows systems displaying and recognising hidden files, and files without 
  specified extensions e.g. within the Eclipse framework, for display in internet 
  browsers etc.
- The clean mode of operation has been added to the pysys.py launcher. This 
  allows removal of testcase output subdirectories, e.g. before importing into 
  a source code control system. The -o option allows specifying the output 
  subdirectory name to be deleted, which defaults to the platform identifier 
  if not specified. 
- The test output summary writer interface has been changed so that the test output 
  is written and updated during the test execution; previously a call to the writer 
  was only made on completion of the test run. This allows monitoring the test output 
  summary during the test execution to monitor the run time status of the tests. 
- Added the XMLFileResultsWriter class to the pysys.writer module. This 
  performs logging of the test output summary in an XML format, suitable for 
  display via XLST in a web browser. A simple XSL stylesheet is included with 
  the PySys distribution to provide better display in internet browsers. 
- Added the ability to specifiy custom test output summary writers in the 
  PySys project file via the <writer> tag. For an example see the .pysysproject
  file in the pysys-examples distribution. Should no <writer> be specified in the 
  project file, the default XMLFileResultsWriter will be used. Multiple writers 
  may be specified in the PySys project file.  
- Added exception logging on parsing errors in the PySys project file, e.g. 
  when the file in badly formed due to invalid XML tokens etc.
- Added variable argument passing to the process.monitor.ProcessMonitor class 
  constructor so that operating specific arguments can be passed into the class 
  on instantiation. The wrapper method pysys.basetest.BaseTest.startProcessMonitor
  has also been updated to allow pass through of the variable arguments. 
- The win32 process.monitor module has been changed so that on windows systems the 
  percentage CPU usage is not normalised by default by the number of available 
  processors, e.g. on a 4 core processor if 2 cores were fully utilized the 
  CPU usage was previously output as 50% - the change means that the reported usage 
  will now be 200% (a value of 100% indicates that one core is fully utilitised). 
  This makes the output consistent with that reported on unix systems. Should the 
  previous behavior be required the numProcessors argument can be passed to the 
  pysys.basetest.BaseTest.startProcessMonitor method in order to normalise the CPU usage 
  statistics by the number of processors. On windows systems the number of processors 
  can be obtained from the NUM_PROCESSORS environment variable.
- Added comments to the PySys Project file distributed with the example testcases, 
  to detail the possible configuration options.

0.5.1 to 0.5.2
--------------
- The lastgrep method has been added to pysys.utils.filegrep, and the assertLastGrep 
  method has been added to the BaseTest class. This allows test validation to be 
  performed based on regular expression matching on the last line of an input file 
  to the assertLastGrep method.
- The win32 process monitor has been modified to calculate the percentage CPU usage 
  statistics as a sum over all available processors. A CPU usage of 100% represents
  the process fully utilising all available processors. 
- The win32 process monitor now also logs the handle count of a process.

0.5.0 to 0.5.1
--------------
- Fixed a bug in pysys.process.user.ProcessUser destructor to explicitly set the 
  process list to null to allow process handles to be cleaned up on destruction. 
  This bug only seemed to be exhibited when the process handle of a process 
  returned in the startProcess() method was set as a data attribute to an instance
  of the class. This handle was then both a data attribute of the class, and was 
  contained in a list data attribute of the class. Under these conditions the handles 
  were not being released correctly.
- The print mode of the pysys.py launcher now supports printing out the test user 
  defined modes, and the printing out of tests that can be run in a given mode.
  
0.4.0 to 0.5.0
--------------
- The OSFAMILY constant has been added to pysys.constants, and takes the value 'windows' 
  on all win32 operating systems, and 'unix' on sunos and linux operating systems. The 
  value of the OSFAMILY can be used within the .pysysproject file using the osfamily 
  attribute to the <property> element. This allows capturing the value to be used in 
  expansion of other properties defined within the project file; see below for an example 
  usage. Should no value be set in a properties file, a default value of "osfamily" is 
  assumed. 
  
- The .pysysproject file now allows explicitly setting the environment value to be used
  in expansions via the environment attribute to the <property> element; see below for an 
  example usage. Should no value be set in a properties file, a default value of "env" is 
  assumed (this allows for backwards compatibility).
   
- The .pysysproject file now takes the file attribute to the <property> element. This
  allows properties to be read from file, where the properties are specified in the 
  name=value syntax, e.g.
  
    <pysysproject>
      <property environment="env"/>
      <property osfamily="osfamily"/>
      <property file="${osfamily}.properties" />
      <property name="lib" value="${library}_${osfamily}_${version}_${env.USER}.so"/>
    </pysysproject>
  
  where the property file contains the following;
  
     version=1.0
     library=jstore${version}.jar
  
  For more details, see testcase PySys_internal_002 in the pysys-examples/internal 
  area which demonstrates this. 
  
- Fixed the issue of removing zero size files from the output subdirectory on win32
  platforms; was due to the stderr and stdout file handles not being properly closed 
  down. Updated the BaseRunner to attempt to remove the zero sized files 3 times to 
  try to avoid race conditions of stopped background processes holding on to the file 
  handles too long before dying.

- The win32 process helper now ensures the environment in which the process runs is 
  converted to unicode to avoid issues encountered with running under certain locales. 

0.3.5 to 0.4.0
--------------
- The pysys.process.ProcessUser class has been added to define an interface to 
  subclasses which use the underlying process helper classes. Both the BaseTest and 
  BaseRunner classes now extend this so as to provide a common interface for process 
  manipulation. A common paradigm for creating extension modules to PySys is to create 
  a helper class which provides the methods for starting an interacting with the 
  application under test (AUT). These helper classes have a call back to an instance 
  of the ProcessUser so that it can make use of the high level process methods. As both 
  the BaseTest and BaseRunner classes are instances of the ProcessUser, the extension 
  module helper classes can be used in extensions to both of these to allow the AUT to 
  be started both within a testcase, and within the runner.
- The method signature to the pysys.utils.filereplace replace method has been changed
  to set the default value for the marker to the empty string
- Bugs fixes for cleaning up leakage of threads from the process helpers, and file 
  handle leakage from the base runner classes.

0.3.4 to 0.3.5
--------------
- Fixed a bug a testcase was not being marked as BLOCKED when unable to start a 
  process using the process helper module.
- Failure on the assertOrderedGrep now prints out the line the failure occurred on.

0.3.3 to 0.3.4
--------------
- Fixed a bug where timedout processes started in the foreground were not being stopped 
  automatically at the end of the testcase.
  
0.3.2 to 0.3.3
--------------
- The default name of the PySys test descriptor file has been changed from 
  "descriptor.xml", to ".pysystest". This change is to maintain a consistent
  naming convention across configuration files within the framework, e.g. the 
  project file ".pysysproject" denotes the project root and project specific 
  information, whilst a test file ".pysystest" denotes a testcase within the 
  project, and contains meta data for the test. Support for the previous name
  is maintained, though it should be noted that testcases created with the 
  'pysys.py make' command will have the new naming convention used.
- The windows installer has been updated to add shortcuts to the uninstaller, 
  and to create a separate directory for the inclusion of project extensions. 
- The getInstanceCount method has been added to the pysys.basetest.BaseTest class
  to reference count the number of named processes started during a test run. The
  startProcess method of the class adds a reference count to an internal dictionary 
  structure keyed on the displayName passed into the method to achieve this. 
- The writeProcess method has been added to the pysys.basetest.BaseTest class to 
  provide a wrapper around the write method of the underlying process helper class.
  This wrapper perform a check on the running status of the process prior to the 
  write, and performs additional logging to the run.log to audit the write. 
- Fixed a bug in the replace method of the filereplace module, where the method 
  signature was missing the marker parameter
- Added support to the pysys project file to allow adding path locations to the 
  Python path. See the .pysysproject file in pysys-examples for more detail.

0.3.1 to 0.3.2
--------------
- Release was superceded immediately by the 0.3.3 release. See release notes for 
  new features for 0.3.3 for more information.

0.3.0 to 0.3.1
--------------
- The process helper modules have been updated to allow the writing to the stdin 
  of a process via the write() method on the process handle. 
- Several bug fixes have been applied to the unix process helper module.
- The pysys-examples/internal directory has been added to the examples module. 
  This will contain internal testcases for self testing the pysys framework. 
  These have been included in the distribution as examples of the use of the 
  framework.
- The pysys project file has been added into the framework to allow the setting 
  of project specific constants within the application. The project file should be 
  written to the base location of the project, with the filename .pysysproject. 
  The location of this file denotes the root location of the project. For an 
  example of the file see pysys-examples/.pysysproject. Any name value properties
  within the file will be set as data attributes of the pysys.Project class, which 
  is referenced in the pysys.constants module using the variable PROJECT. 

0.2.2 to 0.3.0
--------------
- Updates to the epydoc output for documenting the classes and modules of pysys
- Addition of the pysys.py module for printing, running and making new testcase
  directory structures. This allows a single distributed script to be used to 
  perform all functionality available from the console. 
- Remove of the run method from the console launch helper.

0.2.1 to 0.2.2
--------------
- The suites element in the test descriptor has been renamed to groups. This is to 
  allow testcases in a single directory to be classified as a single testsuite, and 
  subsets thereof to be regarded as a group
- Minor bug fixes to the manual tester and process module

0.2.0 to 0.2.1
--------------
- The Manual Tester has been updated to support the <expectedresult> element in the 
  input xml file. This allows display of the expected result for a manual step to 
  be presented in the user interface. The ability to optionally record a defect in 
  the log output is also now included.
- The createDescriptors method has been removed from the pysys.launcher.console 
  package and moved into pysys.launcher. This allows the utility method to be 
  used for other custom launchers.

0.1.7 to 0.2.0
--------------
- This release includes updates to the Python doc strings for automated generation of 
  epydoc using the Epydoc package (http://epydoc.sourceforge.net). The installer now 
  distributes the generated epydoc in the site-packages/pysys-doc directory. For 
  windows installs a link to the epydoc and release notes is now added as a link in the 
  start menu items
- Added the setup() method to the BaseTest class to allow custom setup actions to be 
  performed prior to execution of a particular test case
- Fixed a bug where if the --type option was not supplied to ConsoleMakeTestHelper
  as a command line option, the resulting descriptor had type="None"

0.1.6 to 0.1.7
--------------
- The Manual Tester UI has been updated so that it can be resized, and is easier to 
  navigate through the tests. Many thanks to Tim McCormick who contributed to the UI 
  improvements
- The BaseRunner start method now takes an optional list of result writer class instances 
  to perform test audit logging at the end of a test cycle. This allows custom result 
  writers to be passed to the runner to, for example, write the results to a database, 
  proprietary system etc
  
0.1.5 to 0.1.6
--------------
- Added the ability to differentiate between automated and manual testcases using the 
  test attribute to the pysystest element in the testcase descriptor. If the attribute 
  is not present the test will be assumed to be an automated test. The runTest and 
  printTest launch helpers allow you to differentiate between automated and manual 
  tests using the --type command line argument. For more information see the examples 
  in pysys-examples

0.1.4 to 0.1.5
--------------
- Added support for the requirements traceability. This includes printing requirements 
  covered by a set of testcases, and running testcases which cover a particular 
  requirement id

0.1.3 to 0.1.4
--------------
- Added the ConsoleMakeTestHelper class to pysys.launcher.console. This facilitates 
  the creation of new testcase structures from the command line. Updated 
  pysys-examples/fibonacci to demonstrate the use of the utility class 
