Metadata-Version: 1.1
Name: collective.funkload
Version: 0.4
Summary: Zope and Plone focussed extensions to funkload
Home-page: https://github.com/collective/collective.funkload
Author: Ross Patterson
Author-email: me@rpatterson.net
License: GPL
Description: .. -*-doctest-*-
        
        ===================
        collective.funkload
        ===================
        
        Complex functional load testing and benchmarking
        
        ``collective.funkload`` provides some extensions of `Funkload
        <http://funkload.nuxeo.org/>`_, a web performance testing and
        reporting tool.  These extensions provide flexible yet simple ways to:
        
          - run benchmarks of multiple test scenarios
          - run these benchmarks against multiple setups
          - generate comparisons between those setups
        
        All of the console scripts provided by collective.funkload provide a
        "--help" option which documents the command and its options.
        
        .. contents:: Table of Contents
        
        The collective.funkload Workflow
        ================================
        
        1. Develop test scenarios in Python eggs as with unittest
        ---------------------------------------------------------
        
        Funkload test cases can be generated using the `funkload test recorder
        <http://funkload.nuxeo.org/#test-recorder>`_ and then placed in a
        Python egg's "tests" package as with normal `unittest
        <http://docs.python.org/dev/library/unittest.html>`_ test cases.
        
        These test cases should be developed which reflect all of the
        application's supported usage patterns.  Take care to balance
        between separating tests by usage scenario (anonymous visitors,
        read access, write access, etc.) and keeping the number of tests
        low enough to scan results.
        
        2. Benchmark the baseline setup
        -------------------------------
        
        If there is no single baseline, such as when comparing multiple
        setups to each other, then the term baseline here might be
        slightly inaccurate.  The important part, however, is to
        establish that the test cases successfully cover the usage
        scenarios.
        
        Use the fl-run-bench provided by collective.funkload using
        `zope.testing.testrunner
        <http://pypi.python.org/pypi/zope.testing#test-runner>`_ semantics to
        specify the tests to run and the "--label" option to specify a label
        indicating the benchmark it the baseline (or some other label if
        baseline isn't appropriate)::
        
          $ fl-run-bench -s foo.loadtests --label=baseline
          Running zope.testing.testrunner.layer.UnitTests tests:
            Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds.
          ========================================================================
          Benching FooTestCase.test_FooTest
          ========================================================================
          ...
          Bench status: **SUCCESS**
            Ran # tests with 0 failures and 0 errors in # minutes #.### seconds.
          Tearing down left over layers:
            Tear down zope.testing.testrunner.layer.UnitTests in 0.000 seconds.
        
        Use "fl-build-report --html" to build an HTML report from the XML
        generated by running the benchmark above::
        
          $ fl-build-report --html FooTest-bench-YYYYMMDDThhmmss.xml
          Creating html report ...done:
          file:///.../test_ReadOnly-YYYYMMDDThhmmss-baseline/index.html
        
        Examine the report details.  If the test cases don't sufficiently
        cover the application's supported useage patterns, repeat steps 1
        and 2 until the test cases provide sufficient coverage.
        
        3. Benchmark the other setups
        -----------------------------
        
        In turn, deploy each of the setups.  This procedure will be
        dictated by the application.  If using different buildout
        configurations, for example, deploy each configuration::
        
          $ ...
        
        Then use the same fl-run-bench command as before (or adusted as
        needed for the setup) giving a differen "--label" option
        dsignating the setup::
        
          $ fl-run-bench -s foo.loadtests --label=foo-setup
          Running zope.testing.testrunner.layer.UnitTests tests:
            Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds.
          ========================================================================
          Benching FooTestCase.test_FooTest
          ========================================================================
          ...
          Bench status: **SUCCESS**
            Ran # tests with 0 failures and 0 errors in # minutes #.### seconds.
          Tearing down left over layers:
            Tear down zope.testing.testrunner.layer.UnitTests in 0.000 seconds.
        
        Repeat this step for each setup.
        
        4. Build the HTML and differential reports and the matrix index
        ---------------------------------------------------------------
        
        Use the "fl-build-label-reports" command with the "--x-label" and
        "--y-label" options to automatically build all the HTML reports, the
        differential reports based on the labels, and an index matrix to the
        HTML and differential reports.  The "fl-build-label-reports" script
        will use a default title and sub-title based on the labels but may
        specified using the "--title" and "--sub-title" options.  Arbitrary
        text or HTML may also be included on stdin or using the "--input"
        option::
        
          $ echo "When deciding which setup to use..." | \
          fl-build-label-reports --x-label=baseline --y-label=foo-setup \
          --y-label=bar-setup --title="Setup Comparison"
          --sub-title="Compare setups foo and bar against baseline"
          Creating html report ...done:
          file:///.../test_ReadOnly-YYYYMMDDThhmmss-baseline/index.html
          Creating html report ...done:
          file:///.../test_ReadOnly-YYYYMMDDThhmmss-foo-label/index.html
          Creating html report ...done:
          file:///.../test_ReadOnly-YYYYMMDDThhmmss-bar-label/index.html
          Creating diff report ...done:
          /.../diff_ReadOnly-YYYYMMDDT_hhmmss-foo-label_vs_hhmmss-baseline/index.html
          Creating diff report ...done:
          /.../diff_ReadOnly-YYYYMMDDT_hhmmss-bar-label_vs_hhmmss-baseline/index.html
          Creating report index ...done:
          file:///.../index.html
        
        Both the "--x-label" and "--y-label" options may be given multiple
        times or may use `Python regular expressions
        <http://docs.python.org/dev/library/re.html>`_ to create an MxN matrix
        of differential reports.  See the "fl-build-label-reports --help"
        documentation for more details.
        
        5. Examine the results using the matrix index
        ---------------------------------------------
        
        Open the index.html generated by the last command to survey the
        HTML reports and differential reports.
        
        6. Repeat as changes are made
        -----------------------------
        
        As changes are made in your application or setups or to test new
        setups, repeat steps 3 and 4.  When step 4 is repeated by running
        "fl-build-label-reports" adjusting the  "--x-label" and
        "--y-label" options as appropriate, new HTML and differential
        reports will be generated as appropriate for the new load test
        benchmark results and the matrix index will be updated.
        
        fl-run-bench
        ============
        
        The scripts that Funkload installs generally require that they be
        executed from the directory where the test modules live.  While this
        is appropriate for generating test cases with the Funkload recorder,
        it's often not the desirable behavior when running load test
        benchmarks.  Additionally, the argument handling for the benchmark
        runner doesn't allow for specifying which tests to benchmark using the
        zope.testing.testrunner semantics, such as specifying modules and
        packages with dotted paths, as one is often wont to do when working
        with setuptools and eggs.
        
        To accommodate this usage pattern, the collective.funkload package
        provides a wrapper around the Funkload benchmark runner that handles
        dotted path arguments gracefully.  Specifically, rather than pass
        ``*.py`` file and TestCase.test_method arguments, the
        "fl-bench-runner" provided by collective.funkload supports
        zope.testing.testrunner semantics for finding tests with "-s", "-m"
        and "-t".
        
            >>> from collective.funkload import bench
            >>> bench.run(defaults, (
            ...     'test.py -s foo -t test_foo '
            ...     '--cycles 1 --url http://bar.com').split())
            t...
            Benching FooTestCase.test_foo...
            * Server: http://bar.com...
            * Cycles: [1]...
        
        .. -*-doctest-*-
        
        fl-build-label-reports
        ======================
        
        The fl-build-label-reports script builds HTML (fl-build-report --html)
        and differential (fl-build-report --diff) reports for multiple bench
        results at once based on the bench result labels.  Labels are selected
        for the X and Y axes to be compared against each other using the
        "--x-label" and "--y-label" options.  These options accept the same
        regular expression filters as the zope.testing.testrunner --module and
        --test options and like those options maybe given multiple times.
        
        The direction or polarity of the differential reports, which report is
        the reference and which report is the challenger, is determined by
        sorting the labels involved.  This avoids confusion that could occur
        if differential reports of both directions are included the same
        matrix, one showing green and the other read.  As such, labels should
        be specified such that their sort order will reflect the desired
        differential polarity.  The "--reverse" option can also be used to
        reverse the sort order for polarity only without affecting the sort
        order used on the axes.  IOW, when the polarity of the differentials
        should be the reverse of the order of the axes, use "--reverse".
        
        The title and sub-title rendered on the matrix index may be specified
        using the "--title" and "--sub-title" options.  If not specified a
        default title will be used and a sub-title will be generated based on
        the labels on the X and Y axes.  Arbitrary text or HTML may also be
        included on stdin or using the "--input" option.  If provided, it will
        be rendered beneath the sub-title and above the matrix.
        
        In the examples below, load tests have been run to measure read,
        write, and add performance under Python 2.4, 2.5, and 2.6.  There are
        three different tests to measure read, write and add performance.
        Labels are used to designate which Python version the load tests have
        been run under.  Thus fl-build-label-reports can be used to quickly
        generate reports which can be used to evaluate any performance trade
        offs the various python versions might have for the application being
        tested.
        
        Start with some bench result XML files.
        
            >>> import os
            >>> from collective.funkload import testing
            >>> testing.setUpReports(reports_dir)
            >>> sorted(os.listdir(reports_dir), reverse=True)
            ['write-bench-20081211T071242.xml',
             'write-bench-20081211T071242.log',
             'read-bench-20081211T071242.xml',
             'read-bench-20081211T071242.log',
             'read-bench-20081211T071241.xml',
             'read-bench-20081211T071241.log',
             'read-bench-20081210T071243.xml',
             'read-bench-20081210T071243.log',
             'read-bench-20081210T071241.xml',
             'read-bench-20081210T071241.log',
             'add-bench-20081211T071243.xml',
             'add-bench-20081211T071243.log',
             'add-bench-20081211T071242.xml',
             'add-bench-20081211T071242.log']
        
        These bench results cover multiple tests and have multiple labels.
        Some labels are applied to bench results for multiple tests.
        
            >>> import pprint
            >>> pprint.pprint(testing.listReports(reports_dir))
            [(u'python-2.4',
              [(u'test_add',
                [(u'2008-12-11T07:12:43.000000',
                  Bench(path='add-bench-20081211T071243.xml', diffs={}))]),
               (u'test_read',
                [(u'2008-12-11T07:12:42.000000',
                  Bench(path='read-bench-20081211T071242.xml', diffs={})),
                 (u'2008-12-10T07:12:43.000000',
                  Bench(path='read-bench-20081210T071243.xml', diffs={}))])]),
             (u'python-2.5',
              [(u'test_read',
                [(u'2008-12-10T07:12:41.000000',
                  Bench(path='read-bench-20081210T071241.xml', diffs={}))])]),
             (u'python-2.6',
              [(u'test_add',
                [(u'2008-12-11T07:12:42.000000',
                  Bench(path='add-bench-20081211T071242.xml', diffs={}))]),
               (u'test_read',
                [(u'2008-12-11T07:12:41.000000',
                  Bench(path='read-bench-20081211T071241.xml', diffs={}))])])]
        
        When labels are specified for the X or Y axes, HTML reports are
        generated for the latest bench result XML file for each combination of
        the specified label and each test for which there are bench results
        available.  Then differential reports are generated between the X and
        Y axes forming a grid of reports.  Finally, an index.html file is
        generated providing clear and easy access to the generated reports.
        Generate reports and comparisons for python-2.4 vs python-2.6.  Also
        specify the "--reverse" option so that the differential polarity will
        be the reverse of the axes label order.
        
            >>> from collective.funkload import label
            >>> input_ = os.path.join(reports_dir, 'input.html')
            >>> open(input_, 'w').write('<a href="http://foo.com">foo</a>')
            >>> args = (
            ...     '-o %s --x-label python-2.4 --y-label !.*-2.5 --reverse'
            ...     % reports_dir).split() + [
            ...         '--title', 'Python 2.6 vs Python 2.4',
            ...         '--sub-title', 'Comparing Python versions',
            ...         '--input', input_]
            >>> options, _ = label.parser.parse_args(args=args)
            >>> label.run(options)
            Creating html report ...done: 
            .../reports/test_add-20081211T071242-python-2.6/index.html
            Creating html report ...done: 
            .../reports/test_read-20081211T071241-python-2.6/index.html
            Creating html report ...done: 
            .../reports/test_add-20081211T071243-python-2.4/index.html
            Creating diff report ...done: 
            .../reports/diff_add-20081211T_071242-python-2.6_vs_071243-python-2.4/index.html
            Creating html report ...done: 
            .../reports/test_read-20081211T071242-python-2.4/index.html
            Creating diff report ...done: 
            .../reports/diff_read-20081211T_071241-python-2.6_vs_071242-python-2.4/index.html
            Creating report index ...done:
            .../reports/index.html
            '.../reports/index.html'
        
        The report index renders a table with links to the HTML reports on the
        X and Y axes and links to the differential reports in the table cells.
        In this case there's only one HTML report on the X axis and four
        reports on the Y axis.  Note that report links aren't included in the
        column headers for the X axis to conserve space and avoid duplication.
        When using only one label for the X axis, it may be useful to include
        it in the Y axis even though the differential report cells will be
        empty in order to include the links to the non-differential test
        reports for each test.
        
            >>> print open(os.path.join(reports_dir, 'index.html')).read()
            <...
            <title>Python 2.6 vs Python 2.4</title>...
                 <h1 class="title">Python 2.6 vs Python 2.4</h1>
                 <h2 class="subtitle">Comparing Python versions</h2>
                 <a href="http://foo.com">foo</a>
                  <table class="docutils">
                    <thead>
                      <tr class="field">
                        <th class="field-name" colspan="2">&nbsp;</th>
                        <th class="field-name" colspan="1">
                          Label
                        </th>
                      </tr>
                      <tr class="field">
                        <th class="field-name">Label</th>
                        <th class="field-name">Test</th>
                        <th class="field-name">python-2.4</th>
                      </tr>
                    </thead>
                    <tbody>
                          <tr>
                            <th class="field-name" rowspan="2">python-2.4</th>
                            <th class="field-name">
                              <a href="test_add-20081211T071243-python-2.4/index.html">
                                <img alt="foo.sampletests.FooTestCase.test_add"
                                     src="test_add-20081211T071243-python-2.4/tests.png"
                                     height="120" width="120"/>
                                <div>test_add</div>
                              </a>
                            </th>
                            <td class="field-body">
                            </td>
                        </tr>
                        <tr>
                            <th class="field-name">
                            <a href="test_read-20081211T071242-python-2.4/index.html">
                              <img alt="foo.sampletests.FooTestCase.test_read"
                                   src="test_read-20081211T071242-python-2.4/tests.png"
                                   height="120" width="120"/>
                              <div>test_read</div>
                            </a>
                            </th>
                            <td class="field-body">
                            </td>
                        </tr>
                        <tr>
                            <th class="field-name" rowspan="2">python-2.6</th>
                            <th class="field-name">
                            <a href="test_add-20081211T071242-python-2.6/index.html">
                              <img alt="foo.sampletests.FooTestCase.test_add"
                                   src="test_add-20081211T071242-python-2.6/tests.png"
                                   height="120" width="120"/>
                              <div>test_add</div>
                            </a>
                            </th>
                            <td class="field-body">
                            <a href="diff_add-20081211T_071242-python-2.6_vs_071243-python-2.4/index.html">
                              <img alt="diff of python-2.6 vs python-2.4 for test_add"
                                   src="diff_add-20081211T_071242-python-2.6_vs_071243-python-2.4/spps_diff.png"
                                   height="95" width="160"/>
                              <div>python-2.6 vs python-2.4</div>
                            </a>
                            </td>
                        </tr>
                        <tr>
                            <th class="field-name">
                            <a href="test_read-20081211T071241-python-2.6/index.html">
                              <img alt="foo.sampletests.FooTestCase.test_read"
                                   src="test_read-20081211T071241-python-2.6/tests.png"
                                   height="120" width="120"/>
                              <div>test_read</div>
                            </a>
                            </th>
                            <td class="field-body">
                            <a href="diff_read-20081211T_071241-python-2.6_vs_071242-python-2.4/index.html">
                              <img alt="diff of python-2.6 vs python-2.4 for test_read"
                                   src="diff_read-20081211T_071241-python-2.6_vs_071242-python-2.4/spps_diff.png"
                                   height="95" width="160"/>
                              <div>python-2.6 vs python-2.4</div>
                            </a>
                            </td>
                        </tr>
                    </tbody>
                    <tfooter>
                      <tr class="field">
                        <th class="field-name">Label</th>
                        <th class="field-name">Test</th>
                        <th class="field-name">python-2.4</th>
                      </tr>
                      <tr class="field">
                        <th class="field-name" colspan="2">&nbsp;</th>
                        <th class="field-name" colspan="1">
                          Label
                        </th>
                      </tr>
                    </tfooter>
                  </table>...
        
        If no labels are specified for the X or Y axes then all labels are
        selected for both the X and Y axes for a full NxN comparison.  Both
        HTML and differential reports are only generated if they haven't been
        already.  IOW, existing reports will be re-used.  Reports or results
        without labels will be ignored.  Since the HTML report contains the
        bench run XML results file, the original is removed and any
        corresponding log file is moved into the HTML report directory.
        
            >>> open(input_, 'w').write('')
            >>> args = ('-o %s' % reports_dir).split()+['--input', input_]
            >>> options, _ = label.parser.parse_args(args=args)
            >>> label.run(options)
            Creating html report ...done: 
            .../reports/test_read-20081210T071241-python-2.5/index.html
            Creating diff report ...done: 
            .../reports/diff_read_20081211T071242-python-2.4_vs_20081210T071241-python-2.5/index.html
            Creating diff report ...done: 
            .../reports/diff_read_20081210T071241-python-2.5_vs_20081211T071241-python-2.6/index.html
            Creating report index ...done:
            .../reports/index.html
            '.../reports/index.html'
        
            >>> pprint.pprint(sorted(os.listdir(reports_dir), reverse=True))
            ['write-bench-20081211T071242.xml',
             'write-bench-20081211T071242.log',
             'test_read-20081211T071242-python-2.4',
             'test_read-20081211T071241-python-2.6',
             'test_read-20081210T071241-python-2.5',
             'test_add-20081211T071243-python-2.4',
             'test_add-20081211T071242-python-2.6',
             'read-bench-20081210T071243.xml',
             'read-bench-20081210T071243.log',
             'input.html',
             'index.html',
             'diff_read_20081211T071242-python-2.4_vs_20081210T071241-python-2.5',
             'diff_read_20081210T071241-python-2.5_vs_20081211T071241-python-2.6',
             'diff_read-20081211T_071242-python-2.4_vs_071241-python-2.6',
             'diff_read-20081211T_071241-python-2.6_vs_071242-python-2.4',
             'diff_add-20081211T_071243-python-2.4_vs_071242-python-2.6',
             'diff_add-20081211T_071242-python-2.6_vs_071243-python-2.4']
            >>> os.path.isfile(os.path.join(
            ...     reports_dir, 'test_read-20081211T071242-python-2.4',
            ...     'funkload.log'))
            True
            >>> os.path.isfile(os.path.join(
            ...     reports_dir, 'test_read-20081211T071242-python-2.4',
            ...     'funkload.xml'))
            True
            
        The HTML report index will be updated to reflect the newly included
        results and reports.
        
            >>> print open(os.path.join(reports_dir, 'index.html')).read()
            <...
                <title>
                  collective.funkload label matrix report
                </title>...
                  <h1 class="title">
                    <a href="http://pypi.python.org/pypi/collective.funkload">
                      collective.funkload label matrix report
                    </a>
                  </h1>
                  <h2 class="subtitle">python-2.4, python-2.5, python-2.6 vs python-2.4, python-2.5, python-2.6</h2>
                  <table class="docutils">
                    <thead>
                      <tr class="field">
                        <th class="field-name" colspan="2">&nbsp;</th>
                        <th class="field-name" colspan="3">
                          Label
                        </th>
                      </tr>
                      <tr class="field">
                          <th class="field-name">Label</th>
                          <th class="field-name">Test</th>
                          <th class="field-name">python-2.4</th>
                          <th class="field-name">python-2.5</th>
                          <th class="field-name">python-2.6</th>
                      </tr>
                      </thead>
                      <tbody>
                          <tr>
                              <th class="field-name" rowspan="2">python-2.4</th>
                              <th class="field-name">
                            <a href="test_add-20081211T071243-python-2.4/index.html">
                              <img alt="foo.sampletests.FooTestCase.test_add"
                                   src="test_add-20081211T071243-python-2.4/tests.png"
                                   height="120" width="120"/>
                              <div>test_add</div>
                            </a>
                              </th>
                              <td class="field-body">
                            </td>
                              <td class="field-body">
                            </td>
                              <td class="field-body">
                            <a href="diff_add-20081211T_071243-python-2.4_vs_071242-python-2.6/index.html">
                              <img alt="diff of python-2.4 vs python-2.6 for test_add"
                                   src="diff_add-20081211T_071243-python-2.4_vs_071242-python-2.6/spps_diff.png"
                                   height="95" width="160"/>
                              <div>python-2.4 vs python-2.6</div>
                            </a>
                              </td>
                          </tr>
                          <tr>
                              <th class="field-name">
                            <a href="test_read-20081211T071242-python-2.4/index.html">
                              <img alt="foo.sampletests.FooTestCase.test_read"
                                   src="test_read-20081211T071242-python-2.4/tests.png"
                                   height="120" width="120"/>
                              <div>test_read</div>
                            </a>
                              </th>
                              <td class="field-body">
                            </td>
                              <td class="field-body">
                            <a href="diff_read_20081211T071242-python-2.4_vs_20081210T071241-python-2.5/index.html">
                              <img alt="diff of python-2.4 vs python-2.5 for test_read"
                                   src="diff_read_20081211T071242-python-2.4_vs_20081210T071241-python-2.5/spps_diff.png"
                                   height="95" width="160"/>
                              <div>python-2.4 vs python-2.5</div>
                            </a>
                              </td>
                              <td class="field-body">
                            <a href="diff_read-20081211T_071242-python-2.4_vs_071241-python-2.6/index.html">
                              <img alt="diff of python-2.4 vs python-2.6 for test_read"
                                   src="diff_read-20081211T_071242-python-2.4_vs_071241-python-2.6/spps_diff.png"
                                   height="95" width="160"/>
                              <div>python-2.4 vs python-2.6</div>
                            </a>
                              </td>
                          </tr>
                          <tr>
                              <th class="field-name" rowspan="1">python-2.5</th>
                              <th class="field-name">
                            <a href="test_read-20081210T071241-python-2.5/index.html">
                              <img alt="foo.sampletests.FooTestCase.test_read"
                                   src="test_read-20081210T071241-python-2.5/tests.png"
                                   height="120" width="120"/>
                              <div>test_read</div>
                            </a>
                              </th>
                              <td class="field-body">
                            <a href="diff_read_20081211T071242-python-2.4_vs_20081210T071241-python-2.5/index.html">
                              <img alt="diff of python-2.4 vs python-2.5 for test_read"
                                   src="diff_read_20081211T071242-python-2.4_vs_20081210T071241-python-2.5/spps_diff.png"
                                   height="95" width="160"/>
                              <div>python-2.4 vs python-2.5</div>
                            </a>
                              </td>
                              <td class="field-body">
                            </td>
                              <td class="field-body">
                            <a href="diff_read_20081210T071241-python-2.5_vs_20081211T071241-python-2.6/index.html">
                              <img alt="diff of python-2.5 vs python-2.6 for test_read"
                                   src="diff_read_20081210T071241-python-2.5_vs_20081211T071241-python-2.6/spps_diff.png"
                                   height="95" width="160"/>
                              <div>python-2.5 vs python-2.6</div>
                            </a>
                              </td>
                          </tr>
                          <tr>
                              <th class="field-name" rowspan="2">python-2.6</th>
                              <th class="field-name">
                            <a href="test_add-20081211T071242-python-2.6/index.html">
                              <img alt="foo.sampletests.FooTestCase.test_add"
                                   src="test_add-20081211T071242-python-2.6/tests.png"
                                   height="120" width="120"/>
                              <div>test_add</div>
                            </a>
                              </th>
                              <td class="field-body">
                            <a href="diff_add-20081211T_071243-python-2.4_vs_071242-python-2.6/index.html">
                              <img alt="diff of python-2.4 vs python-2.6 for test_add"
                                   src="diff_add-20081211T_071243-python-2.4_vs_071242-python-2.6/spps_diff.png"
                                   height="95" width="160"/>
                              <div>python-2.4 vs python-2.6</div>
                            </a>
                              </td>
                              <td class="field-body">
                            </td>
                              <td class="field-body">
                            </td>
                          </tr>
                          <tr>
                              <th class="field-name">
                            <a href="test_read-20081211T071241-python-2.6/index.html">
                              <img alt="foo.sampletests.FooTestCase.test_read"
                                   src="test_read-20081211T071241-python-2.6/tests.png"
                                   height="120" width="120"/>
                              <div>test_read</div>
                            </a>
                              </th>
                              <td class="field-body">
                            <a href="diff_read-20081211T_071242-python-2.4_vs_071241-python-2.6/index.html">
                              <img alt="diff of python-2.4 vs python-2.6 for test_read"
                                   src="diff_read-20081211T_071242-python-2.4_vs_071241-python-2.6/spps_diff.png"
                                   height="95" width="160"/>
                              <div>python-2.4 vs python-2.6</div>
                            </a>
                              </td>
                              <td class="field-body">
                              <a href="diff_read_20081210T071241-python-2.5_vs_20081211T071241-python-2.6/index.html">
                                <img alt="diff of python-2.5 vs python-2.6 for test_read"
                                     src="diff_read_20081210T071241-python-2.5_vs_20081211T071241-python-2.6/spps_diff.png"
                                     height="95" width="160"/>
                                <div>python-2.5 vs python-2.6</div>
                              </a>
                            </td>
                            <td class="field-body">
                            </td>
                          </tr>
                    </tbody>
                    <tfooter>
                      <tr class="field">
                        <th class="field-name">Label</th>
                        <th class="field-name">Test</th>
                        <th class="field-name">python-2.4</th>
                        <th class="field-name">python-2.5</th>
                        <th class="field-name">python-2.6</th>
                      </tr>
                      <tr class="field">
                        <th class="field-name" colspan="2">&nbsp;</th>
                        <th class="field-name" colspan="3">
                          Label
                        </th>
                      </tr>
                    </tfooter>
                  </table>...
        
        The "fl-list" script prints out the labeled XML bench result files,
        HTML report directories, and differential report directories that meet
        the criteria of the given options.  The "--old" option lists
        everything for which there is an equivalent with a newer time stamp.
        
            >>> from collective.funkload import report
            >>> options, _ = report.list_parser.parse_args(
            ...     args=('-o %s --old' % reports_dir).split())
            >>> list(report.run(**options.__dict__))
            ['read-bench-20081210T071243.xml']
        
        Changelog
        =========
        
        0.4dev Unreleased
        ------------------
        
        * make flake8 happy again
          [giacomos]
        
        * add method for creating dexterity content types
          [giacomos]
        
        * Fix label.py and tests.py is not working with current zope.testing
          [joka]
        
        * Fix imports. zope.testing.testrunner now lives in the zope.testrunner
          package.
          [nueces]
        
        
        0.4 (2014-06-20)
        ----------------
        
        * add new TestCase class: PloneFLTestCase with two helper methods:
          plone_login and addContent. Please check collective.recipe.funkload
          for example usage
          [amleczko]
        
        0.2.1 - 2010-04-16 
        ------------------
        
        * fix small typo in recorder
          [amleczko]
        
        0.2 - 2010-04-16 
        ----------------
        
        * Add custom version of RecorderProgram to make usage
          of our custom Script tpl
          [amleczko]
        
        0.1.1 - 2009-08-09
        ------------------
        
        * Only run funkload tests when invoking bench with -m, -s, -t [evilbungle]
        
        
        0.1 - 2009-08-09
        ----------------
        
        * Initial release, mainly a snapshot from trunk to compliment the release of
          collective.recipe.funkload
        
        
Platform: UNKNOWN
Classifier: Programming Language :: Python
Classifier: Topic :: Software Development :: Libraries :: Python Modules
