osimons@818: .. -*- mode: rst; encoding: utf-8 -*-
osimons@818: 
osimons@818: ==============
osimons@818: Report Formats
osimons@818: ==============
osimons@818: 
osimons@818: The base element of the report must be "report" and have an attribute
osimons@818: "category" that is one of "test", "coverage" or "lint":
osimons@818: 
osimons@818: .. code-block:: xml
osimons@818: 
osimons@818:     <report category="test|coverage|lint">
osimons@818:     </report>
osimons@818: 
osimons@818: Inside the report there must be elements for each report type.  The way the
osimons@818: data is captured is pretty flexible because it can either be in attributes
osimons@818: or in child elements.
osimons@818: 
osimons@818: Test Reports
osimons@818: ============
osimons@818: 
osimons@818: Test reports must have sub-elements of report of type ``<test />``.  These
osimons@818: elements can have any of these attributes (or subelements with contained
osimons@818: cdata):
osimons@818: 
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | Attribute       | Description                                              |
osimons@818: +=================+==========================================================+
osimons@818: | ``duration``    | Duration of test (float)                                 |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``status``      | "success", "failure", "error", or "ignore" (string)      |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``name``        | Name of the test (string)                                |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``fixture``     | Name of the test fixture (string)                        |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``file``        | Path to test file relative to the base path for the      |
osimons@818: |                 | build configuration (string)                             |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``stdout``      | The output from the test (string)                        |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``traceback``   | The traceback from any error or failure (string)         |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: 
osimons@818: Example:
osimons@818: 
osimons@818: .. code-block:: xml
osimons@818: 
osimons@818:     <report category="test">
osimons@818:        <test duration="0.073" 
osimons@818:              status="success"
osimons@818:              fixture="bitten.tests.model.BuildConfigTestCase"
osimons@818:              name="test_config_update_name" 
osimons@818:              file="bitten/tests/model.py"
osimons@818:              stdout="Renaming build configuration"
osimons@818:              traceback="None">
osimons@818:        </test>
osimons@818:        <test>
osimons@818:          <duration>0.073</duration>
osimons@818:          <status>success</status>
osimons@818:          <fixture>bitten.tests.model.BuildConfigTestCase</fixture>
osimons@818:          <name>test_config_update_name</name>
osimons@818:          <file>bitten/tests/model.py</file>
osimons@818:          <stdout>Renaming build configuration</stdout>
osimons@818:        </test>
osimons@818:     </report>
osimons@818: 
osimons@818: Coverage Reports
osimons@818: ================
osimons@818: 
osimons@818: Coverage reports must be inside ``<coverage />`` elements.  The
osimons@818: allowable attributes are:
osimons@818: 
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | Attribute       | Description                                              |
osimons@818: +=================+==========================================================+
osimons@818: | ``name``        | The name of the module being tested for coverage         |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``file``        | The name of the file relative to the base path in the    |
osimons@818: |                 | build configuration                                      |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``percentage``  | The percentage of lines in that file covered             |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``lines``       | The number of lines covered                              |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``line_hits``   | Line-by-line coverage of the file, where code lines have |
osimons@818: |                 | 0 or more times covered and non-code lines are  marked   |
osimons@818: |                 | as `'-'` (optional)                                      |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: 
osimons@818: Example:
osimons@818: 
osimons@818: .. code-block:: xml
osimons@818: 
osimons@818:     <report category="coverage">
osimons@818:        <coverage name="my_module"
osimons@818:                  file="my_module.py"
osimons@818:                  percentage="75"
osimons@818:                  lines="4"
osimons@818:                  line_hits="2 0 - 1 1">
osimons@818:         </coverage>
osimons@818:     </report>
osimons@818: 
osimons@818: Lint Reports
osimons@818: ============
osimons@818: 
osimons@818: Lint issues are placed inside ``<problem />`` elements, with allowed attributes of:
osimons@818: 
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | Attribute       | Description                                              |
osimons@818: +=================+==========================================================+
osimons@818: | ``file``        | The name of the file relative to the base path in the    |
osimons@818: |                 | build configuration                                      |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``tag``         | Class, method or other useful identifiable location      |
osimons@818: |                 | inside the file                                          |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``line``        | Line number                                              |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: | ``category``    | Category for problem;                                    |
osimons@818: |                 | ``convention \| warning \| refactor \| error``           |
osimons@818: +-----------------+----------------------------------------------------------+
osimons@818: 
osimons@818: Example:
osimons@818: 
osimons@818: .. code-block:: xml
osimons@818: 
osimons@818:     <report category="lint">
osimons@818:        <problem category="convention"
osimons@818:                 line="17"
osimons@818:                 tag="TestResultsChartGenerator"
osimons@818:                 file="bitten/report/testing.py">
osimons@818:             Missing docstring
osimons@818:         </problem>
osimons@818:     </report>