osimons@819: .. -*- mode: rst; encoding: utf-8 -*-
osimons@819: 
osimons@819: ==============
osimons@819: Report Formats
osimons@819: ==============
osimons@819: 
osimons@819: The base element of the report must be "report" and have an attribute
osimons@819: "category" that is one of "test", "coverage" or "lint":
osimons@819: 
osimons@819: .. code-block:: xml
osimons@819: 
osimons@819:     <report category="test|coverage|lint">
osimons@819:     </report>
osimons@819: 
osimons@819: Inside the report there must be elements for each report type.  The way the
osimons@819: data is captured is pretty flexible because it can either be in attributes
osimons@819: or in child elements.
osimons@819: 
osimons@819: Test Reports
osimons@819: ============
osimons@819: 
osimons@819: Test reports must have sub-elements of report of type ``<test />``.  These
osimons@819: elements can have any of these attributes (or subelements with contained
osimons@819: cdata):
osimons@819: 
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | Attribute       | Description                                              |
osimons@819: +=================+==========================================================+
osimons@819: | ``duration``    | Duration of test (float)                                 |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``status``      | "success", "failure", "error", or "ignore" (string)      |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``name``        | Name of the test (string)                                |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``fixture``     | Name of the test fixture (string)                        |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``file``        | Path to test file relative to the base path for the      |
osimons@819: |                 | build configuration (string)                             |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``stdout``      | The output from the test (string)                        |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``traceback``   | The traceback from any error or failure (string)         |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: 
osimons@819: Example:
osimons@819: 
osimons@819: .. code-block:: xml
osimons@819: 
osimons@819:     <report category="test">
osimons@819:        <test duration="0.073" 
osimons@819:              status="success"
osimons@819:              fixture="bitten.tests.model.BuildConfigTestCase"
osimons@819:              name="test_config_update_name" 
osimons@819:              file="bitten/tests/model.py"
osimons@819:              stdout="Renaming build configuration"
osimons@819:              traceback="None">
osimons@819:        </test>
osimons@819:        <test>
osimons@819:          <duration>0.073</duration>
osimons@819:          <status>success</status>
osimons@819:          <fixture>bitten.tests.model.BuildConfigTestCase</fixture>
osimons@819:          <name>test_config_update_name</name>
osimons@819:          <file>bitten/tests/model.py</file>
osimons@819:          <stdout>Renaming build configuration</stdout>
osimons@819:        </test>
osimons@819:     </report>
osimons@819: 
osimons@819: Coverage Reports
osimons@819: ================
osimons@819: 
osimons@819: Coverage reports must be inside ``<coverage />`` elements.  The
osimons@819: allowable attributes are:
osimons@819: 
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | Attribute       | Description                                              |
osimons@819: +=================+==========================================================+
osimons@819: | ``name``        | The name of the module being tested for coverage         |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``file``        | The name of the file relative to the base path in the    |
osimons@819: |                 | build configuration                                      |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``percentage``  | The percentage of lines in that file covered             |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``lines``       | The number of lines covered                              |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``line_hits``   | Line-by-line coverage of the file, where code lines have |
osimons@819: |                 | 0 or more times covered and non-code lines are  marked   |
osimons@819: |                 | as `'-'` (optional)                                      |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: 
osimons@819: Example:
osimons@819: 
osimons@819: .. code-block:: xml
osimons@819: 
osimons@819:     <report category="coverage">
osimons@819:        <coverage name="my_module"
osimons@819:                  file="my_module.py"
osimons@819:                  percentage="75"
osimons@819:                  lines="4"
osimons@819:                  line_hits="2 0 - 1 1">
osimons@819:         </coverage>
osimons@819:     </report>
osimons@819: 
osimons@819: Lint Reports
osimons@819: ============
osimons@819: 
osimons@819: Lint issues are placed inside ``<problem />`` elements, with allowed attributes of:
osimons@819: 
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | Attribute       | Description                                              |
osimons@819: +=================+==========================================================+
osimons@819: | ``file``        | The name of the file relative to the base path in the    |
osimons@819: |                 | build configuration                                      |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``tag``         | Class, method or other useful identifiable location      |
osimons@819: |                 | inside the file                                          |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``line``        | Line number                                              |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: | ``category``    | Category for problem;                                    |
osimons@819: |                 | ``convention \| warning \| refactor \| error``           |
osimons@819: +-----------------+----------------------------------------------------------+
osimons@819: 
osimons@819: Example:
osimons@819: 
osimons@819: .. code-block:: xml
osimons@819: 
osimons@819:     <report category="lint">
osimons@819:        <problem category="convention"
osimons@819:                 line="17"
osimons@819:                 tag="TestResultsChartGenerator"
osimons@819:                 file="bitten/report/testing.py">
osimons@819:             Missing docstring
osimons@819:         </problem>
osimons@819:     </report>