|
819
|
1 .. -*- mode: rst; encoding: utf-8 -*-
|
|
|
2
|
|
|
3 ==============
|
|
|
4 Report Formats
|
|
|
5 ==============
|
|
|
6
|
|
|
7 The base element of the report must be "report" and have an attribute
|
|
|
8 "category" that is one of "test", "coverage" or "lint":
|
|
|
9
|
|
|
10 .. code-block:: xml
|
|
|
11
|
|
|
12 <report category="test|coverage|lint">
|
|
|
13 </report>
|
|
|
14
|
|
|
15 Inside the report there must be elements for each report type. The way the
|
|
|
16 data is captured is pretty flexible because it can either be in attributes
|
|
|
17 or in child elements.
|
|
|
18
|
|
|
19 Test Reports
|
|
|
20 ============
|
|
|
21
|
|
|
22 Test reports must have sub-elements of report of type ``<test />``. These
|
|
|
23 elements can have any of these attributes (or subelements with contained
|
|
|
24 cdata):
|
|
|
25
|
|
|
26 +-----------------+----------------------------------------------------------+
|
|
|
27 | Attribute | Description |
|
|
|
28 +=================+==========================================================+
|
|
|
29 | ``duration`` | Duration of test (float) |
|
|
|
30 +-----------------+----------------------------------------------------------+
|
|
|
31 | ``status`` | "success", "failure", "error", or "ignore" (string) |
|
|
|
32 +-----------------+----------------------------------------------------------+
|
|
|
33 | ``name`` | Name of the test (string) |
|
|
|
34 +-----------------+----------------------------------------------------------+
|
|
|
35 | ``fixture`` | Name of the test fixture (string) |
|
|
|
36 +-----------------+----------------------------------------------------------+
|
|
|
37 | ``file`` | Path to test file relative to the base path for the |
|
|
|
38 | | build configuration (string) |
|
|
|
39 +-----------------+----------------------------------------------------------+
|
|
|
40 | ``stdout`` | The output from the test (string) |
|
|
|
41 +-----------------+----------------------------------------------------------+
|
|
|
42 | ``traceback`` | The traceback from any error or failure (string) |
|
|
|
43 +-----------------+----------------------------------------------------------+
|
|
|
44
|
|
|
45 Example:
|
|
|
46
|
|
|
47 .. code-block:: xml
|
|
|
48
|
|
|
49 <report category="test">
|
|
|
50 <test duration="0.073"
|
|
|
51 status="success"
|
|
|
52 fixture="bitten.tests.model.BuildConfigTestCase"
|
|
|
53 name="test_config_update_name"
|
|
|
54 file="bitten/tests/model.py"
|
|
|
55 stdout="Renaming build configuration"
|
|
|
56 traceback="None">
|
|
|
57 </test>
|
|
|
58 <test>
|
|
|
59 <duration>0.073</duration>
|
|
|
60 <status>success</status>
|
|
|
61 <fixture>bitten.tests.model.BuildConfigTestCase</fixture>
|
|
|
62 <name>test_config_update_name</name>
|
|
|
63 <file>bitten/tests/model.py</file>
|
|
|
64 <stdout>Renaming build configuration</stdout>
|
|
|
65 </test>
|
|
|
66 </report>
|
|
|
67
|
|
|
68 Coverage Reports
|
|
|
69 ================
|
|
|
70
|
|
|
71 Coverage reports must be inside ``<coverage />`` elements. The
|
|
|
72 allowable attributes are:
|
|
|
73
|
|
|
74 +-----------------+----------------------------------------------------------+
|
|
|
75 | Attribute | Description |
|
|
|
76 +=================+==========================================================+
|
|
|
77 | ``name`` | The name of the module being tested for coverage |
|
|
|
78 +-----------------+----------------------------------------------------------+
|
|
|
79 | ``file`` | The name of the file relative to the base path in the |
|
|
|
80 | | build configuration |
|
|
|
81 +-----------------+----------------------------------------------------------+
|
|
|
82 | ``percentage`` | The percentage of lines in that file covered |
|
|
|
83 +-----------------+----------------------------------------------------------+
|
|
|
84 | ``lines`` | The number of lines covered |
|
|
|
85 +-----------------+----------------------------------------------------------+
|
|
|
86 | ``line_hits`` | Line-by-line coverage of the file, where code lines have |
|
|
|
87 | | 0 or more times covered and non-code lines are marked |
|
|
|
88 | | as `'-'` (optional) |
|
|
|
89 +-----------------+----------------------------------------------------------+
|
|
|
90
|
|
|
91 Example:
|
|
|
92
|
|
|
93 .. code-block:: xml
|
|
|
94
|
|
|
95 <report category="coverage">
|
|
|
96 <coverage name="my_module"
|
|
|
97 file="my_module.py"
|
|
|
98 percentage="75"
|
|
|
99 lines="4"
|
|
|
100 line_hits="2 0 - 1 1">
|
|
|
101 </coverage>
|
|
|
102 </report>
|
|
|
103
|
|
|
104 Lint Reports
|
|
|
105 ============
|
|
|
106
|
|
|
107 Lint issues are placed inside ``<problem />`` elements, with allowed attributes of:
|
|
|
108
|
|
|
109 +-----------------+----------------------------------------------------------+
|
|
|
110 | Attribute | Description |
|
|
|
111 +=================+==========================================================+
|
|
|
112 | ``file`` | The name of the file relative to the base path in the |
|
|
|
113 | | build configuration |
|
|
|
114 +-----------------+----------------------------------------------------------+
|
|
|
115 | ``tag`` | Class, method or other useful identifiable location |
|
|
|
116 | | inside the file |
|
|
|
117 +-----------------+----------------------------------------------------------+
|
|
|
118 | ``line`` | Line number |
|
|
|
119 +-----------------+----------------------------------------------------------+
|
|
|
120 | ``category`` | Category for problem; |
|
|
|
121 | | ``convention \| warning \| refactor \| error`` |
|
|
|
122 +-----------------+----------------------------------------------------------+
|
|
|
123
|
|
|
124 Example:
|
|
|
125
|
|
|
126 .. code-block:: xml
|
|
|
127
|
|
|
128 <report category="lint">
|
|
|
129 <problem category="convention"
|
|
|
130 line="17"
|
|
|
131 tag="TestResultsChartGenerator"
|
|
|
132 file="bitten/report/testing.py">
|
|
|
133 Missing docstring
|
|
|
134 </problem>
|
|
|
135 </report>
|
