Mercurial > bitten > bitten-test

comparison doc/reports.txt @ 818:fc90ef7b2b86

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Moving the wiki:ReportFormats page into the repository.
author osimons
date Thu, 16 Sep 2010 09:35:36 +0000
parents
children e36f9b446976
comparison
equal deleted inserted replaced
816:947eb92a7400 818:fc90ef7b2b86
1 .. -*- mode: rst; encoding: utf-8 -*-
2
3 ==============
4 Report Formats
5 ==============
6
7 The base element of the report must be "report" and have an attribute
8 "category" that is one of "test", "coverage" or "lint":
9
10 .. code-block:: xml
11
12 <report category="test|coverage|lint">
13 </report>
14
15 Inside the report there must be elements for each report type. The way the
16 data is captured is pretty flexible because it can either be in attributes
17 or in child elements.
18
19 Test Reports
20 ============
21
22 Test reports must have sub-elements of report of type ``<test />``. These
23 elements can have any of these attributes (or subelements with contained
24 cdata):
25
26 +-----------------+----------------------------------------------------------+
27 | Attribute | Description |
28 +=================+==========================================================+
29 | ``duration`` | Duration of test (float) |
30 +-----------------+----------------------------------------------------------+
31 | ``status`` | "success", "failure", "error", or "ignore" (string) |
32 +-----------------+----------------------------------------------------------+
33 | ``name`` | Name of the test (string) |
34 +-----------------+----------------------------------------------------------+
35 | ``fixture`` | Name of the test fixture (string) |
36 +-----------------+----------------------------------------------------------+
37 | ``file`` | Path to test file relative to the base path for the |
38 | | build configuration (string) |
39 +-----------------+----------------------------------------------------------+
40 | ``stdout`` | The output from the test (string) |
41 +-----------------+----------------------------------------------------------+
42 | ``traceback`` | The traceback from any error or failure (string) |
43 +-----------------+----------------------------------------------------------+
44
45 Example:
46
47 .. code-block:: xml
48
49 <report category="test">
50 <test duration="0.073"
51 status="success"
52 fixture="bitten.tests.model.BuildConfigTestCase"
53 name="test_config_update_name"
54 file="bitten/tests/model.py"
55 stdout="Renaming build configuration"
56 traceback="None">
57 </test>
58 <test>
59 <duration>0.073</duration>
60 <status>success</status>
61 <fixture>bitten.tests.model.BuildConfigTestCase</fixture>
62 <name>test_config_update_name</name>
63 <file>bitten/tests/model.py</file>
64 <stdout>Renaming build configuration</stdout>
65 </test>
66 </report>
67
68 Coverage Reports
69 ================
70
71 Coverage reports must be inside ``<coverage />`` elements. The
72 allowable attributes are:
73
74 +-----------------+----------------------------------------------------------+
75 | Attribute | Description |
76 +=================+==========================================================+
77 | ``name`` | The name of the module being tested for coverage |
78 +-----------------+----------------------------------------------------------+
79 | ``file`` | The name of the file relative to the base path in the |
80 | | build configuration |
81 +-----------------+----------------------------------------------------------+
82 | ``percentage`` | The percentage of lines in that file covered |
83 +-----------------+----------------------------------------------------------+
84 | ``lines`` | The number of lines covered |
85 +-----------------+----------------------------------------------------------+
86 | ``line_hits`` | Line-by-line coverage of the file, where code lines have |
87 | | 0 or more times covered and non-code lines are marked |
88 | | as `'-'` (optional) |
89 +-----------------+----------------------------------------------------------+
90
91 Example:
92
93 .. code-block:: xml
94
95 <report category="coverage">
96 <coverage name="my_module"
97 file="my_module.py"
98 percentage="75"
99 lines="4"
100 line_hits="2 0 - 1 1">
101 </coverage>
102 </report>
103
104 Lint Reports
105 ============
106
107 Lint issues are placed inside ``<problem />`` elements, with allowed attributes of:
108
109 +-----------------+----------------------------------------------------------+
110 | Attribute | Description |
111 +=================+==========================================================+
112 | ``file`` | The name of the file relative to the base path in the |
113 | | build configuration |
114 +-----------------+----------------------------------------------------------+
115 | ``tag`` | Class, method or other useful identifiable location |
116 | | inside the file |
117 +-----------------+----------------------------------------------------------+
118 | ``line`` | Line number |
119 +-----------------+----------------------------------------------------------+
120 | ``category`` | Category for problem; |
121 | | ``convention \| warning \| refactor \| error`` |
122 +-----------------+----------------------------------------------------------+
123
124 Example:
125
126 .. code-block:: xml
127
128 <report category="lint">
129 <problem category="convention"
130 line="17"
131 tag="TestResultsChartGenerator"
132 file="bitten/report/testing.py">
133 Missing docstring
134 </problem>
135 </report>
Copyright (C) 2012-2017 Edgewall Software