Mercurial > bitten > bitten-test
annotate bitten/recipe.py @ 152:56027862f910
Fix global/local ref name conflict introduced in [159].
author | cmlenz |
---|---|
date | Mon, 22 Aug 2005 08:52:27 +0000 |
parents | f3f5895e373c |
children | 2efdc69e63c3 |
rev | line source |
---|---|
21 | 1 # -*- coding: iso8859-1 -*- |
2 # | |
3 # Copyright (C) 2005 Christopher Lenz <cmlenz@gmx.de> | |
4 # | |
5 # Bitten is free software; you can redistribute it and/or | |
6 # modify it under the terms of the GNU General Public License as | |
7 # published by the Free Software Foundation; either version 2 of the | |
8 # License, or (at your option) any later version. | |
9 # | |
10 # Trac is distributed in the hope that it will be useful, | |
11 # but WITHOUT ANY WARRANTY; without even the implied warranty of | |
12 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU | |
13 # General Public License for more details. | |
14 # | |
15 # You should have received a copy of the GNU General Public License | |
16 # along with this program; if not, write to the Free Software | |
17 # Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. | |
18 # | |
19 # Author: Christopher Lenz <cmlenz@gmx.de> | |
20 | |
146
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
21 import keyword |
72
b2d371dac270
Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed.
cmlenz
parents:
68
diff
changeset
|
22 import logging |
129
efd3df0de93a
Canonicalize path names so that comparison works. Fixes #37.
cmlenz
parents:
109
diff
changeset
|
23 import os |
80
dc1c7fc9b915
Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed.
cmlenz
parents:
72
diff
changeset
|
24 import time |
21 | 25 |
72
b2d371dac270
Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed.
cmlenz
parents:
68
diff
changeset
|
26 from bitten.build import BuildError |
51
5caccd7b247e
Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}.
cmlenz
parents:
21
diff
changeset
|
27 from bitten.util import xmlio |
21 | 28 |
29 __all__ = ['Recipe'] | |
30 | |
93
b289e572bc7e
Improved logging; the build master can now optionally log to a file. Closes #13.
cmlenz
parents:
80
diff
changeset
|
31 log = logging.getLogger('bitten.recipe') |
b289e572bc7e
Improved logging; the build master can now optionally log to a file. Closes #13.
cmlenz
parents:
80
diff
changeset
|
32 |
21 | 33 |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
34 class InvalidRecipeError(Exception): |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
35 """Exception raised when a recipe cannot be processed.""" |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
36 |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
37 |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
38 class Context(object): |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
39 """The context in which a recipe command or report is run.""" |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
40 |
80
dc1c7fc9b915
Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed.
cmlenz
parents:
72
diff
changeset
|
41 current_step = None |
dc1c7fc9b915
Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed.
cmlenz
parents:
72
diff
changeset
|
42 current_function = None |
dc1c7fc9b915
Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed.
cmlenz
parents:
72
diff
changeset
|
43 |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
44 def __init__(self, basedir): |
129
efd3df0de93a
Canonicalize path names so that comparison works. Fixes #37.
cmlenz
parents:
109
diff
changeset
|
45 self.basedir = os.path.realpath(basedir) |
109
5bf22bb87915
Transmit build log and generated data back to the build master in XML format. Closes #23.
cmlenz
parents:
93
diff
changeset
|
46 self.output = [] |
80
dc1c7fc9b915
Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed.
cmlenz
parents:
72
diff
changeset
|
47 |
131
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
48 def error(self, message): |
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
49 self.output.append((Recipe.ERROR, self.current_function, message)) |
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
50 |
109
5bf22bb87915
Transmit build log and generated data back to the build master in XML format. Closes #23.
cmlenz
parents:
93
diff
changeset
|
51 def log(self, xml_elem): |
5bf22bb87915
Transmit build log and generated data back to the build master in XML format. Closes #23.
cmlenz
parents:
93
diff
changeset
|
52 self.output.append((Recipe.LOG, self.current_function, xml_elem)) |
5bf22bb87915
Transmit build log and generated data back to the build master in XML format. Closes #23.
cmlenz
parents:
93
diff
changeset
|
53 |
5bf22bb87915
Transmit build log and generated data back to the build master in XML format. Closes #23.
cmlenz
parents:
93
diff
changeset
|
54 def report(self, xml_elem): |
5bf22bb87915
Transmit build log and generated data back to the build master in XML format. Closes #23.
cmlenz
parents:
93
diff
changeset
|
55 self.output.append((Recipe.REPORT, self.current_function, xml_elem)) |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
56 |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
57 def resolve(self, *path): |
68 | 58 return os.path.normpath(os.path.join(self.basedir, *path)) |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
59 |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
60 |
21 | 61 class Step(object): |
62 """Represents a single step of a build recipe. | |
63 | |
64 Iterate over an object of this class to get the commands to execute, and | |
65 their keyword arguments. | |
66 """ | |
67 | |
51
5caccd7b247e
Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}.
cmlenz
parents:
21
diff
changeset
|
68 def __init__(self, elem): |
5caccd7b247e
Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}.
cmlenz
parents:
21
diff
changeset
|
69 self._elem = elem |
55 | 70 self.id = elem.attr['id'] |
71 self.description = elem.attr.get('description') | |
72
b2d371dac270
Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed.
cmlenz
parents:
68
diff
changeset
|
72 self.onerror = elem.attr.get('onerror', 'fail') |
21 | 73 |
74 def __iter__(self): | |
51
5caccd7b247e
Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}.
cmlenz
parents:
21
diff
changeset
|
75 for child in self._elem: |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
76 if child.namespace: # Commands |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
77 yield self._function(child), self._args(child) |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
78 elif child.name == 'reports': # Reports |
51
5caccd7b247e
Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}.
cmlenz
parents:
21
diff
changeset
|
79 for grandchild in child: |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
80 yield self._function(grandchild), self._args(grandchild) |
21 | 81 else: |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
82 raise InvalidRecipeError, "Unknown element <%s>" % child.name |
21 | 83 |
72
b2d371dac270
Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed.
cmlenz
parents:
68
diff
changeset
|
84 def execute(self, ctxt): |
80
dc1c7fc9b915
Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed.
cmlenz
parents:
72
diff
changeset
|
85 ctxt.current_step = self |
72
b2d371dac270
Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed.
cmlenz
parents:
68
diff
changeset
|
86 try: |
131
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
87 for function, args in self: |
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
88 ctxt.current_function = function.__name__ |
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
89 function(ctxt, **args) |
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
90 ctxt.current_function = None |
80
dc1c7fc9b915
Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed.
cmlenz
parents:
72
diff
changeset
|
91 finally: |
dc1c7fc9b915
Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed.
cmlenz
parents:
72
diff
changeset
|
92 ctxt.current_step = None |
131
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
93 errors = [] |
109
5bf22bb87915
Transmit build log and generated data back to the build master in XML format. Closes #23.
cmlenz
parents:
93
diff
changeset
|
94 while ctxt.output: |
144
76dea27af878
* Make the `<python:unittest>` command strip the base dir from file names in the report. Fixes #42.
cmlenz
parents:
131
diff
changeset
|
95 type, function, output = ctxt.output.pop(0) |
131
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
96 yield type, function, output |
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
97 if type == Recipe.ERROR: |
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
98 errors.append((function, output)) |
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
99 if errors: |
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
100 if self.onerror == 'fail': |
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
101 raise BuildError, 'Build step %s failed' % self.id |
146
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
102 log.warning('Ignoring errors in step %s (%s)', self.id, |
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
103 ', '.join([error[1] for error in errors])) |
72
b2d371dac270
Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed.
cmlenz
parents:
68
diff
changeset
|
104 |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
105 def _args(self, elem): |
146
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
106 return dict([(self._translate_name(name), value) for name, value |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
107 in elem.attr.items()]) |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
108 |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
109 def _function(self, elem): |
51
5caccd7b247e
Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}.
cmlenz
parents:
21
diff
changeset
|
110 if not elem.namespace.startswith('bitten:'): |
5caccd7b247e
Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}.
cmlenz
parents:
21
diff
changeset
|
111 # Ignore elements in foreign namespaces |
21 | 112 return None |
146
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
113 func_name = self._translate_name(elem.name) |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
114 try: |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
115 module = __import__(elem.namespace[7:], globals(), locals(), |
147
395b67aa072e
Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents:
146
diff
changeset
|
116 [func_name]) |
146
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
117 func = getattr(module, func_name) |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
118 return func |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
119 except (ImportError, AttributeError), e: |
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
120 raise InvalidRecipeError, 'Cannot load "%s" (%s)' % (elem.name, e) |
21 | 121 |
146
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
122 def _translate_name(self, name): |
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
123 name = name.replace('-', '_') |
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
124 if keyword.iskeyword(name) or name in __builtins__: |
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
125 name = name + '_' |
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
126 return name |
affd91b4c6fb
Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile.
cmlenz
parents:
144
diff
changeset
|
127 |
21 | 128 |
129 class Recipe(object): | |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
130 """A build recipe. |
21 | 131 |
132 Iterate over this object to get the individual build steps in the order they | |
131
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
133 have been defined in the recipe file. |
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
134 """ |
21 | 135 |
131
3ed8f568f60a
Fix error handling so that reports are still generated even if a command has failed.
cmlenz
parents:
129
diff
changeset
|
136 ERROR = 'error' |
109
5bf22bb87915
Transmit build log and generated data back to the build master in XML format. Closes #23.
cmlenz
parents:
93
diff
changeset
|
137 LOG = 'log' |
5bf22bb87915
Transmit build log and generated data back to the build master in XML format. Closes #23.
cmlenz
parents:
93
diff
changeset
|
138 REPORT = 'report' |
5bf22bb87915
Transmit build log and generated data back to the build master in XML format. Closes #23.
cmlenz
parents:
93
diff
changeset
|
139 |
148
f3f5895e373c
Fixes to problems in recipe handling introduced in [155].
cmlenz
parents:
147
diff
changeset
|
140 def __init__(self, xml, basedir=os.getcwd()): |
f3f5895e373c
Fixes to problems in recipe handling introduced in [155].
cmlenz
parents:
147
diff
changeset
|
141 assert isinstance(xml, xmlio.ParsedElement) |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
142 self.ctxt = Context(basedir) |
148
f3f5895e373c
Fixes to problems in recipe handling introduced in [155].
cmlenz
parents:
147
diff
changeset
|
143 self._root = xml |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
144 self.description = self._root.attr.get('description') |
21 | 145 |
146 def __iter__(self): | |
147 """Provide an iterator over the individual steps of the recipe.""" | |
60
055a6c666fa8
* Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc.
cmlenz
parents:
55
diff
changeset
|
148 for child in self._root.children('step'): |
21 | 149 yield Step(child) |