bitten/bitten-test: bitten/recipe.py annotate

annotate bitten/recipe.py @ 630:042c8b49ce7f

0.6dev: Follow-up to [702] - flawed logic in argument testing.

author	osimons
date	Tue, 11 Aug 2009 23:20:13 +0000
parents	f3bb52da9e3c
children	075087a17f06

rev	line source
379 0df178e07fdb Use UTF-8 as encoding of source files. cmlenz parents: 377 diff changeset	1 # -- coding: utf-8 --
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	2 #
408 933105ab516b Update file headers and other stuff pointing to the old home. cmlenz parents: 392 diff changeset	3 # Copyright (C) 2007 Edgewall Software
933105ab516b Update file headers and other stuff pointing to the old home. cmlenz parents: 392 diff changeset	4 # Copyright (C) 2005-2007 Christopher Lenz <cmlenz@gmx.de>
163 634be6cbb808 Flip the switch: Bitten is now BSD-licensed. cmlenz parents: 157 diff changeset	5 # All rights reserved.
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	6 #
163 634be6cbb808 Flip the switch: Bitten is now BSD-licensed. cmlenz parents: 157 diff changeset	7 # This software is licensed as described in the file COPYING, which
634be6cbb808 Flip the switch: Bitten is now BSD-licensed. cmlenz parents: 157 diff changeset	8 # you should have received as part of this distribution. The terms
408 933105ab516b Update file headers and other stuff pointing to the old home. cmlenz parents: 392 diff changeset	9 # are also available at http://bitten.edgewall.org/wiki/License.
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	10
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	11 """Execution of build recipes.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	12
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	13 This module provides various classes that can be used to process build recipes,
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	14 most importantly the `Recipe` class.
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	15 """
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	16
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	17 import inspect
146 affd91b4c6fb Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile. cmlenz parents: 144 diff changeset	18 import keyword
72 b2d371dac270 Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed. cmlenz parents: 68 diff changeset	19 import logging
129 efd3df0de93a Canonicalize path names so that comparison works. Fixes #37. cmlenz parents: 109 diff changeset	20 import os
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	21 import time
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	22 try:
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	23 set
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	24 except NameError:
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	25 from sets import Set as set
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	26
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	27 from pkg_resources import WorkingSet
72 b2d371dac270 Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed. cmlenz parents: 68 diff changeset	28 from bitten.build import BuildError
238 832e64330c31 Add a `<c:configure>` recipe command for running configure scripts. Closes #57. cmlenz parents: 233 diff changeset	29 from bitten.build.config import Configuration
51 5caccd7b247e Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}. cmlenz parents: 21 diff changeset	30 from bitten.util import xmlio
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	31
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	32 __all__ = ['Context', 'Recipe', 'Step', 'InvalidRecipeError']
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	33 __docformat__ = 'restructuredtext en'
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	34
93 b289e572bc7e Improved logging; the build master can now optionally log to a file. Closes #13. cmlenz parents: 80 diff changeset	35 log = logging.getLogger('bitten.recipe')
b289e572bc7e Improved logging; the build master can now optionally log to a file. Closes #13. cmlenz parents: 80 diff changeset	36
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	37
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	38 class InvalidRecipeError(Exception):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	39 """Exception raised when a recipe is not valid."""
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	40
055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	41
055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	42 class Context(object):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	43 """The context in which a build is executed."""
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	44
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	45 step = None # The current step
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	46 generator = None # The current generator (namespace#name)
80 dc1c7fc9b915 Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed. cmlenz parents: 72 diff changeset	47
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	48 def __init__(self, basedir, config=None, vars=None):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	49 """Initialize the context.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	50
466 79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 411 diff changeset	51 :param basedir: a string containing the working directory for the build.
79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 411 diff changeset	52 (may be a pattern for replacement ex: 'build_${build}'
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	53 :param config: the build slave configuration
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	54 :type config: `Configuration`
466 79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 411 diff changeset	55 """
238 832e64330c31 Add a `<c:configure>` recipe command for running configure scripts. Closes #57. cmlenz parents: 233 diff changeset	56 self.config = config or Configuration()
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	57 self.vars = vars or {}
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	58 self.output = []
466 79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 411 diff changeset	59 self.basedir = os.path.realpath(self.config.interpolate(basedir,
79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 411 diff changeset	60 **self.vars))
611 294641e84e89 0.6dev: Adding `${name}` and `${basedir}` (#325) for recipe substitution. Updated docs + new test. osimons parents: 466 diff changeset	61 self.vars['basedir'] = self.basedir
80 dc1c7fc9b915 Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed. cmlenz parents: 72 diff changeset	62
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	63 def run(self, step, namespace, name, attr):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	64 """Run the specified recipe command.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	65
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	66 :param step: the build step that the command belongs to
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	67 :param namespace: the namespace URI of the command
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	68 :param name: the local tag name of the command
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	69 :param attr: a dictionary containing the attributes defined on the
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	70 command element
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	71 """
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	72 self.step = step
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	73
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	74 try:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	75 function = None
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	76 qname = '#'.join(filter(None, [namespace, name]))
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	77 if namespace:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	78 group = 'bitten.recipe_commands'
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	79 for entry_point in WorkingSet().iter_entry_points(group, qname):
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	80 function = entry_point.load()
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	81 break
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	82 elif name == 'report':
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	83 function = Context.report_file
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	84 elif name == 'attach':
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	85 function = Context.attach
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	86 if not function:
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	87 raise InvalidRecipeError('Unknown recipe command %s' % qname)
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	88
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	89 def escape(name):
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	90 name = name.replace('-', '_')
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	91 if keyword.iskeyword(name) or name in __builtins__:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	92 name = name + '_'
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	93 return name
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	94 args = dict([(escape(name),
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	95 self.config.interpolate(attr[name], **self.vars))
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	96 for name in attr])
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	97 function_args, has_kwargs = inspect.getargspec(function)[0:3:2]
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	98 for arg in args:
630 042c8b49ce7f 0.6dev: Follow-up to [702] - flawed logic in argument testing. osimons parents: 629 diff changeset	99 if not (arg in function_args or has_kwargs):
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	100 raise InvalidRecipeError(
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	101 "Unsupported argument '%s' for command %s" % \
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	102 (arg, qname))
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	103
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	104 self.generator = qname
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	105 log.debug('Executing %s with arguments: %s', function, args)
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	106 function(self, **args)
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	107
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	108 finally:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	109 self.generator = None
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	110 self.step = None
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	111
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	112 def error(self, message):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	113 """Record an error message.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	114
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	115 :param message: a string containing the error message.
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	116 """
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	117 self.output.append((Recipe.ERROR, None, self.generator, message))
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	118
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	119 def log(self, xml):
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	120 """Record log output.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	121
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	122 :param xml: an XML fragment containing the log messages
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	123 """
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	124 self.output.append((Recipe.LOG, None, self.generator, xml))
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	125
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	126 def report(self, category, xml):
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	127 """Record report data.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	128
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	129 :param category: the name of category of the report
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	130 :param xml: an XML fragment containing the report data
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	131 """
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	132 self.output.append((Recipe.REPORT, category, self.generator, xml))
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	133
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	134 def report_file(self, category=None, file_=None):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	135 """Read report data from a file and record it.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	136
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	137 :param category: the name of the category of the report
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	138 :param file\_: the path to the file containing the report data, relative
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	139 to the base directory
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	140 """
314 a8fd83c0317d Fix undefined variable in recipe error reporting. Closes #78. cmlenz parents: 313 diff changeset	141 filename = self.resolve(file_)
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	142 try:
314 a8fd83c0317d Fix undefined variable in recipe error reporting. Closes #78. cmlenz parents: 313 diff changeset	143 fileobj = file(filename, 'r')
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	144 try:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	145 xml_elem = xmlio.Fragment()
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	146 for child in xmlio.parse(fileobj).children():
377 a465b5dbfecf Tabs and line length. cmlenz parents: 370 diff changeset	147 child_elem = xmlio.Element(child.name, **dict([
a465b5dbfecf Tabs and line length. cmlenz parents: 370 diff changeset	148 (name, value) for name, value in child.attr.items()
a465b5dbfecf Tabs and line length. cmlenz parents: 370 diff changeset	149 if value is not None
a465b5dbfecf Tabs and line length. cmlenz parents: 370 diff changeset	150 ]))
370 fa5400c2879d Fix for report imports; I would have thought that the original child.attr would work since it's a DictMixin, but at runtime, we're seeing 'argument after ** must be a dictionary' wbell parents: 368 diff changeset	151 xml_elem.append(child_elem[
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	152 [xmlio.Element(grandchild.name)[grandchild.gettext()]
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	153 for grandchild in child.children()]
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	154 ])
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	155 self.output.append((Recipe.REPORT, category, None, xml_elem))
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	156 finally:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	157 fileobj.close()
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	158 except xmlio.ParseError, e:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	159 self.error('Failed to parse %s report at %s: %s'
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	160 % (category, filename, e))
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	161 except IOError, e:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	162 self.error('Failed to read %s report at %s: %s'
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	163 % (category, filename, e))
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	164
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	165 def attach(self, file_=None, description=None, resource=None):
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	166 """Attach a file to the build or build configuration.
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	167
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	168 :param file\_: the path to the file to attach, relative to
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	169 base directory.
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	170 :param description: description saved with attachment
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	171 :resource: which resource to attach the file to,
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	172 either 'build' (default) or 'config'
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	173 :replace: non-empty to replace existing attachment with same name
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	174 """
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	175 filename = self.resolve(file_)
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	176 try:
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	177 fileobj = file(filename, 'r')
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	178 try:
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	179 xml_elem = xmlio.Element('file',
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	180 filename=os.path.basename(filename),
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	181 description=description,
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	182 resource=resource or 'build')
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	183 xml_elem.append(fileobj.read().encode('base64'))
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	184 self.output.append((Recipe.ATTACH, None, None, xml_elem))
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	185
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	186 finally:
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	187 fileobj.close()
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	188 except IOError, e:
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	189 self.error('Failed to read file %s as attachment' % file_)
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	190
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	191 def resolve(self, *path):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	192 """Return the path of a file relative to the base directory.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	193
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	194 Accepts any number of positional arguments, which are joined using the
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	195 system path separator to form the path.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	196 """
68 234600bf0d49 Fixes for Windows compatibility: cmlenz parents: 61 diff changeset	197 return os.path.normpath(os.path.join(self.basedir, *path))
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	198
055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	199
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	200 class Step(object):
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	201 """Represents a single step of a build recipe.
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	202
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	203 Iterate over an object of this class to get the commands to execute, and
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	204 their keyword arguments.
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	205 """
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	206
51 5caccd7b247e Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}. cmlenz parents: 21 diff changeset	207 def __init__(self, elem):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	208 """Create the step.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	209
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	210 :param elem: the XML element representing the step
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	211 :type elem: `ParsedElement`
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	212 """
51 5caccd7b247e Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}. cmlenz parents: 21 diff changeset	213 self._elem = elem
55 649626cabc23 Adjust recipe module to changes to xmlio in [56]. cmlenz parents: 51 diff changeset	214 self.id = elem.attr['id']
649626cabc23 Adjust recipe module to changes to xmlio in [56]. cmlenz parents: 51 diff changeset	215 self.description = elem.attr.get('description')
72 b2d371dac270 Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed. cmlenz parents: 68 diff changeset	216 self.onerror = elem.attr.get('onerror', 'fail')
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	217
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	218 def __repr__(self):
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	219 return '<%s %r>' % (type(self).__name__, self.id)
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	220
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	221 def execute(self, ctxt):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	222 """Execute this step in the given context.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	223
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	224 :param ctxt: the build context
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	225 :type ctxt: `Context`
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	226 """
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	227 last_finish = time.time()
51 5caccd7b247e Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}. cmlenz parents: 21 diff changeset	228 for child in self._elem:
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	229 try:
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	230 ctxt.run(self, child.namespace, child.name, child.attr)
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	231 except (BuildError, InvalidRecipeError), e:
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	232 ctxt.error(e)
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	233 if time.time() < last_finish + 1:
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	234 # Add a delay to make sure steps appear in correct order
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	235 time.sleep(1)
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	236
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	237 errors = []
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	238 while ctxt.output:
203 e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 163 diff changeset	239 type, category, generator, output = ctxt.output.pop(0)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 163 diff changeset	240 yield type, category, generator, output
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	241 if type == Recipe.ERROR:
203 e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 163 diff changeset	242 errors.append((generator, output))
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	243 if errors:
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	244 for _t, error in errors:
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	245 log.error(error)
387 7cd9a26134f5 Default behavior of slaves is now to stop building when an error is raised by a step set to onerror='fail'. onerror='continue' can be used to get the old behavior, while onerror='ignore' is also still available for completely ignoring failed steps. cmlenz parents: 379 diff changeset	246 if self.onerror != 'ignore':
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	247 raise BuildError("Build step '%s' failed" % self.id)
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	248 log.warning("Continuing despite errors in step '%s'", self.id)
72 b2d371dac270 Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed. cmlenz parents: 68 diff changeset	249
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	250
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	251 class Recipe(object):
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	252 """A build recipe.
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	253
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	254 Iterate over this object to get the individual build steps in the order
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	255 they have been defined in the recipe file.
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	256 """
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	257
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	258 ERROR = 'error'
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	259 LOG = 'log'
5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	260 REPORT = 'report'
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	261 ATTACH = 'attach'
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	262
233 8f816147620f * Moved SlaveConfiguration logic into new module ([source:/trunk/bitten/build/config.py bitten.build.config]). cmlenz parents: 213 diff changeset	263 def __init__(self, xml, basedir=os.getcwd(), config=None):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	264 """Create the recipe.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	265
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	266 :param xml: the XML document representing the recipe
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	267 :type xml: `ParsedElement`
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	268 :param basedir: the base directory for the build
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	269 :param config: the slave configuration (optional)
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	270 :type config: `Configuration`
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	271 """
148 f3f5895e373c Fixes to problems in recipe handling introduced in [155]. cmlenz parents: 147 diff changeset	272 assert isinstance(xml, xmlio.ParsedElement)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	273 vars = dict([(name, value) for name, value in xml.attr.items()
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	274 if not name.startswith('xmlns')])
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	275 self.ctxt = Context(basedir, config, vars)
148 f3f5895e373c Fixes to problems in recipe handling introduced in [155]. cmlenz parents: 147 diff changeset	276 self._root = xml
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	277
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	278 def __iter__(self):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	279 """Iterate over the individual steps of the recipe."""
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	280 for child in self._root.children('step'):
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	281 yield Step(child)
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	282
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	283 def validate(self):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	284 """Validate the recipe.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	285
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	286 This method checks a number of constraints:
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	287 - the name of the root element must be "build"
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	288 - the only permitted child elements or the root element with the name
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	289 "step"
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	290 - the recipe must contain at least one step
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	291 - step elements must have a unique "id" attribute
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	292 - a step must contain at least one nested command
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	293 - commands must not have nested content
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	294
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	295 :raise InvalidRecipeError: in case any of the above contraints is
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	296 violated
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	297 """
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	298 if self._root.name != 'build':
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	299 raise InvalidRecipeError('Root element must be <build>')
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	300 steps = list(self._root.children())
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	301 if not steps:
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	302 raise InvalidRecipeError('Recipe defines no build steps')
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	303
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	304 step_ids = set()
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	305 for step in steps:
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	306 if step.name != 'step':
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	307 raise InvalidRecipeError('Only <step> elements allowed at '
2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	308 'top level of recipe')
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	309 if not step.attr.get('id'):
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	310 raise InvalidRecipeError('Steps must have an "id" attribute')
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	311
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	312 if step.attr['id'] in step_ids:
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	313 raise InvalidRecipeError('Duplicate step ID "%s"' %
2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	314 step.attr['id'])
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	315 step_ids.add(step.attr['id'])
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	316
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	317 cmds = list(step.children())
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	318 if not cmds:
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	319 raise InvalidRecipeError('Step "%s" has no recipe commands' %
2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	320 step.attr['id'])
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	321 for cmd in cmds:
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	322 if len(list(cmd.children())):
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	323 raise InvalidRecipeError('Recipe command <%s> has nested '
2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	324 'content' % cmd.name)

Mercurial > bitten > bitten-test

annotate bitten/recipe.py @ 630:042c8b49ce7f