bitten/bitten-test: bitten/recipe.py annotate

annotate bitten/recipe.py @ 895:7d93d6358fe0

Use our own HTTPBasicAuthHandler under Python 2.6 to avoid issue http://bugs.python.org/issue8797. Fixes #658.

author	hodgestar
date	Wed, 09 Mar 2011 14:48:35 +0000
parents	59acaa8b52c0
children

rev	line source
379 0df178e07fdb Use UTF-8 as encoding of source files. cmlenz parents: 377 diff changeset	1 # -- coding: utf-8 --
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	2 #
832 7c80375d4817 Updated copyright to 2010. osimons parents: 754 diff changeset	3 # Copyright (C) 2007-2010 Edgewall Software
408 933105ab516b Update file headers and other stuff pointing to the old home. cmlenz parents: 392 diff changeset	4 # Copyright (C) 2005-2007 Christopher Lenz <cmlenz@gmx.de>
163 634be6cbb808 Flip the switch: Bitten is now BSD-licensed. cmlenz parents: 157 diff changeset	5 # All rights reserved.
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	6 #
163 634be6cbb808 Flip the switch: Bitten is now BSD-licensed. cmlenz parents: 157 diff changeset	7 # This software is licensed as described in the file COPYING, which
634be6cbb808 Flip the switch: Bitten is now BSD-licensed. cmlenz parents: 157 diff changeset	8 # you should have received as part of this distribution. The terms
408 933105ab516b Update file headers and other stuff pointing to the old home. cmlenz parents: 392 diff changeset	9 # are also available at http://bitten.edgewall.org/wiki/License.
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	10
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	11 """Execution of build recipes.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	12
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	13 This module provides various classes that can be used to process build recipes,
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	14 most importantly the `Recipe` class.
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	15 """
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	16
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	17 import inspect
146 affd91b4c6fb Add a `<python:exec>` recipe command so that things like Pylint can be executed without using a Makefile. cmlenz parents: 144 diff changeset	18 import keyword
72 b2d371dac270 Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed. cmlenz parents: 68 diff changeset	19 import logging
129 efd3df0de93a Canonicalize path names so that comparison works. Fixes #37. cmlenz parents: 109 diff changeset	20 import os
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	21 import time
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	22 try:
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	23 set
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	24 except NameError:
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	25 from sets import Set as set
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	26
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	27 from pkg_resources import WorkingSet
752 673ec182679d Allow ''timeout'' parameter to be set on sh:exec, python:exec and wbell parents: 677 diff changeset	28 from bitten.build import BuildError, TimeoutError
238 832e64330c31 Add a `<c:configure>` recipe command for running configure scripts. Closes #57. cmlenz parents: 233 diff changeset	29 from bitten.build.config import Configuration
51 5caccd7b247e Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}. cmlenz parents: 21 diff changeset	30 from bitten.util import xmlio
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	31
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	32 __all__ = ['Context', 'Recipe', 'Step', 'InvalidRecipeError']
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	33 __docformat__ = 'restructuredtext en'
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	34
93 b289e572bc7e Improved logging; the build master can now optionally log to a file. Closes #13. cmlenz parents: 80 diff changeset	35 log = logging.getLogger('bitten.recipe')
b289e572bc7e Improved logging; the build master can now optionally log to a file. Closes #13. cmlenz parents: 80 diff changeset	36
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	37
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	38 class InvalidRecipeError(Exception):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	39 """Exception raised when a recipe is not valid."""
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	40
055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	41
055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	42 class Context(object):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	43 """The context in which a build is executed."""
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	44
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	45 step = None # The current step
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	46 generator = None # The current generator (namespace#name)
80 dc1c7fc9b915 Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed. cmlenz parents: 72 diff changeset	47
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	48 def __init__(self, basedir, config=None, vars=None):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	49 """Initialize the context.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	50
466 79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 411 diff changeset	51 :param basedir: a string containing the working directory for the build.
79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 411 diff changeset	52 (may be a pattern for replacement ex: 'build_${build}'
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	53 :param config: the build slave configuration
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	54 :type config: `Configuration`
466 79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 411 diff changeset	55 """
238 832e64330c31 Add a `<c:configure>` recipe command for running configure scripts. Closes #57. cmlenz parents: 233 diff changeset	56 self.config = config or Configuration()
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	57 self.vars = vars or {}
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	58 self.output = []
466 79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 411 diff changeset	59 self.basedir = os.path.realpath(self.config.interpolate(basedir,
79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 411 diff changeset	60 **self.vars))
660 3a7a1ca990ff 0.6dev: Follow-up to [685] where backslashes of `${basedir}` (ie. Windows) would disappear without double escaping. osimons parents: 659 diff changeset	61 self.vars['basedir'] = self.basedir.replace('\\', '\\\\')
80 dc1c7fc9b915 Record the output of build steps in the database. See #12. Still need to get better granularity in transmitting the log output from slave to master before #12 can be closed. cmlenz parents: 72 diff changeset	62
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	63 def run(self, step, namespace, name, attr):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	64 """Run the specified recipe command.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	65
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	66 :param step: the build step that the command belongs to
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	67 :param namespace: the namespace URI of the command
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	68 :param name: the local tag name of the command
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	69 :param attr: a dictionary containing the attributes defined on the
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	70 command element
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	71 """
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	72 self.step = step
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	73
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	74 try:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	75 function = None
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	76 qname = '#'.join(filter(None, [namespace, name]))
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	77 if namespace:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	78 group = 'bitten.recipe_commands'
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	79 for entry_point in WorkingSet().iter_entry_points(group, qname):
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	80 function = entry_point.load()
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	81 break
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	82 elif name == 'report':
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	83 function = Context.report_file
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	84 elif name == 'attach':
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	85 function = Context.attach
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	86 if not function:
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	87 raise InvalidRecipeError('Unknown recipe command %s' % qname)
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	88
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	89 def escape(name):
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	90 name = name.replace('-', '_')
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	91 if keyword.iskeyword(name) or name in __builtins__:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	92 name = name + '_'
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	93 return name
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	94 args = dict([(escape(name),
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	95 self.config.interpolate(attr[name], **self.vars))
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	96 for name in attr])
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	97 function_args, has_kwargs = inspect.getargspec(function)[0:3:2]
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	98 for arg in args:
630 042c8b49ce7f 0.6dev: Follow-up to [702] - flawed logic in argument testing. osimons parents: 629 diff changeset	99 if not (arg in function_args or has_kwargs):
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	100 raise InvalidRecipeError(
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	101 "Unsupported argument '%s' for command %s" % \
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	102 (arg, qname))
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	103
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	104 self.generator = qname
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	105 log.debug('Executing %s with arguments: %s', function, args)
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	106 function(self, **args)
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	107
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	108 finally:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	109 self.generator = None
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	110 self.step = None
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	111
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	112 def error(self, message):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	113 """Record an error message.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	114
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	115 :param message: a string containing the error message.
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	116 """
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	117 self.output.append((Recipe.ERROR, None, self.generator, message))
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	118
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	119 def log(self, xml):
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	120 """Record log output.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	121
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	122 :param xml: an XML fragment containing the log messages
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	123 """
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	124 self.output.append((Recipe.LOG, None, self.generator, xml))
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	125
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	126 def report(self, category, xml):
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	127 """Record report data.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	128
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	129 :param category: the name of category of the report
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	130 :param xml: an XML fragment containing the report data
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	131 """
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	132 self.output.append((Recipe.REPORT, category, self.generator, xml))
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	133
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	134 def report_file(self, category=None, file_=None):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	135 """Read report data from a file and record it.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	136
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	137 :param category: the name of the category of the report
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	138 :param file\_: the path to the file containing the report data, relative
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	139 to the base directory
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	140 """
314 a8fd83c0317d Fix undefined variable in recipe error reporting. Closes #78. cmlenz parents: 313 diff changeset	141 filename = self.resolve(file_)
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	142 try:
314 a8fd83c0317d Fix undefined variable in recipe error reporting. Closes #78. cmlenz parents: 313 diff changeset	143 fileobj = file(filename, 'r')
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	144 try:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	145 xml_elem = xmlio.Fragment()
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	146 for child in xmlio.parse(fileobj).children():
377 a465b5dbfecf Tabs and line length. cmlenz parents: 370 diff changeset	147 child_elem = xmlio.Element(child.name, **dict([
a465b5dbfecf Tabs and line length. cmlenz parents: 370 diff changeset	148 (name, value) for name, value in child.attr.items()
a465b5dbfecf Tabs and line length. cmlenz parents: 370 diff changeset	149 if value is not None
a465b5dbfecf Tabs and line length. cmlenz parents: 370 diff changeset	150 ]))
370 fa5400c2879d Fix for report imports; I would have thought that the original child.attr would work since it's a DictMixin, but at runtime, we're seeing 'argument after ** must be a dictionary' wbell parents: 368 diff changeset	151 xml_elem.append(child_elem[
265 3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	152 [xmlio.Element(grandchild.name)[grandchild.gettext()]
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	153 for grandchild in child.children()]
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	154 ])
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	155 self.output.append((Recipe.REPORT, category, None, xml_elem))
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	156 finally:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	157 fileobj.close()
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	158 except xmlio.ParseError, e:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	159 self.error('Failed to parse %s report at %s: %s'
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	160 % (category, filename, e))
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	161 except IOError, e:
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	162 self.error('Failed to read %s report at %s: %s'
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	163 % (category, filename, e))
3dd311b4ad27 Add a generic `<report>` recipe command, which can be used to send any XML file to the master as a categorized report. This can be used in combination with `<sh:pipe>` or `<x:transform>` to produce reports not specifically generated by a specialized recipe command. cmlenz parents: 247 diff changeset	164
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	165 def attach(self, file_=None, description=None, resource=None):
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	166 """Attach a file to the build or build configuration.
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	167
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	168 :param file\_: the path to the file to attach, relative to
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	169 base directory.
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	170 :param description: description saved with attachment
677 639e5c466c96 0.6dev: Documentation fixes for all `build_doc` errors and warnings. osimons parents: 660 diff changeset	171 :param resource: which resource to attach the file to,
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	172 either 'build' (default) or 'config'
f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	173 """
835 59acaa8b52c0 Slave attachment support via `<attach />` is totally redone to use multi-part form post instead of inlining it in the XML (ie. like a web file upload form). For larger binaries the previous inlining would effectively be an internal denial-of-service attack... osimons parents: 832 diff changeset	174 # Attachments are not added as inline xml, so only adding
59acaa8b52c0 Slave attachment support via `<attach />` is totally redone to use multi-part form post instead of inlining it in the XML (ie. like a web file upload form). For larger binaries the previous inlining would effectively be an internal denial-of-service attack... osimons parents: 832 diff changeset	175 # the details for later processing.
59acaa8b52c0 Slave attachment support via `<attach />` is totally redone to use multi-part form post instead of inlining it in the XML (ie. like a web file upload form). For larger binaries the previous inlining would effectively be an internal denial-of-service attack... osimons parents: 832 diff changeset	176 if not file_:
59acaa8b52c0 Slave attachment support via `<attach />` is totally redone to use multi-part form post instead of inlining it in the XML (ie. like a web file upload form). For larger binaries the previous inlining would effectively be an internal denial-of-service attack... osimons parents: 832 diff changeset	177 self.error('No attachment file specified.')
59acaa8b52c0 Slave attachment support via `<attach />` is totally redone to use multi-part form post instead of inlining it in the XML (ie. like a web file upload form). For larger binaries the previous inlining would effectively be an internal denial-of-service attack... osimons parents: 832 diff changeset	178 return
59acaa8b52c0 Slave attachment support via `<attach />` is totally redone to use multi-part form post instead of inlining it in the XML (ie. like a web file upload form). For larger binaries the previous inlining would effectively be an internal denial-of-service attack... osimons parents: 832 diff changeset	179 xml_elem = xmlio.Element('file', filename=file_,
59acaa8b52c0 Slave attachment support via `<attach />` is totally redone to use multi-part form post instead of inlining it in the XML (ie. like a web file upload form). For larger binaries the previous inlining would effectively be an internal denial-of-service attack... osimons parents: 832 diff changeset	180 description=description or '',
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	181 resource=resource or 'build')
835 59acaa8b52c0 Slave attachment support via `<attach />` is totally redone to use multi-part form post instead of inlining it in the XML (ie. like a web file upload form). For larger binaries the previous inlining would effectively be an internal denial-of-service attack... osimons parents: 832 diff changeset	182 self.output.append((Recipe.ATTACH, None, None, xml_elem))
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	183
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	184 def resolve(self, *path):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	185 """Return the path of a file relative to the base directory.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	186
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	187 Accepts any number of positional arguments, which are joined using the
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	188 system path separator to form the path.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	189 """
68 234600bf0d49 Fixes for Windows compatibility: cmlenz parents: 61 diff changeset	190 return os.path.normpath(os.path.join(self.basedir, *path))
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	191
055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	192
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	193 class Step(object):
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	194 """Represents a single step of a build recipe.
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	195
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	196 Iterate over an object of this class to get the commands to execute, and
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	197 their keyword arguments.
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	198 """
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	199
754 545be0c8f405 Adding the ability to modify the default ''onerror'' property in the ''<build>'' element. If not specified, the behavior is unchanged; by default any step failure will result in the build failing and stopping. wbell parents: 752 diff changeset	200 def __init__(self, elem, onerror_default):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	201 """Create the step.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	202
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	203 :param elem: the XML element representing the step
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	204 :type elem: `ParsedElement`
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	205 """
51 5caccd7b247e Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}. cmlenz parents: 21 diff changeset	206 self._elem = elem
55 649626cabc23 Adjust recipe module to changes to xmlio in [56]. cmlenz parents: 51 diff changeset	207 self.id = elem.attr['id']
649626cabc23 Adjust recipe module to changes to xmlio in [56]. cmlenz parents: 51 diff changeset	208 self.description = elem.attr.get('description')
754 545be0c8f405 Adding the ability to modify the default ''onerror'' property in the ''<build>'' element. If not specified, the behavior is unchanged; by default any step failure will result in the build failing and stopping. wbell parents: 752 diff changeset	209 self.onerror = elem.attr.get('onerror', onerror_default)
545be0c8f405 Adding the ability to modify the default ''onerror'' property in the ''<build>'' element. If not specified, the behavior is unchanged; by default any step failure will result in the build failing and stopping. wbell parents: 752 diff changeset	210 assert self.onerror in ('fail', 'ignore', 'continue')
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	211
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	212 def __repr__(self):
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	213 return '<%s %r>' % (type(self).__name__, self.id)
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	214
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 203 diff changeset	215 def execute(self, ctxt):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	216 """Execute this step in the given context.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	217
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	218 :param ctxt: the build context
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	219 :type ctxt: `Context`
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	220 """
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	221 last_finish = time.time()
51 5caccd7b247e Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}. cmlenz parents: 21 diff changeset	222 for child in self._elem:
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	223 try:
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	224 ctxt.run(self, child.namespace, child.name, child.attr)
752 673ec182679d Allow ''timeout'' parameter to be set on sh:exec, python:exec and wbell parents: 677 diff changeset	225 except (BuildError, InvalidRecipeError, TimeoutError), e:
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	226 ctxt.error(e)
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	227 if time.time() < last_finish + 1:
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	228 # Add a delay to make sure steps appear in correct order
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	229 time.sleep(1)
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	230
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	231 errors = []
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	232 while ctxt.output:
203 e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 163 diff changeset	233 type, category, generator, output = ctxt.output.pop(0)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 163 diff changeset	234 yield type, category, generator, output
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	235 if type == Recipe.ERROR:
203 e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 163 diff changeset	236 errors.append((generator, output))
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	237 if errors:
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	238 for _t, error in errors:
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	239 log.error(error)
387 7cd9a26134f5 Default behavior of slaves is now to stop building when an error is raised by a step set to onerror='fail'. onerror='continue' can be used to get the old behavior, while onerror='ignore' is also still available for completely ignoring failed steps. cmlenz parents: 379 diff changeset	240 if self.onerror != 'ignore':
628 05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	241 raise BuildError("Build step '%s' failed" % self.id)
05686657e989 0.6dev: Implement a `subprocess` implementation of `execute()` that is now the default for all slaves where this is available - essentially all slaves with Python 2.4 and higher, or where module is installed separately. This fixes: osimons parents: 611 diff changeset	242 log.warning("Continuing despite errors in step '%s'", self.id)
72 b2d371dac270 Allow individual steps of a recipe to be marked as optional, i.e. that an error in such a step should not mean that the build failed. cmlenz parents: 68 diff changeset	243
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	244
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	245 class Recipe(object):
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	246 """A build recipe.
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	247
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	248 Iterate over this object to get the individual build steps in the order
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	249 they have been defined in the recipe file.
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	250 """
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	251
131 3ed8f568f60a Fix error handling so that reports are still generated even if a command has failed. cmlenz parents: 129 diff changeset	252 ERROR = 'error'
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	253 LOG = 'log'
5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	254 REPORT = 'report'
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 628 diff changeset	255 ATTACH = 'attach'
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 93 diff changeset	256
233 8f816147620f * Moved SlaveConfiguration logic into new module ([source:/trunk/bitten/build/config.py bitten.build.config]). cmlenz parents: 213 diff changeset	257 def __init__(self, xml, basedir=os.getcwd(), config=None):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	258 """Create the recipe.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	259
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	260 :param xml: the XML document representing the recipe
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	261 :type xml: `ParsedElement`
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	262 :param basedir: the base directory for the build
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	263 :param config: the slave configuration (optional)
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	264 :type config: `Configuration`
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	265 """
148 f3f5895e373c Fixes to problems in recipe handling introduced in [155]. cmlenz parents: 147 diff changeset	266 assert isinstance(xml, xmlio.ParsedElement)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	267 vars = dict([(name, value) for name, value in xml.attr.items()
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	268 if not name.startswith('xmlns')])
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 387 diff changeset	269 self.ctxt = Context(basedir, config, vars)
148 f3f5895e373c Fixes to problems in recipe handling introduced in [155]. cmlenz parents: 147 diff changeset	270 self._root = xml
754 545be0c8f405 Adding the ability to modify the default ''onerror'' property in the ''<build>'' element. If not specified, the behavior is unchanged; by default any step failure will result in the build failing and stopping. wbell parents: 752 diff changeset	271 self.onerror_default = vars.get('onerror', 'fail')
545be0c8f405 Adding the ability to modify the default ''onerror'' property in the ''<build>'' element. If not specified, the behavior is unchanged; by default any step failure will result in the build failing and stopping. wbell parents: 752 diff changeset	272 assert self.onerror_default in ('fail', 'ignore', 'continue')
21 07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	273
07053ecfb124 Cleanup package namespace a bit. cmlenz parents: diff changeset	274 def __iter__(self):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	275 """Iterate over the individual steps of the recipe."""
60 055a6c666fa8 * Pass a {{{Context}}} object to recipe commands as the first argument. Currently this only has the basedir, but will be extended to also provide output recording etc. cmlenz parents: 55 diff changeset	276 for child in self._root.children('step'):
754 545be0c8f405 Adding the ability to modify the default ''onerror'' property in the ''<build>'' element. If not specified, the behavior is unchanged; by default any step failure will result in the build failing and stopping. wbell parents: 752 diff changeset	277 yield Step(child, self.onerror_default)
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	278
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	279 def validate(self):
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	280 """Validate the recipe.
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	281
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	282 This method checks a number of constraints:
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	283 - the name of the root element must be "build"
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	284 - the only permitted child elements or the root element with the name
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	285 "step"
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	286 - the recipe must contain at least one step
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	287 - step elements must have a unique "id" attribute
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	288 - a step must contain at least one nested command
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	289 - commands must not have nested content
90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	290
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	291 :raise InvalidRecipeError: in case any of the above contraints is
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 408 diff changeset	292 violated
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 295 diff changeset	293 """
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	294 if self._root.name != 'build':
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	295 raise InvalidRecipeError('Root element must be <build>')
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	296 steps = list(self._root.children())
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	297 if not steps:
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	298 raise InvalidRecipeError('Recipe defines no build steps')
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	299
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	300 step_ids = set()
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	301 for step in steps:
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	302 if step.name != 'step':
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	303 raise InvalidRecipeError('Only <step> elements allowed at '
2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	304 'top level of recipe')
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	305 if not step.attr.get('id'):
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	306 raise InvalidRecipeError('Steps must have an "id" attribute')
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	307
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	308 if step.attr['id'] in step_ids:
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	309 raise InvalidRecipeError('Duplicate step ID "%s"' %
2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	310 step.attr['id'])
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	311 step_ids.add(step.attr['id'])
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	312
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	313 cmds = list(step.children())
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	314 if not cmds:
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	315 raise InvalidRecipeError('Step "%s" has no recipe commands' %
2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	316 step.attr['id'])
247 b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	317 for cmd in cmds:
b28285d3ceec Add validation for build configurations, and in particular for build recipes. Closes #48. cmlenz parents: 238 diff changeset	318 if len(list(cmd.children())):
368 2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	319 raise InvalidRecipeError('Recipe command <%s> has nested '
2d58319eea84 Use modern syntax for raising exceptions. cmlenz parents: 314 diff changeset	320 'content' % cmd.name)

Mercurial > bitten > bitten-test

annotate bitten/recipe.py @ 895:7d93d6358fe0