bitten/bitten-test: bitten/master.py annotate

annotate bitten/master.py @ 836:8df9bc903edb 0.6.x

0.6dev: Merged [912:913] from trunk.

author	osimons
date	Sun, 10 Oct 2010 20:20:37 +0000
parents	f4d07544722b
children	4b6ca8cfd8c6

rev	line source
379 0df178e07fdb Use UTF-8 as encoding of source files. cmlenz parents: 377 diff changeset	1 # -- coding: utf-8 --
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	2 #
833 f4d07544722b 0.6dev: Merged [910] from trunk. osimons parents: 827 diff changeset	3 # Copyright (C) 2007-2010 Edgewall Software
408 933105ab516b Update file headers and other stuff pointing to the old home. cmlenz parents: 402 diff changeset	4 # Copyright (C) 2005-2007 Christopher Lenz <cmlenz@gmx.de>
163 634be6cbb808 Flip the switch: Bitten is now BSD-licensed. cmlenz parents: 157 diff changeset	5 # All rights reserved.
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	6 #
163 634be6cbb808 Flip the switch: Bitten is now BSD-licensed. cmlenz parents: 157 diff changeset	7 # This software is licensed as described in the file COPYING, which
634be6cbb808 Flip the switch: Bitten is now BSD-licensed. cmlenz parents: 157 diff changeset	8 # you should have received as part of this distribution. The terms
408 933105ab516b Update file headers and other stuff pointing to the old home. cmlenz parents: 402 diff changeset	9 # are also available at http://bitten.edgewall.org/wiki/License.
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	10
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	11 """Build master implementation."""
313 90422699a594 More and improved docstrings (using epydoc format). cmlenz parents: 312 diff changeset	12
312 1016c3d12cbc Fix UTC timestamp handling in build master. cmlenz parents: 310 diff changeset	13 import calendar
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	14 import re
47 083e848088ee * Improvements to the model classes, and a couple of unit tests. cmlenz parents: 45 diff changeset	15 import time
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 611 diff changeset	16 from StringIO import StringIO
18 591a5a836ecc * {{{beep.Listener}}} now has an event loop (based on code mostly from medusa) cmlenz parents: 15 diff changeset	17
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 611 diff changeset	18 from trac.attachment import Attachment
577 dcee8ff20e82 0.6dev: The `logs_dir` option is not a real `PathOption` as for that it would need to support config-relative path with possibly chained configs following trac:ticket:8358 (for 0.11.5). The fix just makes it a regular `Option` to keep current behaviour for all Trac 0.11+ versions. osimons parents: 568 diff changeset	19 from trac.config import BoolOption, IntOption, Option
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	20 from trac.core import *
629 f3bb52da9e3c 0.6dev: Adding support for attachments to configurations and build - full web implementation that mirrors what is available in Ticket and Wiki. Also added a new generic `<attach/>` command that enables attaching files to be part of a recipe and uploaded by slaves as part of build. osimons parents: 611 diff changeset	21 from trac.resource import ResourceNotFound
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	22 from trac.web import IRequestHandler, RequestDone
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	23
654 cc1fefd5d814 0.6dev: Moved a constant declaration so that master should still work with Python 2.4. Updated docs with regards to Python requirements + added a changelog message from cmlenz. osimons parents: 649 diff changeset	24 from bitten import PROTOCOL_VERSION
554 2c27f3581100 Supply the target platform as a variable `platform` for build recipes, and adjust test accordingly - fixes #376 dfraser parents: 542 diff changeset	25 from bitten.model import BuildConfig, Build, BuildStep, BuildLog, Report, \
2c27f3581100 Supply the target platform as a variable `platform` for build recipes, and adjust test accordingly - fixes #376 dfraser parents: 542 diff changeset	26 TargetPlatform
2c27f3581100 Supply the target platform as a variable `platform` for build recipes, and adjust test accordingly - fixes #376 dfraser parents: 542 diff changeset	27
410 7930cdd83d13 More restructuring: got rid of the `trac_ext` subpackage, which makes no sense now that the master is also coupled to Trac. cmlenz parents: 408 diff changeset	28 from bitten.main import BuildSystem
227 014bc6c29dff * Factor build queue logic into a class separate from the build master. cmlenz parents: 213 diff changeset	29 from bitten.queue import BuildQueue
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	30 from bitten.recipe import Recipe
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	31 from bitten.util import xmlio
83 42970c14524a Perform slave/platform matching at slave registration. Use builtin {{{set}}} type on Python >= 2.4. cmlenz parents: 82 diff changeset	32
411 a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 410 diff changeset	33 __all__ = ['BuildMaster']
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 410 diff changeset	34 __docformat__ = 'restructuredtext en'
a169d2e96463 Use reStructuredText as the API documentation syntax. cmlenz parents: 410 diff changeset	35
56 033366d81def Build slave now executes the build. Closes #10. cmlenz parents: 51 diff changeset	36
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	37 HTTP_BAD_REQUEST = 400
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	38 HTTP_FORBIDDEN = 403
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	39 HTTP_NOT_FOUND = 404
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	40 HTTP_METHOD_NOT_ALLOWED = 405
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	41 HTTP_CONFLICT = 409
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	42
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	43
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	44 class BuildMaster(Component):
542 cacf018eb190 Small documentation fixes. wbell parents: 516 diff changeset	45 """Trac request handler implementation for the build master."""
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	46
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	47 implements(IRequestHandler)
51 5caccd7b247e Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}. cmlenz parents: 49 diff changeset	48
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	49 # Configuration options
51 5caccd7b247e Proper archive format negotiation; improved representation of parsed XML content in {{{bitten.util.xmlio}}}. cmlenz parents: 49 diff changeset	50
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	51 adjust_timestamps = BoolOption('bitten', 'adjust_timestamps', False, doc=
432 74c51f648466 Started some tests for the new admin interface. cmlenz parents: 426 diff changeset	52 """Whether the timestamps of builds should be adjusted to be close
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	53 to the timestamps of the corresponding changesets.""")
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	54
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	55 build_all = BoolOption('bitten', 'build_all', False, doc=
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	56 """Whether to request builds of older revisions even if a younger
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	57 revision has already been built.""")
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	58
468 44c2b4ac6157 Add stabilization time parameter to build master. Closes #189. Many thanks to Allen Bierbaum for the patch. cmlenz parents: 466 diff changeset	59 stabilize_wait = IntOption('bitten', 'stabilize_wait', 0, doc=
44c2b4ac6157 Add stabilization time parameter to build master. Closes #189. Many thanks to Allen Bierbaum for the patch. cmlenz parents: 466 diff changeset	60 """The time in seconds to wait for the repository to stabilize before
44c2b4ac6157 Add stabilization time parameter to build master. Closes #189. Many thanks to Allen Bierbaum for the patch. cmlenz parents: 466 diff changeset	61 queuing up a new build. This allows time for developers to check in
44c2b4ac6157 Add stabilization time parameter to build master. Closes #189. Many thanks to Allen Bierbaum for the patch. cmlenz parents: 466 diff changeset	62 a group of related changes back to back without spawning multiple
44c2b4ac6157 Add stabilization time parameter to build master. Closes #189. Many thanks to Allen Bierbaum for the patch. cmlenz parents: 466 diff changeset	63 builds.""")
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	64
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	65 slave_timeout = IntOption('bitten', 'slave_timeout', 3600, doc=
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	66 """The time in seconds after which a build is cancelled if the slave
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	67 does not report progress.""")
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	68
577 dcee8ff20e82 0.6dev: The `logs_dir` option is not a real `PathOption` as for that it would need to support config-relative path with possibly chained configs following trac:ticket:8358 (for 0.11.5). The fix just makes it a regular `Option` to keep current behaviour for all Trac 0.11+ versions. osimons parents: 568 diff changeset	69 logs_dir = Option('bitten', 'logs_dir', "log/bitten", doc=
516 2f3b7c17d3c3 Switch to storing log messages in files rather than in database rows: dfraser parents: 494 diff changeset	70 """The directory on the server in which client log files will be stored.""")
2f3b7c17d3c3 Switch to storing log messages in files rather than in database rows: dfraser parents: 494 diff changeset	71
557 b4d3d9cbf200 Alter the appearance of the ''Build Status'' button, to show the current build status [eblot] - fixes #373 dfraser parents: 554 diff changeset	72 quick_status = BoolOption('bitten', 'quick_status', False, doc=
807 d87227fdb9c2 0.6dev: Merged [884] from trunk osimons parents: 805 diff changeset	73 """Whether to show the current build status within the Trac main
d87227fdb9c2 0.6dev: Merged [884] from trunk osimons parents: 805 diff changeset	74 navigation bar. '''Note:''' The feature requires expensive database and
d87227fdb9c2 0.6dev: Merged [884] from trunk osimons parents: 805 diff changeset	75 repository checks for every page request, and should not be enabled
d87227fdb9c2 0.6dev: Merged [884] from trunk osimons parents: 805 diff changeset	76 if the project has a large repository or uses a non-Subversion
d87227fdb9c2 0.6dev: Merged [884] from trunk osimons parents: 805 diff changeset	77 repository such as Mercurial or Git.""")
557 b4d3d9cbf200 Alter the appearance of the ''Build Status'' button, to show the current build status [eblot] - fixes #373 dfraser parents: 554 diff changeset	78
568 84a45f7c9833 0.6dev: Adding Bitten version info to About Trac. osimons parents: 557 diff changeset	79 def __init__(self):
84a45f7c9833 0.6dev: Adding Bitten version info to About Trac. osimons parents: 557 diff changeset	80 self.env.systeminfo.append(('Bitten',
84a45f7c9833 0.6dev: Adding Bitten version info to About Trac. osimons parents: 557 diff changeset	81 __import__('bitten', ['__version__']).__version__))
84a45f7c9833 0.6dev: Adding Bitten version info to About Trac. osimons parents: 557 diff changeset	82
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	83 # IRequestHandler methods
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	84
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	85 def match_request(self, req):
401 a10942252ebc Use POST for submitting step results, instead of PUT. cmlenz parents: 392 diff changeset	86 match = re.match(r'/builds(?:/(\d+)(?:/(\w+)/([^/]+)?)?)?$',
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	87 req.path_info)
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	88 if match:
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	89 if match.group(1):
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	90 req.args['id'] = match.group(1)
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	91 req.args['collection'] = match.group(2)
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	92 req.args['member'] = match.group(3)
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	93 return True
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	94
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	95 def process_request(self, req):
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	96 req.perm.assert_permission('BUILD_EXEC')
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	97
649 eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	98 if 'trac_auth' in req.incookie:
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	99 slave_token = req.incookie['trac_auth'].value
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	100 else:
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	101 slave_token = req.session.sid
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	102
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	103 if 'id' not in req.args:
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	104 if req.method != 'POST':
716 cc620fb0ee60 0.6dev: Merging [792:793] from trunk. osimons parents: 703 diff changeset	105 self._send_response(req,
cc620fb0ee60 0.6dev: Merging [792:793] from trunk. osimons parents: 703 diff changeset	106 body='Only POST allowed for build creation.')
649 eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	107 return self._process_build_creation(req, slave_token)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	108
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	109 build = Build.fetch(self.env, req.args['id'])
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	110 if not build:
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	111 self._send_error(req, HTTP_NOT_FOUND,
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	112 'No such build (%s)' % req.args['id'])
649 eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	113
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	114 build_token = build.slave_info.get('token', '')
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	115 if build_token != slave_token:
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	116 self._send_error(req, HTTP_CONFLICT,
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	117 'Token mismatch (wrong slave): slave=%s, build=%s' \
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	118 % (slave_token, build_token))
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	119
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	120 config = BuildConfig.fetch(self.env, build.config)
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	121
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	122 if not req.args['collection']:
420 23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	123 if req.method == 'DELETE':
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	124 return self._process_build_cancellation(req, config, build)
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	125 else:
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	126 return self._process_build_initiation(req, config, build)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	127
401 a10942252ebc Use POST for submitting step results, instead of PUT. cmlenz parents: 392 diff changeset	128 if req.method != 'POST':
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	129 self._send_error(req, HTTP_METHOD_NOT_ALLOWED,
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	130 'Method %s not allowed' % req.method)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	131
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	132 if req.args['collection'] == 'steps':
401 a10942252ebc Use POST for submitting step results, instead of PUT. cmlenz parents: 392 diff changeset	133 return self._process_build_step(req, config, build)
836 8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	134 elif req.args['collection'] == 'attach':
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	135 return self._process_attachment(req, config, build)
786 38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	136 elif req.args['collection'] == 'keepalive':
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	137 return self._process_keepalive(req, config, build)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	138 else:
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	139 self._send_error(req, HTTP_NOT_FOUND,
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	140 "No such collection '%s'" % req.args['collection'])
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	141
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	142 # Internal methods
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	143
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	144 def _send_response(self, req, code=200, body='', headers=None):
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	145 """ Formats and sends the response, raising ``RequestDone``. """
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	146 req.send_response(code)
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	147 headers = headers or {}
703 4e27920699b2 0.6dev: Merging [778:780] from trunk. osimons parents: 654 diff changeset	148 headers.setdefault('Content-Length', len(body))
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	149 for header in headers:
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	150 req.send_header(header, headers[header])
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	151 req.write(body)
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	152 raise RequestDone
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	153
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	154 def _send_error(self, req, code=500, message=''):
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	155 """ Formats and sends the error, raising ``RequestDone``. """
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	156 headers = {'Content-Type': 'text/plain',
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	157 'Content-Length': str(len(message))}
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	158 self._send_response(req, code, body=message, headers=headers)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	159
649 eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	160 def _process_build_creation(self, req, slave_token):
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	161 queue = BuildQueue(self.env, build_all=self.build_all,
468 44c2b4ac6157 Add stabilization time parameter to build master. Closes #189. Many thanks to Allen Bierbaum for the patch. cmlenz parents: 466 diff changeset	162 stabilize_wait=self.stabilize_wait,
419 b72802dc0632 Fix resetting of builds when multiple slaves are building simultaneously, and implement the `slave_timeout` trac.ini option. cmlenz parents: 411 diff changeset	163 timeout=self.slave_timeout)
805 4a1e2d555626 0.6dev: Merged [882] from trunk. Basic Trac 0.12 support. osimons parents: 786 diff changeset	164 try:
4a1e2d555626 0.6dev: Merged [882] from trunk. Basic Trac 0.12 support. osimons parents: 786 diff changeset	165 queue.populate()
4a1e2d555626 0.6dev: Merged [882] from trunk. Basic Trac 0.12 support. osimons parents: 786 diff changeset	166 except AssertionError, e:
4a1e2d555626 0.6dev: Merged [882] from trunk. Basic Trac 0.12 support. osimons parents: 786 diff changeset	167 self.log.error(e.message, exc_info=True)
4a1e2d555626 0.6dev: Merged [882] from trunk. Basic Trac 0.12 support. osimons parents: 786 diff changeset	168 self._send_error(req, HTTP_BAD_REQUEST, e.message)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	169
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	170 try:
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	171 elem = xmlio.parse(req.read())
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	172 except xmlio.ParseError, e:
492 56f0ad35c60a Log errors when build master encounters an XML parse error. cmlenz parents: 479 diff changeset	173 self.log.error('Error parsing build initialization request: %s', e,
56f0ad35c60a Log errors when build master encounters an XML parse error. cmlenz parents: 479 diff changeset	174 exc_info=True)
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	175 self._send_error(req, HTTP_BAD_REQUEST, 'XML parser error')
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	176
649 eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	177 slave_version = int(elem.attr.get('version', 1))
786 38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	178
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	179 # FIXME: Remove version compatibility code.
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	180 # The initial difference between protocol version 3 and 4 is that
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	181 # the master allows keepalive requests-- the master must be
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	182 # at least 4 before slaves supporting version 4 are allowed. When
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	183 # the first force master/slave upgrade requirement comes in
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	184 # (or we bump the) version number again, remove this code.
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	185 if slave_version == 3 and PROTOCOL_VERSION == 4:
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	186 self.log.info('Allowing slave version %d to process build for '
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	187 'compatibility. Upgrade slave to support build '
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	188 'keepalives.', slave_version)
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	189 elif slave_version != PROTOCOL_VERSION:
649 eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	190 self._send_error(req, HTTP_BAD_REQUEST,
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	191 "Master-Slave version mismatch: master=%d, slave=%d" % \
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	192 (PROTOCOL_VERSION, slave_version))
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	193
426 a08fa5ce9aff Applied patch by Sam Steele for #167. cmlenz parents: 420 diff changeset	194 slavename = elem.attr['name']
649 eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	195 properties = {'name': slavename, Build.IP_ADDRESS: req.remote_addr,
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	196 Build.TOKEN: slave_token}
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	197 self.log.info('Build slave %r connected from %s with token %s',
eed0149c302a 0.6dev: Switching to use the new cookie-support, and using trac auth/session ID as unique identification for linking builds with slaves. This overcomes problems with IP address not being unique behind NAT, and also where IP address may change during a build. Closes #421. osimons parents: 645 diff changeset	198 slavename, req.remote_addr, slave_token)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	199
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	200 for child in elem.children():
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	201 if child.name == 'platform':
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	202 properties[Build.MACHINE] = child.gettext()
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	203 properties[Build.PROCESSOR] = child.attr.get('processor')
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	204 elif child.name == 'os':
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	205 properties[Build.OS_NAME] = child.gettext()
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	206 properties[Build.OS_FAMILY] = child.attr.get('family')
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	207 properties[Build.OS_VERSION] = child.attr.get('version')
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	208 elif child.name == 'package':
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	209 for name, value in child.attr.items():
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	210 if name == 'name':
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	211 continue
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	212 properties[child.attr['name'] + '.' + name] = value
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	213
444 22d6a7da8777 Another minor improvement to logging in the build master. cmlenz parents: 436 diff changeset	214 self.log.debug('Build slave configuration: %r', properties)
22d6a7da8777 Another minor improvement to logging in the build master. cmlenz parents: 436 diff changeset	215
426 a08fa5ce9aff Applied patch by Sam Steele for #167. cmlenz parents: 420 diff changeset	216 build = queue.get_build_for_slave(slavename, properties)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	217 if not build:
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	218 self._send_response(req, 204, '', {})
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	219
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	220 self._send_response(req, 201, 'Build pending', headers={
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	221 'Content-Type': 'text/plain',
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	222 'Location': req.abs_href.builds(build.id)})
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	223
420 23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	224 def _process_build_cancellation(self, req, config, build):
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	225 self.log.info('Build slave %r cancelled build %d', build.slave,
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	226 build.id)
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	227 build.status = Build.PENDING
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	228 build.slave = None
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	229 build.slave_info = {}
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	230 build.started = 0
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	231 db = self.env.get_db_cnx()
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	232 for step in list(BuildStep.select(self.env, build=build.id, db=db)):
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	233 step.delete(db=db)
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	234 build.update(db=db)
751 db5f7388535d Merge [821,822,823,824,825,826] from trunk. Thanks Walter\! hodgestar parents: 716 diff changeset	235
db5f7388535d Merge [821,822,823,824,825,826] from trunk. Thanks Walter\! hodgestar parents: 716 diff changeset	236 Attachment.delete_all(self.env, 'build', build.resource.id, db)
db5f7388535d Merge [821,822,823,824,825,826] from trunk. Thanks Walter\! hodgestar parents: 716 diff changeset	237
420 23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	238 db.commit()
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	239
458 c9ac97df8a5e Fix build listener invocation. cmlenz parents: 444 diff changeset	240 for listener in BuildSystem(self.env).listeners:
c9ac97df8a5e Fix build listener invocation. cmlenz parents: 444 diff changeset	241 listener.build_aborted(build)
c9ac97df8a5e Fix build listener invocation. cmlenz parents: 444 diff changeset	242
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	243 self._send_response(req, 204, '', {})
420 23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	244
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	245 def _process_build_initiation(self, req, config, build):
420 23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	246 self.log.info('Build slave %r initiated build %d', build.slave,
23de253435b8 Slaves now attempt to explicitly cancel builds when they are interrupted. cmlenz parents: 419 diff changeset	247 build.id)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	248 build.started = int(time.time())
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	249 build.last_activity = build.started
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	250 build.update()
277 1141027071b3 Changes to snapshot archive creation/transmission: cmlenz parents: 267 diff changeset	251
458 c9ac97df8a5e Fix build listener invocation. cmlenz parents: 444 diff changeset	252 for listener in BuildSystem(self.env).listeners:
c9ac97df8a5e Fix build listener invocation. cmlenz parents: 444 diff changeset	253 listener.build_started(build)
c9ac97df8a5e Fix build listener invocation. cmlenz parents: 444 diff changeset	254
287 6abd43d0cd8a The build slave now stores snapshot archives and the corresponding work directories in project folders of the main work folder, to keep build configurations from different projects that share the same name separate. This also requires transmitting the project name (simply the name of the environment directory) with the build initiation. cmlenz parents: 284 diff changeset	255 xml = xmlio.parse(config.recipe)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	256 xml.attr['path'] = config.path
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	257 xml.attr['revision'] = build.rev
466 79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 463 diff changeset	258 xml.attr['config'] = config.name
79be3c00ae69 Applied patch to #188 for stable/configurable names of build directories. Thanks to Allen Bierbaum for the patch. cmlenz parents: 463 diff changeset	259 xml.attr['build'] = str(build.id)
554 2c27f3581100 Supply the target platform as a variable `platform` for build recipes, and adjust test accordingly - fixes #376 dfraser parents: 542 diff changeset	260 target_platform = TargetPlatform.fetch(self.env, build.platform)
2c27f3581100 Supply the target platform as a variable `platform` for build recipes, and adjust test accordingly - fixes #376 dfraser parents: 542 diff changeset	261 xml.attr['platform'] = target_platform.name
611 294641e84e89 0.6dev: Adding `${name}` and `${basedir}` (#325) for recipe substitution. Updated docs + new test. osimons parents: 577 diff changeset	262 xml.attr['name'] = build.slave
836 8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	263 xml.attr['form_token'] = req.form_token # For posting attachments
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	264 body = str(xml)
374 446092a2d2fe Don't accept build results from a slave if the build has been invalidated or it's being built by another slave. Closes #100. wbell parents: 373 diff changeset	265
473 b1d346c4e539 Only accept build results from the slave that's supposed to be processing this build. Additional logging. wbell parents: 468 diff changeset	266 self.log.info('Build slave %r initiated build %d', build.slave,
b1d346c4e539 Only accept build results from the slave that's supposed to be processing this build. Additional logging. wbell parents: 468 diff changeset	267 build.id)
b1d346c4e539 Only accept build results from the slave that's supposed to be processing this build. Additional logging. wbell parents: 468 diff changeset	268
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	269 # create the first step, mark it as in-progress.
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	270
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	271 recipe = Recipe(xmlio.parse(config.recipe))
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	272 stepname = recipe.__iter__().next().id
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	273
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	274 step = self._start_new_step(build, stepname)
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	275 step.insert()
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	276
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	277 self._send_response(req, 200, body, headers={
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	278 'Content-Type': 'application/x-bitten+xml',
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	279 'Content-Length': str(len(body)),
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	280 'Content-Disposition':
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	281 'attachment; filename=recipe_%s_r%s.xml' %
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	282 (config.name, build.rev)})
227 014bc6c29dff * Factor build queue logic into a class separate from the build master. cmlenz parents: 213 diff changeset	283
401 a10942252ebc Use POST for submitting step results, instead of PUT. cmlenz parents: 392 diff changeset	284 def _process_build_step(self, req, config, build):
a10942252ebc Use POST for submitting step results, instead of PUT. cmlenz parents: 392 diff changeset	285 try:
a10942252ebc Use POST for submitting step results, instead of PUT. cmlenz parents: 392 diff changeset	286 elem = xmlio.parse(req.read())
a10942252ebc Use POST for submitting step results, instead of PUT. cmlenz parents: 392 diff changeset	287 except xmlio.ParseError, e:
492 56f0ad35c60a Log errors when build master encounters an XML parse error. cmlenz parents: 479 diff changeset	288 self.log.error('Error parsing build step result: %s', e,
56f0ad35c60a Log errors when build master encounters an XML parse error. cmlenz parents: 479 diff changeset	289 exc_info=True)
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	290 self._send_error(req, HTTP_BAD_REQUEST, 'XML parser error')
401 a10942252ebc Use POST for submitting step results, instead of PUT. cmlenz parents: 392 diff changeset	291 stepname = elem.attr['step']
632 01c9848950d5 0.6dev: Cleaning all usage of tabs (replaced with spaces). osimons parents: 629 diff changeset	292
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	293 # we should have created this step previously; if it hasn't,
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	294 # the master and slave are processing steps out of order.
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	295 step = BuildStep.fetch(self.env, build=build.id, name=stepname)
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	296 if not step:
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	297 self._send_error(req, HTTP_CONFLICT, 'Build step has not been created.')
253 cda723f3ac31 Provide hooks for build notification. Closes #62. cmlenz parents: 245 diff changeset	298
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	299 recipe = Recipe(xmlio.parse(config.recipe))
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	300 index = None
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	301 current_step = None
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	302 for num, recipe_step in enumerate(recipe):
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	303 if recipe_step.id == stepname:
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	304 index = num
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	305 current_step = recipe_step
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	306 if index is None:
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	307 self._send_error(req, HTTP_FORBIDDEN,
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	308 'No such build step' % stepname)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	309 last_step = index == num
200 692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47. cmlenz parents: 197 diff changeset	310
494 9c9bf7a69bf0 Apply patch by Emmanuel Blot to allow a single slave to connect to multiple masters. Closes #271. cmlenz parents: 492 diff changeset	311 self.log.debug('Slave %s (build %d) completed step %d (%s) with '
9c9bf7a69bf0 Apply patch by Emmanuel Blot to allow a single slave to connect to multiple masters. Closes #271. cmlenz parents: 492 diff changeset	312 'status %s', build.slave, build.id, index, stepname,
9c9bf7a69bf0 Apply patch by Emmanuel Blot to allow a single slave to connect to multiple masters. Closes #271. cmlenz parents: 492 diff changeset	313 elem.attr['status'])
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	314
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	315 db = self.env.get_db_cnx()
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	316
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	317 step.stopped = int(time.time())
759 bfd5fd75d1d6 Port of [836] to 0.6.x wbell parents: 755 diff changeset	318
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	319 if elem.attr['status'] == 'failure':
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	320 self.log.warning('Build %s step %s failed', build.id, stepname)
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 96 diff changeset	321 step.status = BuildStep.FAILURE
459 6537f01b4730 Clarify `last_step` logic in master. cmlenz parents: 458 diff changeset	322 if current_step.onerror == 'fail':
6537f01b4730 Clarify `last_step` logic in master. cmlenz parents: 458 diff changeset	323 last_step = True
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 96 diff changeset	324 else:
5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 96 diff changeset	325 step.status = BuildStep.SUCCESS
277 1141027071b3 Changes to snapshot archive creation/transmission: cmlenz parents: 267 diff changeset	326 step.errors += [error.gettext() for error in elem.children('error')]
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	327
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	328 # TODO: step.update(db=db)
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	329 step.delete(db=db)
112 a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream. cmlenz parents: 110 diff changeset	330 step.insert(db=db)
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 96 diff changeset	331
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	332 # Collect log messages from the request body
203 e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 202 diff changeset	333 for idx, log_elem in enumerate(elem.children('log')):
401 a10942252ebc Use POST for submitting step results, instead of PUT. cmlenz parents: 392 diff changeset	334 build_log = BuildLog(self.env, build=build.id, step=stepname,
203 e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 202 diff changeset	335 generator=log_elem.attr.get('generator'),
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 202 diff changeset	336 orderno=idx)
115 16d69eb6e047 Add support for XML fragments to the {{{xmlio}}} module, so that build output and reports don't need to be nested in a meaningless element (such as {{{<log type="distutils"><messages><message ...>}}}). cmlenz parents: 112 diff changeset	337 for message_elem in log_elem.children('message'):
16d69eb6e047 Add support for XML fragments to the {{{xmlio}}} module, so that build output and reports don't need to be nested in a meaningless element (such as {{{<log type="distutils"><messages><message ...>}}}). cmlenz parents: 112 diff changeset	338 build_log.messages.append((message_elem.attr['level'],
16d69eb6e047 Add support for XML fragments to the {{{xmlio}}} module, so that build output and reports don't need to be nested in a meaningless element (such as {{{<log type="distutils"><messages><message ...>}}}). cmlenz parents: 112 diff changeset	339 message_elem.gettext()))
112 a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream. cmlenz parents: 110 diff changeset	340 build_log.insert(db=db)
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 96 diff changeset	341
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	342 # Collect report data from the request body
203 e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 202 diff changeset	343 for report_elem in elem.children('report'):
401 a10942252ebc Use POST for submitting step results, instead of PUT. cmlenz parents: 392 diff changeset	344 report = Report(self.env, build=build.id, step=stepname,
213 25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 204 diff changeset	345 category=report_elem.attr.get('category'),
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly: cmlenz parents: 204 diff changeset	346 generator=report_elem.attr.get('generator'))
203 e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 202 diff changeset	347 for item_elem in report_elem.children():
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 202 diff changeset	348 item = {'type': item_elem.name}
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 202 diff changeset	349 item.update(item_elem.attr)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 202 diff changeset	350 for child_elem in item_elem.children():
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 202 diff changeset	351 item[child_elem.name] = child_elem.gettext()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 202 diff changeset	352 report.items.append(item)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL). cmlenz parents: 202 diff changeset	353 report.insert(db=db)
116 86439c2aa6d6 Store report data in BDB XML database. Closes #31. cmlenz parents: 115 diff changeset	354
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	355 # If this was the last step in the recipe we mark the build as
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	356 # completed otherwise just update last_activity
459 6537f01b4730 Clarify `last_step` logic in master. cmlenz parents: 458 diff changeset	357 if last_step:
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	358 self.log.info('Slave %s completed build %d ("%s" as of [%s])',
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	359 build.slave, build.id, build.config, build.rev)
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	360 build.stopped = step.stopped
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	361 build.last_activity = build.stopped
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	362
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	363 # Determine overall outcome of the build by checking the outcome
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	364 # of the individual steps against the "onerror" specification of
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	365 # each step in the recipe
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	366 for num, recipe_step in enumerate(recipe):
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	367 step = BuildStep.fetch(self.env, build.id, recipe_step.id)
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	368 if step.status == BuildStep.FAILURE:
755 ab91df14f670 Merge of [832] from trunk. wbell parents: 751 diff changeset	369 if recipe_step.onerror == 'fail' or \
ab91df14f670 Merge of [832] from trunk. wbell parents: 751 diff changeset	370 recipe_step.onerror == 'continue':
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	371 build.status = Build.FAILURE
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	372 break
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	373 else:
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	374 build.status = Build.SUCCESS
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	375
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	376 build.update(db=db)
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	377 else:
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	378 build.last_activity = step.stopped
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	379 build.update(db=db)
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	380
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	381 # start the next step.
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	382 for num, recipe_step in enumerate(recipe):
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	383 if num == index + 1:
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	384 next_step = recipe_step
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	385 if next_step is None:
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	386 self._send_error(req, HTTP_FORBIDDEN,
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	387 'Unable to find step after ' % stepname)
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	388
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	389 step = self._start_new_step(build, next_step.id)
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	390 step.insert(db=db)
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	391
200 692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47. cmlenz parents: 197 diff changeset	392 db.commit()
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47. cmlenz parents: 197 diff changeset	393
459 6537f01b4730 Clarify `last_step` logic in master. cmlenz parents: 458 diff changeset	394 if last_step:
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	395 for listener in BuildSystem(self.env).listeners:
026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	396 listener.build_completed(build)
253 cda723f3ac31 Provide hooks for build notification. Closes #62. cmlenz parents: 245 diff changeset	397
392 026d9aa41b85 Merged HTTP branch into trunk. cmlenz parents: 379 diff changeset	398 body = 'Build step processed'
645 8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	399 self._send_response(req, 201, body, {
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	400 'Content-Type': 'text/plain',
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	401 'Content-Length': str(len(body)),
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	402 'Location': req.abs_href.builds(
8c824b14e1c5 0.6dev: Switching `master.py` to use new `self._send_response()` and `self._send_error() methods. Simplifies code, but most importantly for errors it allows a consistent method for transmitting plain-text error messages to the slave (that the slave will now output as part of debug logging). osimons parents: 632 diff changeset	403 build.id, 'steps', stepname)})
109 5bf22bb87915 Transmit build log and generated data back to the build master in XML format. Closes #23. cmlenz parents: 96 diff changeset	404
836 8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	405 def _process_attachment(self, req, config, build):
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	406 resource_id = req.args['member'] == 'config' \
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	407 and build.config or build.resource.id
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	408 upload = req.args['file']
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	409 if not upload.file:
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	410 send_error(req, message="Attachment not received.")
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	411 self.log.debug('Received attachment %s for attaching to build:%s',
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	412 upload.filename, resource_id)
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	413
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	414 # Determine size of file
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	415 upload.file.seek(0, 2) # to the end
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	416 size = upload.file.tell()
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	417 upload.file.seek(0) # beginning again
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	418
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	419 # Delete attachment if it already exists
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	420 try:
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	421 old_attach = Attachment(self.env, 'build',
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	422 parent_id=resource_id, filename=upload.filename)
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	423 old_attach.delete()
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	424 except ResourceNotFound:
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	425 pass
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	426
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	427 # Save new attachment
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	428 attachment = Attachment(self.env, 'build', parent_id=resource_id)
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	429 attachment.description = req.args.get('description', '')
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	430 attachment.author = req.authname
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	431 attachment.insert(upload.filename, upload.file, size)
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	432
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	433 self._send_response(req, 201, 'Attachment created', headers={
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	434 'Content-Type': 'text/plain',
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	435 'Content-Length': str(len('Attachment created'))})
8df9bc903edb 0.6dev: Merged [912:913] from trunk. osimons parents: 833 diff changeset	436
786 38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	437 def _process_keepalive(self, req, config, build):
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	438 build.last_activity = int(time.time())
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	439 build.update()
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	440
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	441 self.log.info('Slave %s build %d keepalive ("%s" as of [%s])',
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	442 build.slave, build.id, build.config, build.rev)
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	443
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	444 body = 'Keepalive processed'
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	445 self._send_response(req, 200, body, {
38e11e15d95e Port of [864] to 0.6.x wbell parents: 763 diff changeset	446 'Content-Type': 'text/plain',
827 46f37b649a23 0.6dev: Merged [903] from trunk. wbell parents: 807 diff changeset	447 'Content-Length': str(len(body))})
46f37b649a23 0.6dev: Merged [903] from trunk. wbell parents: 807 diff changeset	448
763 de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	449 def _start_new_step(self, build, stepname):
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	450 """Creates the in-memory representation for a newly started
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	451 step, ready to be persisted to the database.
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	452 """
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	453 step = BuildStep(self.env, build=build.id, name=stepname)
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	454 step.status = BuildStep.IN_PROGRESS
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	455 step.started = int(time.time())
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	456 step.stopped = 0
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	457
de466e590545 Port of [638], [639], [640] to 0.6.x wbell parents: 759 diff changeset	458 return step

Mercurial > bitten > bitten-test

annotate bitten/master.py @ 836:8df9bc903edb 0.6.x