bitten/bitten-test: bitten/slave.py annotate

annotate bitten/slave.py @ 49:b4c51e32952b

Build slave now also handles snapshot archives in bzip2 and zip format.

author	cmlenz
date	Fri, 24 Jun 2005 17:26:47 +0000
parents	757aa3bf9594
children	5caccd7b247e

rev	line source
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	1 # -- coding: iso8859-1 --
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	2 #
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	3 # Copyright (C) 2005 Christopher Lenz <cmlenz@gmx.de>
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	4 #
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	5 # Bitten is free software; you can redistribute it and/or
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	6 # modify it under the terms of the GNU General Public License as
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	7 # published by the Free Software Foundation; either version 2 of the
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	8 # License, or (at your option) any later version.
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	9 #
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	10 # Trac is distributed in the hope that it will be useful,
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	11 # but WITHOUT ANY WARRANTY; without even the implied warranty of
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	12 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	13 # General Public License for more details.
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	14 #
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	15 # You should have received a copy of the GNU General Public License
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	16 # along with this program; if not, write to the Free Software
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	17 # Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	18 #
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	19 # Author: Christopher Lenz <cmlenz@gmx.de>
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	20
15 06207499c58c * Use logging in the BEEP core as well as in the master and slave scripts. Closes #4. cmlenz parents: 14 diff changeset	21 import logging
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	22 import os
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	23 import sys
42 efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	24 import tempfile
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	25 import time
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	26
19 9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	27 from bitten import __version__ as VERSION
48 757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	28 from bitten.util import archive, beep, xmlio
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	29
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	30
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	31 class Slave(beep.Initiator):
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	32
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	33 def greeting_received(self, profiles):
19 9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	34 if OrchestrationProfileHandler.URI not in profiles:
47 083e848088ee * Improvements to the model classes, and a couple of unit tests. cmlenz parents: 43 diff changeset	35 err = 'Peer does not support the Bitten orchestration profile'
083e848088ee * Improvements to the model classes, and a couple of unit tests. cmlenz parents: 43 diff changeset	36 logging.error(err)
083e848088ee * Improvements to the model classes, and a couple of unit tests. cmlenz parents: 43 diff changeset	37 raise beep.TerminateSession, err
29 2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	38 self.channels[0].profile.send_start([OrchestrationProfileHandler])
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	39
14 1733c601d2f8 Refactored the asyncore loop and shutdown procedure into {{{beep.Initiator}}}. cmlenz parents: 13 diff changeset	40
19 9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	41 class OrchestrationProfileHandler(beep.ProfileHandler):
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	42 """Handler for communication on the Bitten build orchestration profile from
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	43 the perspective of the build slave.
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	44 """
19 9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	45 URI = 'http://bitten.cmlenz.net/beep/orchestration'
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	46
29 2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	47 def handle_connect(self, init_elem=None):
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	48 """Register with the build master."""
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	49 def handle_reply(cmd, msgno, msg):
15 06207499c58c * Use logging in the BEEP core as well as in the master and slave scripts. Closes #4. cmlenz parents: 14 diff changeset	50 if cmd == 'ERR':
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	51 if msg.get_content_type() == beep.BEEP_XML:
29 2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	52 elem = xmlio.parse(msg.get_payload())
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	53 if elem.tagname == 'error':
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	54 raise beep.TerminateSession, \
29 2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	55 '%s (%d)' % (elem.gettext(), int(elem.code))
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	56 raise beep.TerminateSession, 'Registration failed!'
15 06207499c58c * Use logging in the BEEP core as well as in the master and slave scripts. Closes #4. cmlenz parents: 14 diff changeset	57 logging.info('Registration successful')
29 2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	58
2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	59 sysname, nodename, release, version, machine = os.uname()
2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	60 logging.info('Registering with build master as %s', nodename)
2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	61 xml = xmlio.Element('register', name=nodename)[
2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	62 xmlio.Element('platform')[machine],
2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	63 xmlio.Element('os', family=os.name, version=release)[sysname]
2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	64 ]
2adc3480e4aa Some cleanup and additional docstrings. cmlenz parents: 19 diff changeset	65 self.channel.send_msg(beep.MIMEMessage(xml), handle_reply)
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	66
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	67 def handle_msg(self, msgno, msg):
49 b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	68 content_type = msg.get_content_type()
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	69 if content_type in ('application/tar', 'application/zip'):
42 efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	70 workdir = tempfile.mkdtemp(prefix='bitten')
49 b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	71
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	72 archive_name = msg.get('Content-Disposition')
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	73 if not archive_name:
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	74 if content_type == 'application/tar':
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	75 encoding = msg.get('Content-Transfer-Encoding')
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	76 if encoding == 'gzip':
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	77 archive_name = 'snapshot.tar.gz'
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	78 elif encoding == 'bzip2':
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	79 archive_name = 'snapshot.tar.bz2'
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	80 else:
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	81 archive_name = 'snapshot.tar'
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	82 else:
b4c51e32952b Build slave now also handles snapshot archives in bzip2 and zip format. cmlenz parents: 48 diff changeset	83 archive_name = 'snapshot.zip'
42 efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	84 archive_path = os.path.join(workdir, archive_name)
efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	85 file(archive_path, 'wb').write(msg.get_payload())
48 757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	86 logging.debug('Received snapshot archive: %s', archive_path)
42 efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	87
48 757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	88 # Unpack the archive
757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	89 prefix = archive.unpack(archive_path, workdir)
757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	90 path = os.path.join(workdir, prefix)
757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	91 logging.info('Unpacked snapshot to %s' % path)
757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	92
757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	93 # Fix permissions
757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	94 for root, dirs, files in os.walk(workdir, topdown=False):
757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	95 for dirname in dirs:
757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	96 os.chmod(os.path.join(root, dirname), 0700)
757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	97 for filename in files:
757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	98 os.chmod(os.path.join(root, filename), 0400)
42 efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	99
efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	100 xml = xmlio.Element('ok')
efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	101 self.channel.send_rpy(msgno, beep.MIMEMessage(xml))
efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	102
48 757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	103 # TODO: Start the build process
757aa3bf9594 * Simplify code for making snapshot archives. cmlenz parents: 47 diff changeset	104
42 efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	105 else:
efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	106 xml = xmlio.Element('error', code=500)['Sorry, what?']
efa525876b1e Basic infrastructure for transmission of snapshot archives to build slaves. See #8. cmlenz parents: 34 diff changeset	107 self.channel.send_err(msgno, beep.MIMEMessage(xml))
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	108
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	109
31 0b82d012e42c Add thin script shells around master and slave. cmlenz parents: 29 diff changeset	110 def main():
19 9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	111 from optparse import OptionParser
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	112
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	113 parser = OptionParser(usage='usage: %prog [options] host [port]',
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	114 version='%%prog %s' % VERSION)
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	115 parser.add_option('--debug', action='store_const', dest='loglevel',
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	116 const=logging.DEBUG, help='enable debugging output')
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	117 parser.add_option('-v', '--verbose', action='store_const', dest='loglevel',
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	118 const=logging.INFO, help='print as much as possible')
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	119 parser.add_option('-q', '--quiet', action='store_const', dest='loglevel',
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	120 const=logging.ERROR, help='print as little as possible')
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	121 parser.set_defaults(loglevel=logging.WARNING)
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	122 options, args = parser.parse_args()
9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	123
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	124 if len(args) < 1:
19 9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	125 parser.error('incorrect number of arguments')
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	126 host = args[0]
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	127 if len(args) > 1:
18 591a5a836ecc * {{{beep.Listener}}} now has an event loop (based on code mostly from medusa) cmlenz parents: 15 diff changeset	128 try:
591a5a836ecc * {{{beep.Listener}}} now has an event loop (based on code mostly from medusa) cmlenz parents: 15 diff changeset	129 port = int(args[1])
19 9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	130 assert (1 <= port <= 65535), 'port number out of range'
47 083e848088ee * Improvements to the model classes, and a couple of unit tests. cmlenz parents: 43 diff changeset	131 except (AssertionError, ValueError):
19 9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	132 parser.error('port must be an integer in the range 1-65535')
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	133 else:
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	134 port = 7633
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	135
19 9db5f8eddb0d Proper {{{optparse}}}-based command-line interface for master and slave. cmlenz parents: 18 diff changeset	136 logging.getLogger().setLevel(options.loglevel)
13 21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	137
21aa17f97522 Initial code for build master and slave... these don't do a lot yet. cmlenz parents: diff changeset	138 slave = Slave(host, port)
34 6da9468a6879 The build master now gracefully exits by first terminating all active sessions. Fixes #7. cmlenz parents: 33 diff changeset	139 try:
6da9468a6879 The build master now gracefully exits by first terminating all active sessions. Fixes #7. cmlenz parents: 33 diff changeset	140 slave.run()
6da9468a6879 The build master now gracefully exits by first terminating all active sessions. Fixes #7. cmlenz parents: 33 diff changeset	141 except KeyboardInterrupt:
6da9468a6879 The build master now gracefully exits by first terminating all active sessions. Fixes #7. cmlenz parents: 33 diff changeset	142 slave.quit()
31 0b82d012e42c Add thin script shells around master and slave. cmlenz parents: 29 diff changeset	143
0b82d012e42c Add thin script shells around master and slave. cmlenz parents: 29 diff changeset	144 if __name__ == '__main__':
33 d8d44216258a Exit the slave script when the master disconnects; and other minor fixes. cmlenz parents: 31 diff changeset	145 main()

Mercurial > bitten > bitten-test

annotate bitten/slave.py @ 49:b4c51e32952b