Mercurial > bitten > bitten-test

annotate bitten/tests/upgrades.py @ 860:ac28bc3feb2a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Refuse to create log folder during database upgrade if the folder already exists. The existence of such a folder is an indication that a previous attempt to migrate the logs into files has failed and that the folder should be removed before continuing. See #462 for further discussion.
author hodgestar
date Sun, 17 Oct 2010 23:20:44 +0000
parents ee95915922dd
children
rev   line source
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
1 # -*- coding: utf-8 -*-
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
2 #
832
7c80375d4817 Updated copyright to 2010.
osimons
parents: 790
diff changeset
3 # Copyright (C) 2009-2010 Edgewall Software
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
4 # All rights reserved.
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
5 #
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
6 # This software is licensed as described in the file COPYING, which
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
7 # you should have received as part of this distribution. The terms
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
8 # are also available at http://bitten.edgewall.org/wiki/License.
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
9
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
10 import unittest
735
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
11 import logging
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
12
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
13 import warnings
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
14 warnings.filterwarnings('ignore', '^Unknown table')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
15 warnings.filterwarnings('ignore', '^the sets module is deprecated')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
16
860
ac28bc3feb2a Refuse to create log folder during database upgrade if the folder already exists. The existence of such a folder is an indication that a previous attempt to migrate the logs into files has failed and that the folder should be removed before continuing. See #462 for further discussion.
hodgestar
parents: 846
diff changeset
17 from trac.core import TracError
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
18 from trac.test import EnvironmentStub
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
19 from trac.db import Table, Column, Index, DatabaseManager
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
20 from bitten.upgrades import update_sequence, drop_index
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
21 from bitten import upgrades, main, model
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
22
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
23 import os
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
24 import shutil
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
25 import tempfile
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
26
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
27
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
28 class BaseUpgradeTestCase(unittest.TestCase):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
29
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
30 schema = None
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
31 other_tables = []
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
32
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
33 def setUp(self):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
34 self.env = EnvironmentStub()
710
c599830cf185 Only patch test configuration database uri if the version of Trac has support for different test databases.
hodgestar
parents: 708
diff changeset
35 if hasattr(self.env, 'dburi'):
c599830cf185 Only patch test configuration database uri if the version of Trac has support for different test databases.
hodgestar
parents: 708
diff changeset
36 # Trac gained support for testing against different databases in 0.11.5
c599830cf185 Only patch test configuration database uri if the version of Trac has support for different test databases.
hodgestar
parents: 708
diff changeset
37 # If this support is available, we copy the test db uri configuration
c599830cf185 Only patch test configuration database uri if the version of Trac has support for different test databases.
hodgestar
parents: 708
diff changeset
38 # into the main test config so it can be picked up by
846
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
39 # upgrades.parse_scheme()
710
c599830cf185 Only patch test configuration database uri if the version of Trac has support for different test databases.
hodgestar
parents: 708
diff changeset
40 self.env.config.set('trac', 'database', self.env.dburi)
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
41 self.env.path = tempfile.mkdtemp()
735
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
42 logs_dir = self.env.config.get("bitten", "logs_dir", "log/bitten")
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
43 if os.path.isabs(logs_dir):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
44 raise ValueError("Should not have absolute logs directory for temporary test")
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
45 logs_dir = os.path.join(self.env.path, logs_dir)
735
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
46 self.logs_dir = logs_dir
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
47
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
48 db = self.env.get_db_cnx()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
49 cursor = db.cursor()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
50
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
51 for table_name in self.other_tables:
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
52 cursor.execute("DROP TABLE IF EXISTS %s" % (table_name,))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
53
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
54 connector, _ = DatabaseManager(self.env)._get_connector()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
55 for table in self.schema:
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
56 cursor.execute("DROP TABLE IF EXISTS %s" % (table.name,))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
57 for stmt in connector.to_sql(table):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
58 cursor.execute(stmt)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
59
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
60 db.commit()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
61
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
62 def tearDown(self):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
63 shutil.rmtree(self.env.path)
735
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
64 del self.logs_dir
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
65 del self.env
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
66
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
67
735
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
68 class LogWatcher(logging.Handler):
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
69
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
70 def __init__(self, level=0):
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
71 logging.Handler.__init__(self, level=0)
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
72 self.records = []
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
73
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
74 def emit(self, record):
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
75 self.records.append(record)
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
76
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
77
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
78 class UpgradeHelperTestCase(BaseUpgradeTestCase):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
79
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
80 schema = [
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
81 Table('test_update_sequence', key='id')[
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
82 Column('id', auto_increment=True), Column('name'),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
83 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
84 Table('test_drop_index', key='id')[
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
85 Column('id', type='int'), Column('name', size=20),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
86 Index(['name'])
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
87 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
88 ]
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
89
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
90 def test_update_sequence(self):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
91 db = self.env.get_db_cnx()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
92 cursor = db.cursor()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
93
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
94 for rowid, name in [(1, 'a'), (2, 'b'), (3, 'c')]:
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
95 cursor.execute("INSERT INTO test_update_sequence (id, name)"
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
96 " VALUES (%s, %s)", (rowid, name))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
97 update_sequence(self.env, db, 'test_update_sequence', 'id')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
98
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
99 cursor.execute("INSERT INTO test_update_sequence (name)"
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
100 " VALUES (%s)", ('d',))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
101
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
102 cursor.execute("SELECT id FROM test_update_sequence WHERE name = %s",
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
103 ('d',))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
104 row = cursor.fetchone()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
105 self.assertEqual(row[0], 4)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
106
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
107 def test_drop_index(self):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
108 db = self.env.get_db_cnx()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
109 cursor = db.cursor()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
110
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
111 cursor.execute("INSERT INTO test_drop_index (id, name)"
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
112 " VALUES (%s, %s)", (1, 'a'))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
113
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
114 def do_drop():
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
115 drop_index(self.env, db, 'test_drop_index', 'test_drop_index_name_idx')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
116
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
117 # dropping the index should succeed the first time and fail the next
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
118 do_drop()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
119 self.assertRaises(Exception, do_drop)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
120
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
121
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
122 class UpgradeScriptsTestCase(BaseUpgradeTestCase):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
123
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
124 schema = [
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
125 # Sytem
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
126 Table('system', key='name')[
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
127 Column('name'), Column('value')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
128 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
129 # Config
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
130 Table('bitten_config', key='name')[
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
131 Column('name'), Column('path'), Column('label'),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
132 Column('active', type='int'), Column('description')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
133 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
134 # Platform
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
135 Table('bitten_platform', key='id')[
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
136 Column('id', auto_increment=True), Column('config'), Column('name')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
137 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
138 Table('bitten_rule', key=('id', 'propname'))[
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
139 Column('id'), Column('propname'), Column('pattern'),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
140 Column('orderno', type='int')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
141 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
142 # Build
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
143 Table('bitten_build', key='id')[
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
144 Column('id', auto_increment=True), Column('config'), Column('rev'),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
145 Column('rev_time', type='int'), Column('platform', type='int'),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
146 Column('slave'), Column('started', type='int'),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
147 Column('stopped', type='int'), Column('status', size=1),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
148 Index(['config', 'rev', 'slave'])
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
149 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
150 Table('bitten_slave', key=('build', 'propname'))[
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
151 Column('build', type='int'), Column('propname'), Column('propvalue')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
152 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
153 # Build Step
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
154 Table('bitten_step', key=('build', 'name'))[
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
155 Column('build', type='int'), Column('name'), Column('description'),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
156 Column('status', size=1), Column('log'),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
157 Column('started', type='int'), Column('stopped', type='int')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
158 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
159 ]
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
160
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
161 other_tables = [
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
162 'bitten_log',
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
163 'bitten_log_message',
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
164 'bitten_report',
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
165 'bitten_report_item',
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
166 'bitten_error',
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
167 'old_step',
846
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
168 'old_config_v2',
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
169 'old_log_v5',
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
170 'old_log_v8',
846
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
171 'old_rule_v9',
762
5f0cfee44540 Add new last_activity field to build. I considered reusing stopped, but this seemed cleaner and more obvious, which seems like the right way to go.
wbell
parents: 737
diff changeset
172 'old_build_v11',
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
173 ]
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
174
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
175 basic_data = [
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
176 ['system',
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
177 ('name', 'value'), [
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
178 ('bitten_version', '1'),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
179 ]
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
180 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
181 ['bitten_config',
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
182 ('name',), [
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
183 ('test_config',),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
184 ]
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
185 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
186 ['bitten_platform',
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
187 ('config', 'name'), [
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
188 ('test_config', 'test_plat'),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
189 ]
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
190 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
191 ['bitten_build',
790
330fa697bc85 Extend upgrade test to check that build ids are preserved.
hodgestar
parents: 762
diff changeset
192 ('id', 'config', 'rev', 'platform', 'rev_time'), [
330fa697bc85 Extend upgrade test to check that build ids are preserved.
hodgestar
parents: 762
diff changeset
193 (12, 'test_config', '123', 1, 456),
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
194 ]
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
195 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
196 ['bitten_step',
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
197 ('build', 'name', 'log'), [
790
330fa697bc85 Extend upgrade test to check that build ids are preserved.
hodgestar
parents: 762
diff changeset
198 (12, 'step1', None),
330fa697bc85 Extend upgrade test to check that build ids are preserved.
hodgestar
parents: 762
diff changeset
199 (12, 'step2', "line1\nline2"),
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
200 ]
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
201 ],
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
202 ]
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
203
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
204 def _do_upgrade(self):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
205 """Do an full upgrade."""
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
206 import inspect
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
207 db = self.env.get_db_cnx()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
208
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
209 versions = sorted(upgrades.map.keys())
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
210 for version in versions:
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
211 for function in upgrades.map.get(version):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
212 self.assertTrue(inspect.getdoc(function))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
213 function(self.env, db)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
214
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
215 db.commit()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
216
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
217 def _insert_data(self, data):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
218 """Insert data for upgrading."""
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
219 db = self.env.get_db_cnx()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
220 cursor = db.cursor()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
221
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
222 for table, cols, vals in data:
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
223 cursor.executemany("INSERT INTO %s (%s) VALUES (%s)"
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
224 % (table, ','.join(cols),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
225 ','.join(['%s' for c in cols])),
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
226 vals)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
227
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
228 db.commit()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
229
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
230 def _check_basic_upgrade(self):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
231 """Check the results of an upgrade of basic data."""
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
232 configs = list(model.BuildConfig.select(self.env,
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
233 include_inactive=True))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
234 platforms = list(model.TargetPlatform.select(self.env))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
235 builds = list(model.Build.select(self.env))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
236 steps = list(model.BuildStep.select(self.env))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
237 logs = list(model.BuildLog.select(self.env))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
238
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
239 self.assertEqual(len(configs), 1)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
240 self.assertEqual(configs[0].name, 'test_config')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
241
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
242 self.assertEqual(len(platforms), 1)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
243 self.assertEqual(platforms[0].config, 'test_config')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
244 self.assertEqual(platforms[0].name, 'test_plat')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
245
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
246 self.assertEqual(len(builds), 1)
790
330fa697bc85 Extend upgrade test to check that build ids are preserved.
hodgestar
parents: 762
diff changeset
247 self.assertEqual(builds[0].id, 12)
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
248 self.assertEqual(builds[0].config, 'test_config')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
249 self.assertEqual(builds[0].rev, '123')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
250 self.assertEqual(builds[0].platform, 1)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
251 self.assertEqual(builds[0].rev_time, 456)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
252
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
253 self.assertEqual(len(steps), 2)
790
330fa697bc85 Extend upgrade test to check that build ids are preserved.
hodgestar
parents: 762
diff changeset
254 self.assertEqual(steps[0].build, 12)
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
255 self.assertEqual(steps[0].name, 'step1')
790
330fa697bc85 Extend upgrade test to check that build ids are preserved.
hodgestar
parents: 762
diff changeset
256 self.assertEqual(steps[1].build, 12)
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
257 self.assertEqual(steps[1].name, 'step2')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
258
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
259 self.assertEqual(len(logs), 1)
790
330fa697bc85 Extend upgrade test to check that build ids are preserved.
hodgestar
parents: 762
diff changeset
260 self.assertEqual(logs[0].build, 12)
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
261 self.assertEqual(logs[0].step, 'step2')
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
262 log_file = logs[0].get_log_file(logs[0].filename)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
263 self.assertEqual(file(log_file, "rU").read(), "line1\nline2\n")
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
264
846
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
265 # check final sequences
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
266 for tbl, col in [
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
267 ('bitten_build', 'id'),
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
268 ('bitten_log', 'id'),
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
269 ('bitten_platform', 'id'),
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
270 ('bitten_report', 'id'),
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
271 ]:
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
272 self._check_sequence(tbl, col)
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
273
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
274 def _check_sequence(self, tbl, col):
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
275 scheme = upgrades.parse_scheme(self.env)
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
276 if scheme == "postgres":
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
277 self._check_postgres_sequence(tbl, col)
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
278
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
279 def _check_postgres_sequence(self, tbl, col):
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
280 """Check a PostgreSQL sequence for the given table and column."""
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
281 seq = '%s_%s_seq' % (tbl, col)
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
282 cursor = self.env.get_db_cnx().cursor()
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
283 cursor.execute("SELECT MAX(%s) FROM %s" % (col, tbl))
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
284 current_max = cursor.fetchone()[0] or 0 # if currently None
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
285 cursor.execute("SELECT nextval('%s')" % (seq,))
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
286 current_seq = cursor.fetchone()[0] - 1
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
287 self.assertEqual(current_max, current_seq,
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
288 "On %s (col: %s) expected column max (%d) "
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
289 "and sequence value (%d) to match"
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
290 % (tbl, col, current_max, current_seq))
ee95915922dd Add checks that (PostgreSQL) sequences have ended up in the correct state at the end of the upgrade scripts. Test currently fails on PostgreSQL as a result of #632.
hodgestar
parents: 832
diff changeset
291
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
292 def test_null_upgrade(self):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
293 self._do_upgrade()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
294
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
295 def test_basic_upgrade(self):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
296 self._insert_data(self.basic_data)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
297 self._do_upgrade()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
298 self._check_basic_upgrade()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
299
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
300 def test_upgrade_via_buildsetup(self):
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
301 self._insert_data(self.basic_data)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
302 db = self.env.get_db_cnx()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
303 build_setup = main.BuildSetup(self.env)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
304 self.assertTrue(build_setup.environment_needs_upgrade(db))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
305 build_setup.upgrade_environment(db)
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
306 self._check_basic_upgrade()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
307
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
308 # check bitten table version
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
309 cursor = db.cursor()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
310 cursor.execute("SELECT value FROM system WHERE name='bitten_version'")
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
311 rows = cursor.fetchall()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
312 self.assertEqual(rows, [(str(model.schema_version),)])
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
313
735
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
314 def test_fix_log_levels_misnaming(self):
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
315 logfiles = {
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
316 "1.log": "",
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
317 "2.log": "",
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
318 "3.log": "",
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
319 "1.log.level": "info\n",
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
320 "2.log.levels": "info\ninfo\n",
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
321 "3.log.level": "warn\n",
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
322 "3.log.levels": "warn\nwarn\n",
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
323 "4.log.level": "error\n",
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
324 }
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
325 expected_deletions = [
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
326 "4.log.level",
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
327 ]
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
328
860
ac28bc3feb2a Refuse to create log folder during database upgrade if the folder already exists. The existence of such a folder is an indication that a previous attempt to migrate the logs into files has failed and that the folder should be removed before continuing. See #462 for further discussion.
hodgestar
parents: 846
diff changeset
329 os.makedirs(self.logs_dir)
735
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
330 for filename, data in logfiles.items():
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
331 path = os.path.join(self.logs_dir, filename)
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
332 logfile = open(path, "w")
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
333 logfile.write(data)
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
334 logfile.close()
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
335
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
336 logwatch = LogWatcher(logging.INFO)
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
337 self.env.log.setLevel(logging.INFO)
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
338 self.env.log.addHandler(logwatch)
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
339
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
340 upgrades.fix_log_levels_misnaming(self.env, None)
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
341
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
342 filenames = sorted(os.listdir(self.logs_dir))
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
343 for filename in filenames:
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
344 path = os.path.join(self.logs_dir, filename)
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
345 origfile = filename in logfiles and filename or filename.replace("levels", "level")
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
346 self.assertEqual(logfiles[origfile], open(path).read())
737
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
347 self.assertTrue(filename not in expected_deletions)
735
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
348
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
349 self.assertEqual(len(filenames), len(logfiles) - len(expected_deletions))
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
350
737
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
351 logs = sorted(logwatch.records, key=lambda rec: rec.getMessage())
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
352 self.assertEqual(len(logs), 5)
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
353 self.assertTrue(logs[0].getMessage().startswith(
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
354 "Deleted 1 stray log level (0 errors)"))
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
355 self.assertTrue(logs[1].getMessage().startswith(
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
356 "Deleted stray log level file 4.log.level"))
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
357 self.assertTrue(logs[2].getMessage().startswith(
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
358 "Error renaming"))
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
359 self.assertTrue(logs[3].getMessage().startswith(
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
360 "Renamed 1 incorrectly named log level files from previous migrate (1 errors)"))
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
361 self.assertTrue(logs[4].getMessage().startswith(
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
362 "Renamed incorrectly named log level file"))
735
90903d6cc932 Add upgrade script to fix badly named .log.level files. Don't use BuildLog.LEVELS_SUFFIX in upgrade scripts. See #517.
hodgestar
parents: 710
diff changeset
363
737
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
364 def test_remove_stray_log_levels_files(self):
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
365 logfiles = {
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
366 "1.log": "",
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
367 "1.log.levels": "info\n",
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
368 "2.log.levels": "info\ninfo\n",
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
369 }
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
370 expected_deletions = [
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
371 "2.log.levels",
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
372 ]
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
373
860
ac28bc3feb2a Refuse to create log folder during database upgrade if the folder already exists. The existence of such a folder is an indication that a previous attempt to migrate the logs into files has failed and that the folder should be removed before continuing. See #462 for further discussion.
hodgestar
parents: 846
diff changeset
374 os.makedirs(self.logs_dir)
737
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
375 for filename, data in logfiles.items():
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
376 path = os.path.join(self.logs_dir, filename)
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
377 logfile = open(path, "w")
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
378 logfile.write(data)
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
379 logfile.close()
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
380
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
381 logwatch = LogWatcher(logging.INFO)
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
382 self.env.log.setLevel(logging.INFO)
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
383 self.env.log.addHandler(logwatch)
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
384
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
385 upgrades.remove_stray_log_levels_files(self.env, None)
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
386
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
387 filenames = sorted(os.listdir(self.logs_dir))
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
388 for filename in filenames:
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
389 path = os.path.join(self.logs_dir, filename)
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
390 self.assertEqual(logfiles[filename], open(path).read())
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
391 self.assertTrue(filename not in expected_deletions)
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
392
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
393 self.assertEqual(len(filenames), len(logfiles) - len(expected_deletions))
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
394
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
395 logs = sorted(logwatch.records, key=lambda rec: rec.getMessage())
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
396 self.assertEqual(len(logs), 2)
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
397 self.assertTrue(logs[0].getMessage().startswith(
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
398 "Deleted 1 stray log levels (0 errors)"))
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
399 self.assertTrue(logs[1].getMessage().startswith(
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
400 "Deleted stray log levels file 2.log.levels"))
5f95a6f38490 Add upgrade method that cleans up .log.levels files which at one point where not properly deleted when builds were removed (with test). Fix fix_log_levels_misnaming test to not rely on the order of logging messages (log message order reflects the directory listing order which can vary).
hodgestar
parents: 735
diff changeset
401
860
ac28bc3feb2a Refuse to create log folder during database upgrade if the folder already exists. The existence of such a folder is an indication that a previous attempt to migrate the logs into files has failed and that the folder should be removed before continuing. See #462 for further discussion.
hodgestar
parents: 846
diff changeset
402 def test_migrate_logs_to_files_with_logs_dir(self):
ac28bc3feb2a Refuse to create log folder during database upgrade if the folder already exists. The existence of such a folder is an indication that a previous attempt to migrate the logs into files has failed and that the folder should be removed before continuing. See #462 for further discussion.
hodgestar
parents: 846
diff changeset
403 os.makedirs(self.logs_dir)
ac28bc3feb2a Refuse to create log folder during database upgrade if the folder already exists. The existence of such a folder is an indication that a previous attempt to migrate the logs into files has failed and that the folder should be removed before continuing. See #462 for further discussion.
hodgestar
parents: 846
diff changeset
404 self.assertRaises(TracError, upgrades.migrate_logs_to_files,
ac28bc3feb2a Refuse to create log folder during database upgrade if the folder already exists. The existence of such a folder is an indication that a previous attempt to migrate the logs into files has failed and that the folder should be removed before continuing. See #462 for further discussion.
hodgestar
parents: 846
diff changeset
405 self.env, None)
ac28bc3feb2a Refuse to create log folder during database upgrade if the folder already exists. The existence of such a folder is an indication that a previous attempt to migrate the logs into files has failed and that the folder should be removed before continuing. See #462 for further discussion.
hodgestar
parents: 846
diff changeset
406
708
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
407
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
408 def suite():
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
409 suite = unittest.TestSuite()
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
410 suite.addTest(unittest.makeSuite(UpgradeHelperTestCase, 'test'))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
411 suite.addTest(unittest.makeSuite(UpgradeScriptsTestCase, 'test'))
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
412 return suite
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
413
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
414 if __name__ == '__main__':
0d7d6552477e Database upgrade cleanup:
hodgestar
parents:
diff changeset
415 unittest.main(defaultTest='suite')
Copyright (C) 2012-2017 Edgewall Software