Mercurial > bitten > bitten-test

annotate bitten/upgrades.py @ 379:0df178e07fdb

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Use UTF-8 as encoding of source files.
author cmlenz
date Tue, 24 Jul 2007 17:32:02 +0000
parents bc6285c0e71f
children 933105ab516b
rev   line source
379
0df178e07fdb Use UTF-8 as encoding of source files.
cmlenz
parents: 323
diff changeset
1 # -*- coding: utf-8 -*-
112
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
2 #
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
3 # Copyright (C) 2005 Christopher Lenz <cmlenz@gmx.de>
163
634be6cbb808 Flip the switch: Bitten is now BSD-licensed.
cmlenz
parents: 147
diff changeset
4 # All rights reserved.
112
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
5 #
163
634be6cbb808 Flip the switch: Bitten is now BSD-licensed.
cmlenz
parents: 147
diff changeset
6 # This software is licensed as described in the file COPYING, which
634be6cbb808 Flip the switch: Bitten is now BSD-licensed.
cmlenz
parents: 147
diff changeset
7 # you should have received as part of this distribution. The terms
634be6cbb808 Flip the switch: Bitten is now BSD-licensed.
cmlenz
parents: 147
diff changeset
8 # are also available at http://bitten.cmlenz.net/wiki/License.
112
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
9
316
87c9b1e8f086 More docstring improvements.
cmlenz
parents: 245
diff changeset
10 """Automated upgrades for the Bitten database tables, and other data stored
87c9b1e8f086 More docstring improvements.
cmlenz
parents: 245
diff changeset
11 in the Trac environment."""
87c9b1e8f086 More docstring improvements.
cmlenz
parents: 245
diff changeset
12
174
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
13 import os
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
14 import sys
174
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
15
323
bc6285c0e71f More fixes with interoperability with Trac trunk. Closes #91. Thanks to Sven Reimers for the patch.
cmlenz
parents: 320
diff changeset
16 from bitten.trac_ext.compat import schema_to_sql
320
a8b713254286 Fixes for compatibility with Trac trunk and 0.9.3.
cmlenz
parents: 316
diff changeset
17
112
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
18 def add_log_table(env, db):
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
19 """Add a table for storing the builds logs."""
112
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
20 from bitten.model import BuildLog, BuildStep
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
21 cursor = db.cursor()
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
22
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
23 for table in BuildLog._schema:
320
a8b713254286 Fixes for compatibility with Trac trunk and 0.9.3.
cmlenz
parents: 316
diff changeset
24 for stmt in schema_to_sql(env, db, table):
112
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
25 cursor.execute(stmt)
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
26
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
27 cursor.execute("SELECT build,name,log FROM bitten_step "
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
28 "WHERE log IS NOT NULL")
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
29 for build, step, log in cursor:
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
30 build_log = BuildLog(env, build, step)
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
31 build_log.messages = [(BuildLog.INFO, msg) for msg in log.splitlines()]
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
32 build_log.insert(db)
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
33
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
34 cursor.execute("CREATE TEMP TABLE old_step AS SELECT * FROM bitten_step")
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
35 cursor.execute("DROP TABLE bitten_step")
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
36 for table in BuildStep._schema:
320
a8b713254286 Fixes for compatibility with Trac trunk and 0.9.3.
cmlenz
parents: 316
diff changeset
37 for stmt in schema_to_sql(env, db, table):
112
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
38 cursor.execute(stmt)
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
39 cursor.execute("INSERT INTO bitten_step (build,name,description,status,"
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
40 "started,stopped) SELECT build,name,description,status,"
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
41 "started,stopped FROM old_step")
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
42
147
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
43 def add_recipe_to_config(env, db):
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
44 """Add a column for storing the build recipe to the build configuration
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
45 table."""
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
46
147
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
47 from bitten.model import BuildConfig
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
48 cursor = db.cursor()
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
49
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
50 cursor.execute("CREATE TEMP TABLE old_config AS "
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
51 "SELECT * FROM bitten_config")
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
52 cursor.execute("DROP TABLE bitten_config")
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
53 for table in BuildConfig._schema:
320
a8b713254286 Fixes for compatibility with Trac trunk and 0.9.3.
cmlenz
parents: 316
diff changeset
54 for stmt in schema_to_sql(env, db, table):
147
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
55 cursor.execute(stmt)
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
56 cursor.execute("INSERT INTO bitten_config (name,path,active,recipe,min_rev,"
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
57 "max_rev,label,description) SELECT name,path,0,'',NULL,"
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
58 "NULL,label,description FROM old_config")
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
59
174
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
60 def add_config_to_reports(env, db):
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
61 """Add the name of the build configuration as metadata to report documents
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
62 stored in the BDB XML database."""
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
63
174
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
64 from bitten.model import Build
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
65 try:
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
66 from bsddb3 import db as bdb
174
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
67 import dbxml
219
aef09843d367 Some pylint-inspired cleanup.
cmlenz
parents: 213
diff changeset
68 except ImportError:
174
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
69 return
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
70
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
71 dbfile = os.path.join(env.path, 'db', 'bitten.dbxml')
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
72 if not os.path.isfile(dbfile):
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
73 return
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
74
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
75 dbenv = bdb.DBEnv()
200
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
76 dbenv.open(os.path.dirname(dbfile),
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
77 bdb.DB_CREATE | bdb.DB_INIT_LOCK | bdb.DB_INIT_LOG |
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
78 bdb.DB_INIT_MPOOL | bdb.DB_INIT_TXN, 0)
200
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
79
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
80 mgr = dbxml.XmlManager(dbenv, 0)
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
81 xtn = mgr.createTransaction()
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
82 container = mgr.openContainer(dbfile, dbxml.DBXML_TRANSACTIONAL)
174
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
83 uc = mgr.createUpdateContext()
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
84
200
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
85 container.addIndex(xtn, '', 'config', 'node-metadata-equality-string', uc)
174
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
86
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
87 qc = mgr.createQueryContext()
200
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
88 for value in mgr.query(xtn, 'collection("%s")/report' % dbfile, qc):
174
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
89 doc = value.asDocument()
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
90 metaval = dbxml.XmlValue()
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
91 if doc.getMetaData('', 'build', metaval):
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
92 build_id = int(metaval.asNumber())
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
93 build = Build.fetch(env, id=build_id, db=db)
175
f7c2f112afe6 Fix the BDB XML upgrade procedure: If a document is found that is associated with a non-existing build, just remove it.
cmlenz
parents: 174
diff changeset
94 if build:
f7c2f112afe6 Fix the BDB XML upgrade procedure: If a document is found that is associated with a non-existing build, just remove it.
cmlenz
parents: 174
diff changeset
95 doc.setMetaData('', 'config', dbxml.XmlValue(build.config))
200
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
96 container.updateDocument(xtn, doc, uc)
175
f7c2f112afe6 Fix the BDB XML upgrade procedure: If a document is found that is associated with a non-existing build, just remove it.
cmlenz
parents: 174
diff changeset
97 else:
f7c2f112afe6 Fix the BDB XML upgrade procedure: If a document is found that is associated with a non-existing build, just remove it.
cmlenz
parents: 174
diff changeset
98 # an orphaned report, for whatever reason... just remove it
200
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
99 container.deleteDocument(xtn, doc, uc)
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
100
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
101 xtn.commit()
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
102 container.close()
692924ffed80 Changes to the BDB XML report store to support transactions. Closes #47.
cmlenz
parents: 196
diff changeset
103 dbenv.close(0)
174
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
104
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
105 def add_order_to_log(env, db):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
106 """Add order column to log table to make sure that build logs are displayed
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
107 in the order they were generated."""
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
108 from bitten.model import BuildLog
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
109 cursor = db.cursor()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
110
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
111 cursor.execute("CREATE TEMP TABLE old_log AS "
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
112 "SELECT * FROM bitten_log")
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
113 cursor.execute("DROP TABLE bitten_log")
320
a8b713254286 Fixes for compatibility with Trac trunk and 0.9.3.
cmlenz
parents: 316
diff changeset
114 for stmt in schema_to_sql(env, db, BuildLog._schema[0]):
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
115 cursor.execute(stmt)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
116 cursor.execute("INSERT INTO bitten_log (id,build,step,generator,orderno) "
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
117 "SELECT id,build,step,type,0 FROM old_log")
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
118
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
119 def add_report_tables(env, db):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
120 """Add database tables for report storage."""
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
121 from bitten.model import Report
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
122 cursor = db.cursor()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
123
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
124 for table in Report._schema:
320
a8b713254286 Fixes for compatibility with Trac trunk and 0.9.3.
cmlenz
parents: 316
diff changeset
125 for stmt in schema_to_sql(env, db, table):
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
126 cursor.execute(stmt)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
127
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
128 def xmldb_to_db(env, db):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
129 """Migrate report data from Berkeley DB XML to SQL database.
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
130
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
131 Depending on the number of reports stored, this might take rather long.
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
132 After the upgrade is done, the bitten.dbxml file (and any BDB XML log files)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
133 may be deleted. BDB XML is no longer used by Bitten.
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
134 """
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
135 from bitten.model import Report
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
136 from bitten.util import xmlio
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
137 try:
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
138 from bsddb3 import db as bdb
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
139 import dbxml
219
aef09843d367 Some pylint-inspired cleanup.
cmlenz
parents: 213
diff changeset
140 except ImportError:
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
141 return
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
142
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
143 dbfile = os.path.join(env.path, 'db', 'bitten.dbxml')
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
144 if not os.path.isfile(dbfile):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
145 return
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
146
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
147 dbenv = bdb.DBEnv()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
148 dbenv.open(os.path.dirname(dbfile),
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
149 bdb.DB_CREATE | bdb.DB_INIT_LOCK | bdb.DB_INIT_LOG |
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
150 bdb.DB_INIT_MPOOL | bdb.DB_INIT_TXN, 0)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
151
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
152 mgr = dbxml.XmlManager(dbenv, 0)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
153 xtn = mgr.createTransaction()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
154 container = mgr.openContainer(dbfile, dbxml.DBXML_TRANSACTIONAL)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
155
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
156 def get_pylint_items(xml):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
157 for problems_elem in xml.children('problems'):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
158 for problem_elem in problems_elem.children('problem'):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
159 item = {'type': 'problem'}
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
160 item.update(problem_elem.attr)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
161 yield item
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
162
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
163 def get_trace_items(xml):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
164 for cov_elem in xml.children('coverage'):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
165 item = {'type': 'coverage', 'name': cov_elem.attr['module'],
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
166 'file': cov_elem.attr['file'],
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
167 'percentage': cov_elem.attr['percentage']}
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
168 lines = 0
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
169 line_hits = []
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
170 for line_elem in cov_elem.children('line'):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
171 lines += 1
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
172 line_hits.append(line_elem.attr['hits'])
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
173 item['lines'] = lines
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
174 item['line_hits'] = ' '.join(line_hits)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
175 yield item
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
176
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
177 def get_unittest_items(xml):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
178 for test_elem in xml.children('test'):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
179 item = {'type': 'test'}
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
180 item.update(test_elem.attr)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
181 for child_elem in test_elem.children():
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
182 item[child_elem.name] = child_elem.gettext()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
183 yield item
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
184
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
185 qc = mgr.createQueryContext()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
186 for value in mgr.query(xtn, 'collection("%s")/report' % dbfile, qc, 0):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
187 doc = value.asDocument()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
188 metaval = dbxml.XmlValue()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
189 build, step = None, None
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
190 if doc.getMetaData('', 'build', metaval):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
191 build = metaval.asNumber()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
192 if doc.getMetaData('', 'step', metaval):
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
193 step = metaval.asString()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
194
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
195 report_types = {'pylint': ('lint', get_pylint_items),
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
196 'trace': ('coverage', get_trace_items),
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
197 'unittest': ('test', get_unittest_items)}
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
198 xml = xmlio.parse(value.asString())
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
199 report_type = xml.attr['type']
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
200 category, get_items = report_types[report_type]
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
201 sys.stderr.write('.')
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
202 sys.stderr.flush()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
203 report = Report(env, build, step, category=category,
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
204 generator=report_type)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
205 report.items = list(get_items(xml))
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
206 try:
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
207 report.insert(db=db)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
208 except AssertionError:
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
209 # Duplicate report, skip
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
210 pass
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
211 sys.stderr.write('\n')
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
212 sys.stderr.flush()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
213
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
214 xtn.abort()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
215 container.close()
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
216 dbenv.close(0)
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
217
213
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
218 def normalize_file_paths(env, db):
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
219 """Normalize the file separator in file names in reports."""
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
220 cursor = db.cursor()
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
221 cursor.execute("SELECT report,item,value FROM bitten_report_item "
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
222 "WHERE name='file'")
223
067bde207c23 Fix upgrade for installs with no stored reports. Thanks to Matt Good for catching this.
cmlenz
parents: 219
diff changeset
223 rows = cursor.fetchall() or []
213
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
224 for report, item, value in rows:
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
225 if '\\' in value:
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
226 cursor.execute("UPDATE bitten_report_item SET value=%s "
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
227 "WHERE report=%s AND item=%s AND name='file'",
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
228 (value.replace('\\', '/'), report, item))
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
229
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
230 def fixup_generators(env, db):
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
231 """Upgrade the identifiers for the recipe commands that generated log
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
232 messages and report data."""
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
233
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
234 mapping = {
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
235 'pipe': 'http://bitten.cmlenz.net/tools/sh#pipe',
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
236 'make': 'http://bitten.cmlenz.net/tools/c#make',
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
237 'distutils': 'http://bitten.cmlenz.net/tools/python#distutils',
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
238 'exec_': 'http://bitten.cmlenz.net/tools/python#exec' # Ambigious
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
239 }
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
240 cursor = db.cursor()
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
241 cursor.execute("SELECT id,generator FROM bitten_log "
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
242 "WHERE generator IN (%s)"
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
243 % ','.join([repr(key) for key in mapping.keys()]))
219
aef09843d367 Some pylint-inspired cleanup.
cmlenz
parents: 213
diff changeset
244 for log_id, generator in cursor:
213
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
245 cursor.execute("UPDATE bitten_log SET generator=%s "
219
aef09843d367 Some pylint-inspired cleanup.
cmlenz
parents: 213
diff changeset
246 "WHERE id=%s", (mapping[generator], log_id))
213
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
247
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
248 mapping = {
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
249 'unittest': 'http://bitten.cmlenz.net/tools/python#unittest',
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
250 'trace': 'http://bitten.cmlenz.net/tools/python#trace',
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
251 'pylint': 'http://bitten.cmlenz.net/tools/python#pylint'
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
252 }
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
253 cursor.execute("SELECT id,generator FROM bitten_report "
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
254 "WHERE generator IN (%s)"
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
255 % ','.join([repr(key) for key in mapping.keys()]))
219
aef09843d367 Some pylint-inspired cleanup.
cmlenz
parents: 213
diff changeset
256 for report_id, generator in cursor:
213
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
257 cursor.execute("UPDATE bitten_report SET generator=%s "
219
aef09843d367 Some pylint-inspired cleanup.
cmlenz
parents: 213
diff changeset
258 "WHERE id=%s", (mapping[generator], report_id))
213
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
259
245
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
260 def add_error_table(env, db):
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
261 """Add the bitten_error table for recording step failure reasons."""
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
262 from trac.db import Table, Column
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
263
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
264 table = Table('bitten_error', key=('build', 'step', 'orderno'))[
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
265 Column('build', type='int'), Column('step'), Column('message'),
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
266 Column('orderno', type='int')
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
267 ]
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
268 cursor = db.cursor()
320
a8b713254286 Fixes for compatibility with Trac trunk and 0.9.3.
cmlenz
parents: 316
diff changeset
269 for stmt in schema_to_sql(env, db, table):
245
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
270 cursor.execute(stmt)
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
271
112
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
272 map = {
147
395b67aa072e Build recipes are now stored in the database with the build configuration. This means that it is no longer necessary to store the recipe in the repository. Closes #41.
cmlenz
parents: 112
diff changeset
273 2: [add_log_table],
174
79c61c26a4e1 * Changed the `IReportStore` interface to allow querying with [http://www.w3.org/XML/Query/ XQuery]. This should make it possible to efficiently query the report store for any existing metrics.
cmlenz
parents: 163
diff changeset
274 3: [add_recipe_to_config],
203
e6ddca1e5712 Huge refactoring to remove dependency on BDB XML. Report data is now stored in the Trac database (SQLite/PostgreSQL).
cmlenz
parents: 200
diff changeset
275 4: [add_config_to_reports],
213
25f84dd9f159 * Refactoring of build recipes, the file format has changed slightly:
cmlenz
parents: 203
diff changeset
276 5: [add_order_to_log, add_report_tables, xmldb_to_db],
245
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
277 6: [normalize_file_paths, fixup_generators],
a22ec8fce6c9 Store the reason(s) for build step failure in the database.
cmlenz
parents: 223
diff changeset
278 7: [add_error_table]
112
a38eabd4b6e1 * Store build logs in a structured way, for example to highlight messages on the error stream.
cmlenz
parents:
diff changeset
279 }
Copyright (C) 2012-2017 Edgewall Software