genshi/mirror: markup/input.py annotate

annotate markup/input.py @ 139:8332287b5508 trunk

Implement position reporting for XPath syntax errors. Closes #20.

author	cmlenz
date	Wed, 09 Aug 2006 12:25:02 +0000
parents	d681d2c3cd8d
children	c1f4390d50f8

rev	line source
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	1 # -- coding: utf-8 --
5479aae32f5a Initial import. cmlenz parents: diff changeset	2 #
66 59eb24184e9c Switch copyright to Edgewall and URLs to markup.edgewall.org. cmlenz parents: 27 diff changeset	3 # Copyright (C) 2006 Edgewall Software
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	4 # All rights reserved.
5479aae32f5a Initial import. cmlenz parents: diff changeset	5 #
5479aae32f5a Initial import. cmlenz parents: diff changeset	6 # This software is licensed as described in the file COPYING, which
5479aae32f5a Initial import. cmlenz parents: diff changeset	7 # you should have received as part of this distribution. The terms
66 59eb24184e9c Switch copyright to Edgewall and URLs to markup.edgewall.org. cmlenz parents: 27 diff changeset	8 # are also available at http://markup.edgewall.org/wiki/License.
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	9 #
5479aae32f5a Initial import. cmlenz parents: diff changeset	10 # This software consists of voluntary contributions made by many
5479aae32f5a Initial import. cmlenz parents: diff changeset	11 # individuals. For the exact contribution history, see the revision
66 59eb24184e9c Switch copyright to Edgewall and URLs to markup.edgewall.org. cmlenz parents: 27 diff changeset	12 # history and logs, available at http://markup.edgewall.org/log/.
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	13
5479aae32f5a Initial import. cmlenz parents: diff changeset	14 from xml.parsers import expat
5479aae32f5a Initial import. cmlenz parents: diff changeset	15 try:
5479aae32f5a Initial import. cmlenz parents: diff changeset	16 frozenset
5479aae32f5a Initial import. cmlenz parents: diff changeset	17 except NameError:
5479aae32f5a Initial import. cmlenz parents: diff changeset	18 from sets import ImmutableSet as frozenset
5479aae32f5a Initial import. cmlenz parents: diff changeset	19 import HTMLParser as html
5479aae32f5a Initial import. cmlenz parents: diff changeset	20 import htmlentitydefs
5479aae32f5a Initial import. cmlenz parents: diff changeset	21 from StringIO import StringIO
5479aae32f5a Initial import. cmlenz parents: diff changeset	22
5479aae32f5a Initial import. cmlenz parents: diff changeset	23 from markup.core import Attributes, Markup, QName, Stream
5479aae32f5a Initial import. cmlenz parents: diff changeset	24
5479aae32f5a Initial import. cmlenz parents: diff changeset	25
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	26 class ParseError(Exception):
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	27 """Exception raised when fatal syntax errors are found in the input being
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	28 parsed."""
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	29
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	30 def __init__(self, message, filename='<string>', lineno=-1, offset=-1):
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	31 Exception.__init__(self, message)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	32 self.filename = filename
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	33 self.lineno = lineno
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	34 self.offset = offset
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	35
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	36
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	37 class XMLParser(object):
5479aae32f5a Initial import. cmlenz parents: diff changeset	38 """Generator-based XML parser based on roughly equivalent code in
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	39 Kid/ElementTree.
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	40
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	41 The parsing is initiated by iterating over the parser object:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	42
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	43 >>> parser = XMLParser(StringIO('<root id="2"><child>Foo</child></root>'))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	44 >>> for kind, data, pos in parser:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	45 ... print kind, data
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	46 START (u'root', [(u'id', u'2')])
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	47 START (u'child', [])
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	48 TEXT Foo
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	49 END child
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	50 END root
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	51 """
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	52
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	53 def __init__(self, source, filename=None):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	54 """Initialize the parser for the given XML text.
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	55
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	56 @param source: the XML text as a file-like object
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	57 @param filename: the name of the file, if appropriate
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	58 """
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	59 self.source = source
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	60 self.filename = filename
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	61
5479aae32f5a Initial import. cmlenz parents: diff changeset	62 # Setup the Expat parser
5479aae32f5a Initial import. cmlenz parents: diff changeset	63 parser = expat.ParserCreate('utf-8', '}')
5479aae32f5a Initial import. cmlenz parents: diff changeset	64 parser.buffer_text = True
5479aae32f5a Initial import. cmlenz parents: diff changeset	65 parser.returns_unicode = True
5479aae32f5a Initial import. cmlenz parents: diff changeset	66 parser.StartElementHandler = self._handle_start
5479aae32f5a Initial import. cmlenz parents: diff changeset	67 parser.EndElementHandler = self._handle_end
5479aae32f5a Initial import. cmlenz parents: diff changeset	68 parser.CharacterDataHandler = self._handle_data
5479aae32f5a Initial import. cmlenz parents: diff changeset	69 parser.XmlDeclHandler = self._handle_prolog
5479aae32f5a Initial import. cmlenz parents: diff changeset	70 parser.StartDoctypeDeclHandler = self._handle_doctype
5479aae32f5a Initial import. cmlenz parents: diff changeset	71 parser.StartNamespaceDeclHandler = self._handle_start_ns
5479aae32f5a Initial import. cmlenz parents: diff changeset	72 parser.EndNamespaceDeclHandler = self._handle_end_ns
5479aae32f5a Initial import. cmlenz parents: diff changeset	73 parser.ProcessingInstructionHandler = self._handle_pi
5479aae32f5a Initial import. cmlenz parents: diff changeset	74 parser.CommentHandler = self._handle_comment
5479aae32f5a Initial import. cmlenz parents: diff changeset	75 parser.DefaultHandler = self._handle_other
5479aae32f5a Initial import. cmlenz parents: diff changeset	76
5479aae32f5a Initial import. cmlenz parents: diff changeset	77 # Location reporting is only support in Python >= 2.4
5479aae32f5a Initial import. cmlenz parents: diff changeset	78 if not hasattr(parser, 'CurrentLineNumber'):
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	79 self._getpos = self._getpos_unknown
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	80
5479aae32f5a Initial import. cmlenz parents: diff changeset	81 self.expat = parser
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	82 self._queue = []
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	83
5479aae32f5a Initial import. cmlenz parents: diff changeset	84 def __iter__(self):
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	85 try:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	86 bufsize = 4 * 1024 # 4K
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	87 done = False
69 c40a5dcd2b55 A couple of minor performance improvements. cmlenz parents: 66 diff changeset	88 while 1:
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	89 while not done and len(self._queue) == 0:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	90 data = self.source.read(bufsize)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	91 if data == '': # end of data
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	92 if hasattr(self, 'expat'):
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	93 self.expat.Parse('', True)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	94 del self.expat # get rid of circular references
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	95 done = True
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	96 else:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	97 self.expat.Parse(data, False)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	98 for event in self._queue:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	99 yield event
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	100 self._queue = []
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	101 if done:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	102 break
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	103 except expat.ExpatError, e:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	104 msg = str(e)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	105 if self.filename:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	106 msg += ', in ' + self.filename
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	107 raise ParseError(msg, self.filename, e.lineno, e.offset)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	108
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	109 def _enqueue(self, kind, data, pos=None):
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	110 if pos is None:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	111 pos = self._getpos()
134 d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	112 if kind is Stream.TEXT:
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	113 # Expat reports the end of the text event as current position. We
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	114 # try to fix that up here as much as possible. Unfortunately, the
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	115 # offset is only valid for single-line text. For multi-line text,
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	116 # it is apparently not possible to determine at what offset it
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	117 # started
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	118 if '\n' in data:
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	119 lines = data.splitlines()
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	120 lineno = pos[1] - len(lines) + 1
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	121 offset = -1
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	122 else:
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	123 lineno = pos[1]
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	124 offset = pos[2] - len(data)
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	125 pos = (pos[0], lineno, offset)
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	126 self._queue.append((kind, data, pos))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	127
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	128 def _getpos_unknown(self):
134 d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	129 return (self.filename, -1, -1)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	130
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	131 def _getpos(self):
134 d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	132 return (self.filename, self.expat.CurrentLineNumber,
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	133 self.expat.CurrentColumnNumber)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	134
5479aae32f5a Initial import. cmlenz parents: diff changeset	135 def _handle_start(self, tag, attrib):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	136 self._enqueue(Stream.START, (QName(tag), Attributes(attrib.items())))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	137
5479aae32f5a Initial import. cmlenz parents: diff changeset	138 def _handle_end(self, tag):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	139 self._enqueue(Stream.END, QName(tag))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	140
5479aae32f5a Initial import. cmlenz parents: diff changeset	141 def _handle_data(self, text):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	142 self._enqueue(Stream.TEXT, text)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	143
5479aae32f5a Initial import. cmlenz parents: diff changeset	144 def _handle_prolog(self, version, encoding, standalone):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	145 self._enqueue(Stream.PROLOG, (version, encoding, standalone))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	146
5479aae32f5a Initial import. cmlenz parents: diff changeset	147 def _handle_doctype(self, name, sysid, pubid, has_internal_subset):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	148 self._enqueue(Stream.DOCTYPE, (name, pubid, sysid))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	149
5479aae32f5a Initial import. cmlenz parents: diff changeset	150 def _handle_start_ns(self, prefix, uri):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	151 self._enqueue(Stream.START_NS, (prefix or '', uri))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	152
5479aae32f5a Initial import. cmlenz parents: diff changeset	153 def _handle_end_ns(self, prefix):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	154 self._enqueue(Stream.END_NS, prefix or '')
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	155
5479aae32f5a Initial import. cmlenz parents: diff changeset	156 def _handle_pi(self, target, data):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	157 self._enqueue(Stream.PI, (target, data))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	158
5479aae32f5a Initial import. cmlenz parents: diff changeset	159 def _handle_comment(self, text):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	160 self._enqueue(Stream.COMMENT, text)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	161
5479aae32f5a Initial import. cmlenz parents: diff changeset	162 def _handle_other(self, text):
5479aae32f5a Initial import. cmlenz parents: diff changeset	163 if text.startswith('&'):
5479aae32f5a Initial import. cmlenz parents: diff changeset	164 # deal with undefined entities
5479aae32f5a Initial import. cmlenz parents: diff changeset	165 try:
5479aae32f5a Initial import. cmlenz parents: diff changeset	166 text = unichr(htmlentitydefs.name2codepoint[text[1:-1]])
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	167 self._enqueue(Stream.TEXT, text)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	168 except KeyError:
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	169 lineno, offset = self._getpos()
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	170 raise expat.error("undefined entity %s: line %d, column %d" %
5479aae32f5a Initial import. cmlenz parents: diff changeset	171 (text, lineno, offset))
5479aae32f5a Initial import. cmlenz parents: diff changeset	172
5479aae32f5a Initial import. cmlenz parents: diff changeset	173
5479aae32f5a Initial import. cmlenz parents: diff changeset	174 def XML(text):
5479aae32f5a Initial import. cmlenz parents: diff changeset	175 return Stream(list(XMLParser(StringIO(text))))
5479aae32f5a Initial import. cmlenz parents: diff changeset	176
5479aae32f5a Initial import. cmlenz parents: diff changeset	177
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	178 class HTMLParser(html.HTMLParser, object):
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	179 """Parser for HTML input based on the Python `HTMLParser` module.
5479aae32f5a Initial import. cmlenz parents: diff changeset	180
5479aae32f5a Initial import. cmlenz parents: diff changeset	181 This class provides the same interface for generating stream events as
5479aae32f5a Initial import. cmlenz parents: diff changeset	182 `XMLParser`, and attempts to automatically balance tags.
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	183
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	184 The parsing is initiated by iterating over the parser object:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	185
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	186 >>> parser = HTMLParser(StringIO('<UL compact><LI>Foo</UL>'))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	187 >>> for kind, data, pos in parser:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	188 ... print kind, data
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	189 START (u'ul', [(u'compact', u'compact')])
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	190 START (u'li', [])
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	191 TEXT Foo
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	192 END li
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	193 END ul
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	194 """
5479aae32f5a Initial import. cmlenz parents: diff changeset	195
5479aae32f5a Initial import. cmlenz parents: diff changeset	196 _EMPTY_ELEMS = frozenset(['area', 'base', 'basefont', 'br', 'col', 'frame',
5479aae32f5a Initial import. cmlenz parents: diff changeset	197 'hr', 'img', 'input', 'isindex', 'link', 'meta',
5479aae32f5a Initial import. cmlenz parents: diff changeset	198 'param'])
5479aae32f5a Initial import. cmlenz parents: diff changeset	199
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	200 def __init__(self, source, filename=None):
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	201 html.HTMLParser.__init__(self)
5479aae32f5a Initial import. cmlenz parents: diff changeset	202 self.source = source
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	203 self.filename = filename
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	204 self._queue = []
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	205 self._open_tags = []
5479aae32f5a Initial import. cmlenz parents: diff changeset	206
5479aae32f5a Initial import. cmlenz parents: diff changeset	207 def __iter__(self):
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	208 try:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	209 bufsize = 4 * 1024 # 4K
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	210 done = False
69 c40a5dcd2b55 A couple of minor performance improvements. cmlenz parents: 66 diff changeset	211 while 1:
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	212 while not done and len(self._queue) == 0:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	213 data = self.source.read(bufsize)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	214 if data == '': # end of data
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	215 self.close()
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	216 done = True
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	217 else:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	218 self.feed(data)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	219 for kind, data, pos in self._queue:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	220 yield kind, data, pos
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	221 self._queue = []
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	222 if done:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	223 open_tags = self._open_tags
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	224 open_tags.reverse()
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	225 for tag in open_tags:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	226 yield Stream.END, QName(tag), pos
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	227 break
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	228 except html.HTMLParseError, e:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	229 msg = '%s: line %d, column %d' % (e.msg, e.lineno, e.offset)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	230 if self.filename:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	231 msg += ', in %s' % self.filename
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	232 raise ParseError(msg, self.filename, e.lineno, e.offset)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	233
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	234 def _enqueue(self, kind, data, pos=None):
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	235 if pos is None:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	236 pos = self._getpos()
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	237 self._queue.append((kind, data, pos))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	238
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	239 def _getpos(self):
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	240 lineno, column = self.getpos()
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	241 return (self.filename, lineno, column)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	242
5479aae32f5a Initial import. cmlenz parents: diff changeset	243 def handle_starttag(self, tag, attrib):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	244 fixed_attrib = []
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	245 for name, value in attrib: # Fixup minimized attributes
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	246 if value is None:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	247 value = name
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	248 fixed_attrib.append((name, unicode(value)))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	249
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	250 self._enqueue(Stream.START, (QName(tag), Attributes(fixed_attrib)))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	251 if tag in self._EMPTY_ELEMS:
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	252 self._enqueue(Stream.END, QName(tag))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	253 else:
5479aae32f5a Initial import. cmlenz parents: diff changeset	254 self._open_tags.append(tag)
5479aae32f5a Initial import. cmlenz parents: diff changeset	255
5479aae32f5a Initial import. cmlenz parents: diff changeset	256 def handle_endtag(self, tag):
5479aae32f5a Initial import. cmlenz parents: diff changeset	257 if tag not in self._EMPTY_ELEMS:
5479aae32f5a Initial import. cmlenz parents: diff changeset	258 while self._open_tags:
5479aae32f5a Initial import. cmlenz parents: diff changeset	259 open_tag = self._open_tags.pop()
5479aae32f5a Initial import. cmlenz parents: diff changeset	260 if open_tag.lower() == tag.lower():
5479aae32f5a Initial import. cmlenz parents: diff changeset	261 break
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	262 self._enqueue(Stream.END, QName(open_tag))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	263 self._enqueue(Stream.END, QName(tag))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	264
5479aae32f5a Initial import. cmlenz parents: diff changeset	265 def handle_data(self, text):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	266 self._enqueue(Stream.TEXT, text)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	267
5479aae32f5a Initial import. cmlenz parents: diff changeset	268 def handle_charref(self, name):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	269 self._enqueue(Stream.TEXT, Markup('&#%s;' % name))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	270
5479aae32f5a Initial import. cmlenz parents: diff changeset	271 def handle_entityref(self, name):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	272 self._enqueue(Stream.TEXT, Markup('&%s;' % name))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	273
5479aae32f5a Initial import. cmlenz parents: diff changeset	274 def handle_pi(self, data):
5479aae32f5a Initial import. cmlenz parents: diff changeset	275 target, data = data.split(maxsplit=1)
5479aae32f5a Initial import. cmlenz parents: diff changeset	276 data = data.rstrip('?')
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	277 self._enqueue(Stream.PI, (target.strip(), data.strip()))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	278
5479aae32f5a Initial import. cmlenz parents: diff changeset	279 def handle_comment(self, text):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	280 self._enqueue(Stream.COMMENT, text)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	281
5479aae32f5a Initial import. cmlenz parents: diff changeset	282
5479aae32f5a Initial import. cmlenz parents: diff changeset	283 def HTML(text):
5479aae32f5a Initial import. cmlenz parents: diff changeset	284 return Stream(list(HTMLParser(StringIO(text))))

Mercurial > genshi > mirror

annotate markup/input.py @ 139:8332287b5508 trunk