genshi/mirror: markup/input.py annotate

annotate markup/input.py @ 140:c1f4390d50f8 trunk

Fix bug in HTML serializer, plus some other minor tweaks.

author	cmlenz
date	Wed, 09 Aug 2006 21:00:15 +0000
parents	d681d2c3cd8d
children	3d4c214c979a

rev	line source
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	1 # -- coding: utf-8 --
5479aae32f5a Initial import. cmlenz parents: diff changeset	2 #
66 59eb24184e9c Switch copyright to Edgewall and URLs to markup.edgewall.org. cmlenz parents: 27 diff changeset	3 # Copyright (C) 2006 Edgewall Software
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	4 # All rights reserved.
5479aae32f5a Initial import. cmlenz parents: diff changeset	5 #
5479aae32f5a Initial import. cmlenz parents: diff changeset	6 # This software is licensed as described in the file COPYING, which
5479aae32f5a Initial import. cmlenz parents: diff changeset	7 # you should have received as part of this distribution. The terms
66 59eb24184e9c Switch copyright to Edgewall and URLs to markup.edgewall.org. cmlenz parents: 27 diff changeset	8 # are also available at http://markup.edgewall.org/wiki/License.
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	9 #
5479aae32f5a Initial import. cmlenz parents: diff changeset	10 # This software consists of voluntary contributions made by many
5479aae32f5a Initial import. cmlenz parents: diff changeset	11 # individuals. For the exact contribution history, see the revision
66 59eb24184e9c Switch copyright to Edgewall and URLs to markup.edgewall.org. cmlenz parents: 27 diff changeset	12 # history and logs, available at http://markup.edgewall.org/log/.
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	13
5479aae32f5a Initial import. cmlenz parents: diff changeset	14 from xml.parsers import expat
5479aae32f5a Initial import. cmlenz parents: diff changeset	15 try:
5479aae32f5a Initial import. cmlenz parents: diff changeset	16 frozenset
5479aae32f5a Initial import. cmlenz parents: diff changeset	17 except NameError:
5479aae32f5a Initial import. cmlenz parents: diff changeset	18 from sets import ImmutableSet as frozenset
5479aae32f5a Initial import. cmlenz parents: diff changeset	19 import HTMLParser as html
5479aae32f5a Initial import. cmlenz parents: diff changeset	20 import htmlentitydefs
5479aae32f5a Initial import. cmlenz parents: diff changeset	21 from StringIO import StringIO
5479aae32f5a Initial import. cmlenz parents: diff changeset	22
5479aae32f5a Initial import. cmlenz parents: diff changeset	23 from markup.core import Attributes, Markup, QName, Stream
5479aae32f5a Initial import. cmlenz parents: diff changeset	24
5479aae32f5a Initial import. cmlenz parents: diff changeset	25
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	26 class ParseError(Exception):
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	27 """Exception raised when fatal syntax errors are found in the input being
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	28 parsed."""
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	29
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	30 def __init__(self, message, filename='<string>', lineno=-1, offset=-1):
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	31 Exception.__init__(self, message)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	32 self.filename = filename
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	33 self.lineno = lineno
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	34 self.offset = offset
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	35
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	36
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	37 class XMLParser(object):
5479aae32f5a Initial import. cmlenz parents: diff changeset	38 """Generator-based XML parser based on roughly equivalent code in
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	39 Kid/ElementTree.
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	40
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	41 The parsing is initiated by iterating over the parser object:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	42
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	43 >>> parser = XMLParser(StringIO('<root id="2"><child>Foo</child></root>'))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	44 >>> for kind, data, pos in parser:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	45 ... print kind, data
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	46 START (u'root', [(u'id', u'2')])
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	47 START (u'child', [])
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	48 TEXT Foo
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	49 END child
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	50 END root
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	51 """
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	52
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	53 def __init__(self, source, filename=None):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	54 """Initialize the parser for the given XML text.
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	55
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	56 @param source: the XML text as a file-like object
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	57 @param filename: the name of the file, if appropriate
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	58 """
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	59 self.source = source
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	60 self.filename = filename
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	61
5479aae32f5a Initial import. cmlenz parents: diff changeset	62 # Setup the Expat parser
5479aae32f5a Initial import. cmlenz parents: diff changeset	63 parser = expat.ParserCreate('utf-8', '}')
5479aae32f5a Initial import. cmlenz parents: diff changeset	64 parser.buffer_text = True
5479aae32f5a Initial import. cmlenz parents: diff changeset	65 parser.returns_unicode = True
5479aae32f5a Initial import. cmlenz parents: diff changeset	66 parser.StartElementHandler = self._handle_start
5479aae32f5a Initial import. cmlenz parents: diff changeset	67 parser.EndElementHandler = self._handle_end
5479aae32f5a Initial import. cmlenz parents: diff changeset	68 parser.CharacterDataHandler = self._handle_data
5479aae32f5a Initial import. cmlenz parents: diff changeset	69 parser.StartDoctypeDeclHandler = self._handle_doctype
5479aae32f5a Initial import. cmlenz parents: diff changeset	70 parser.StartNamespaceDeclHandler = self._handle_start_ns
5479aae32f5a Initial import. cmlenz parents: diff changeset	71 parser.EndNamespaceDeclHandler = self._handle_end_ns
5479aae32f5a Initial import. cmlenz parents: diff changeset	72 parser.ProcessingInstructionHandler = self._handle_pi
5479aae32f5a Initial import. cmlenz parents: diff changeset	73 parser.CommentHandler = self._handle_comment
5479aae32f5a Initial import. cmlenz parents: diff changeset	74 parser.DefaultHandler = self._handle_other
5479aae32f5a Initial import. cmlenz parents: diff changeset	75
5479aae32f5a Initial import. cmlenz parents: diff changeset	76 # Location reporting is only support in Python >= 2.4
5479aae32f5a Initial import. cmlenz parents: diff changeset	77 if not hasattr(parser, 'CurrentLineNumber'):
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	78 self._getpos = self._getpos_unknown
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	79
5479aae32f5a Initial import. cmlenz parents: diff changeset	80 self.expat = parser
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	81 self._queue = []
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	82
5479aae32f5a Initial import. cmlenz parents: diff changeset	83 def __iter__(self):
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	84 try:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	85 bufsize = 4 * 1024 # 4K
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	86 done = False
69 c40a5dcd2b55 A couple of minor performance improvements. cmlenz parents: 66 diff changeset	87 while 1:
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	88 while not done and len(self._queue) == 0:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	89 data = self.source.read(bufsize)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	90 if data == '': # end of data
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	91 if hasattr(self, 'expat'):
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	92 self.expat.Parse('', True)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	93 del self.expat # get rid of circular references
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	94 done = True
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	95 else:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	96 self.expat.Parse(data, False)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	97 for event in self._queue:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	98 yield event
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	99 self._queue = []
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	100 if done:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	101 break
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	102 except expat.ExpatError, e:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	103 msg = str(e)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	104 if self.filename:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	105 msg += ', in ' + self.filename
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	106 raise ParseError(msg, self.filename, e.lineno, e.offset)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	107
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	108 def _enqueue(self, kind, data, pos=None):
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	109 if pos is None:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	110 pos = self._getpos()
134 d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	111 if kind is Stream.TEXT:
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	112 # Expat reports the end of the text event as current position. We
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	113 # try to fix that up here as much as possible. Unfortunately, the
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	114 # offset is only valid for single-line text. For multi-line text,
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	115 # it is apparently not possible to determine at what offset it
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	116 # started
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	117 if '\n' in data:
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	118 lines = data.splitlines()
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	119 lineno = pos[1] - len(lines) + 1
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	120 offset = -1
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	121 else:
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	122 lineno = pos[1]
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	123 offset = pos[2] - len(data)
d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	124 pos = (pos[0], lineno, offset)
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	125 self._queue.append((kind, data, pos))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	126
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	127 def _getpos_unknown(self):
134 d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	128 return (self.filename, -1, -1)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	129
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	130 def _getpos(self):
134 d681d2c3cd8d * Improve the accuracy of line numbers for text nodes, so that reported errors about syntax or evaluation errors in expressions point to the right line (not quite perfect yet, though). cmlenz parents: 69 diff changeset	131 return (self.filename, self.expat.CurrentLineNumber,
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	132 self.expat.CurrentColumnNumber)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	133
5479aae32f5a Initial import. cmlenz parents: diff changeset	134 def _handle_start(self, tag, attrib):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	135 self._enqueue(Stream.START, (QName(tag), Attributes(attrib.items())))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	136
5479aae32f5a Initial import. cmlenz parents: diff changeset	137 def _handle_end(self, tag):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	138 self._enqueue(Stream.END, QName(tag))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	139
5479aae32f5a Initial import. cmlenz parents: diff changeset	140 def _handle_data(self, text):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	141 self._enqueue(Stream.TEXT, text)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	142
5479aae32f5a Initial import. cmlenz parents: diff changeset	143 def _handle_doctype(self, name, sysid, pubid, has_internal_subset):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	144 self._enqueue(Stream.DOCTYPE, (name, pubid, sysid))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	145
5479aae32f5a Initial import. cmlenz parents: diff changeset	146 def _handle_start_ns(self, prefix, uri):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	147 self._enqueue(Stream.START_NS, (prefix or '', uri))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	148
5479aae32f5a Initial import. cmlenz parents: diff changeset	149 def _handle_end_ns(self, prefix):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	150 self._enqueue(Stream.END_NS, prefix or '')
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	151
5479aae32f5a Initial import. cmlenz parents: diff changeset	152 def _handle_pi(self, target, data):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	153 self._enqueue(Stream.PI, (target, data))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	154
5479aae32f5a Initial import. cmlenz parents: diff changeset	155 def _handle_comment(self, text):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	156 self._enqueue(Stream.COMMENT, text)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	157
5479aae32f5a Initial import. cmlenz parents: diff changeset	158 def _handle_other(self, text):
5479aae32f5a Initial import. cmlenz parents: diff changeset	159 if text.startswith('&'):
5479aae32f5a Initial import. cmlenz parents: diff changeset	160 # deal with undefined entities
5479aae32f5a Initial import. cmlenz parents: diff changeset	161 try:
5479aae32f5a Initial import. cmlenz parents: diff changeset	162 text = unichr(htmlentitydefs.name2codepoint[text[1:-1]])
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	163 self._enqueue(Stream.TEXT, text)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	164 except KeyError:
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	165 lineno, offset = self._getpos()
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	166 raise expat.error("undefined entity %s: line %d, column %d" %
5479aae32f5a Initial import. cmlenz parents: diff changeset	167 (text, lineno, offset))
5479aae32f5a Initial import. cmlenz parents: diff changeset	168
5479aae32f5a Initial import. cmlenz parents: diff changeset	169
5479aae32f5a Initial import. cmlenz parents: diff changeset	170 def XML(text):
5479aae32f5a Initial import. cmlenz parents: diff changeset	171 return Stream(list(XMLParser(StringIO(text))))
5479aae32f5a Initial import. cmlenz parents: diff changeset	172
5479aae32f5a Initial import. cmlenz parents: diff changeset	173
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	174 class HTMLParser(html.HTMLParser, object):
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	175 """Parser for HTML input based on the Python `HTMLParser` module.
5479aae32f5a Initial import. cmlenz parents: diff changeset	176
5479aae32f5a Initial import. cmlenz parents: diff changeset	177 This class provides the same interface for generating stream events as
5479aae32f5a Initial import. cmlenz parents: diff changeset	178 `XMLParser`, and attempts to automatically balance tags.
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	179
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	180 The parsing is initiated by iterating over the parser object:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	181
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	182 >>> parser = HTMLParser(StringIO('<UL compact><LI>Foo</UL>'))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	183 >>> for kind, data, pos in parser:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	184 ... print kind, data
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	185 START (u'ul', [(u'compact', u'compact')])
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	186 START (u'li', [])
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	187 TEXT Foo
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	188 END li
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	189 END ul
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	190 """
5479aae32f5a Initial import. cmlenz parents: diff changeset	191
5479aae32f5a Initial import. cmlenz parents: diff changeset	192 _EMPTY_ELEMS = frozenset(['area', 'base', 'basefont', 'br', 'col', 'frame',
5479aae32f5a Initial import. cmlenz parents: diff changeset	193 'hr', 'img', 'input', 'isindex', 'link', 'meta',
5479aae32f5a Initial import. cmlenz parents: diff changeset	194 'param'])
5479aae32f5a Initial import. cmlenz parents: diff changeset	195
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	196 def __init__(self, source, filename=None):
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	197 html.HTMLParser.__init__(self)
5479aae32f5a Initial import. cmlenz parents: diff changeset	198 self.source = source
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	199 self.filename = filename
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	200 self._queue = []
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	201 self._open_tags = []
5479aae32f5a Initial import. cmlenz parents: diff changeset	202
5479aae32f5a Initial import. cmlenz parents: diff changeset	203 def __iter__(self):
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	204 try:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	205 bufsize = 4 * 1024 # 4K
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	206 done = False
69 c40a5dcd2b55 A couple of minor performance improvements. cmlenz parents: 66 diff changeset	207 while 1:
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	208 while not done and len(self._queue) == 0:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	209 data = self.source.read(bufsize)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	210 if data == '': # end of data
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	211 self.close()
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	212 done = True
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	213 else:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	214 self.feed(data)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	215 for kind, data, pos in self._queue:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	216 yield kind, data, pos
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	217 self._queue = []
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	218 if done:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	219 open_tags = self._open_tags
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	220 open_tags.reverse()
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	221 for tag in open_tags:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	222 yield Stream.END, QName(tag), pos
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	223 break
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	224 except html.HTMLParseError, e:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	225 msg = '%s: line %d, column %d' % (e.msg, e.lineno, e.offset)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	226 if self.filename:
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	227 msg += ', in %s' % self.filename
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	228 raise ParseError(msg, self.filename, e.lineno, e.offset)
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	229
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	230 def _enqueue(self, kind, data, pos=None):
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	231 if pos is None:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	232 pos = self._getpos()
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	233 self._queue.append((kind, data, pos))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	234
21 b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	235 def _getpos(self):
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	236 lineno, column = self.getpos()
b4d17897d053 * Include paths are now interpreted relative to the path of the including template. Closes #3. cmlenz parents: 1 diff changeset	237 return (self.filename, lineno, column)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	238
5479aae32f5a Initial import. cmlenz parents: diff changeset	239 def handle_starttag(self, tag, attrib):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	240 fixed_attrib = []
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	241 for name, value in attrib: # Fixup minimized attributes
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	242 if value is None:
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	243 value = name
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	244 fixed_attrib.append((name, unicode(value)))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	245
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	246 self._enqueue(Stream.START, (QName(tag), Attributes(fixed_attrib)))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	247 if tag in self._EMPTY_ELEMS:
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	248 self._enqueue(Stream.END, QName(tag))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	249 else:
5479aae32f5a Initial import. cmlenz parents: diff changeset	250 self._open_tags.append(tag)
5479aae32f5a Initial import. cmlenz parents: diff changeset	251
5479aae32f5a Initial import. cmlenz parents: diff changeset	252 def handle_endtag(self, tag):
5479aae32f5a Initial import. cmlenz parents: diff changeset	253 if tag not in self._EMPTY_ELEMS:
5479aae32f5a Initial import. cmlenz parents: diff changeset	254 while self._open_tags:
5479aae32f5a Initial import. cmlenz parents: diff changeset	255 open_tag = self._open_tags.pop()
5479aae32f5a Initial import. cmlenz parents: diff changeset	256 if open_tag.lower() == tag.lower():
5479aae32f5a Initial import. cmlenz parents: diff changeset	257 break
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	258 self._enqueue(Stream.END, QName(open_tag))
3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	259 self._enqueue(Stream.END, QName(tag))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	260
5479aae32f5a Initial import. cmlenz parents: diff changeset	261 def handle_data(self, text):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	262 self._enqueue(Stream.TEXT, text)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	263
5479aae32f5a Initial import. cmlenz parents: diff changeset	264 def handle_charref(self, name):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	265 self._enqueue(Stream.TEXT, Markup('&#%s;' % name))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	266
5479aae32f5a Initial import. cmlenz parents: diff changeset	267 def handle_entityref(self, name):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	268 self._enqueue(Stream.TEXT, Markup('&%s;' % name))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	269
5479aae32f5a Initial import. cmlenz parents: diff changeset	270 def handle_pi(self, data):
5479aae32f5a Initial import. cmlenz parents: diff changeset	271 target, data = data.split(maxsplit=1)
5479aae32f5a Initial import. cmlenz parents: diff changeset	272 data = data.rstrip('?')
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	273 self._enqueue(Stream.PI, (target.strip(), data.strip()))
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	274
5479aae32f5a Initial import. cmlenz parents: diff changeset	275 def handle_comment(self, text):
26 3c1a022be04c * Split out the XPath tests into a separate `unittest`-based file. cmlenz parents: 21 diff changeset	276 self._enqueue(Stream.COMMENT, text)
1 5479aae32f5a Initial import. cmlenz parents: diff changeset	277
5479aae32f5a Initial import. cmlenz parents: diff changeset	278
5479aae32f5a Initial import. cmlenz parents: diff changeset	279 def HTML(text):
5479aae32f5a Initial import. cmlenz parents: diff changeset	280 return Stream(list(HTMLParser(StringIO(text))))

Mercurial > genshi > mirror

annotate markup/input.py @ 140:c1f4390d50f8 trunk