Mercurial > genshi > mirror
annotate genshi/core.py @ 347:ffa7dea6e8fd experimental-inline
cspeedups branch: Merged [423:426/trunk].
author | cmlenz |
---|---|
date | Fri, 10 Nov 2006 17:38:50 +0000 |
parents | f999da894391 |
children | 9aa6aa18fa35 a81675590258 |
rev | line source |
---|---|
1 | 1 # -*- coding: utf-8 -*- |
2 # | |
66
59eb24184e9c
Switch copyright to Edgewall and URLs to markup.edgewall.org.
cmlenz
parents:
54
diff
changeset
|
3 # Copyright (C) 2006 Edgewall Software |
1 | 4 # All rights reserved. |
5 # | |
6 # This software is licensed as described in the file COPYING, which | |
7 # you should have received as part of this distribution. The terms | |
230 | 8 # are also available at http://genshi.edgewall.org/wiki/License. |
1 | 9 # |
10 # This software consists of voluntary contributions made by many | |
11 # individuals. For the exact contribution history, see the revision | |
230 | 12 # history and logs, available at http://genshi.edgewall.org/log/. |
1 | 13 |
14 """Core classes for markup processing.""" | |
15 | |
16 import htmlentitydefs | |
204
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
17 import operator |
1 | 18 import re |
19 | |
20 __all__ = ['Stream', 'Markup', 'escape', 'unescape', 'Namespace', 'QName'] | |
21 | |
22 | |
17
74cc70129d04
Refactoring to address #6: all match templates are now processed by a single filter, which means that match templates added by included templates are properly applied. A side effect of this refactoring is that `Context` objects may not be reused across multiple template processing runs.
cmlenz
parents:
10
diff
changeset
|
23 class StreamEventKind(str): |
1 | 24 """A kind of event on an XML stream.""" |
279
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
25 __slots__ = [] |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
26 _instances = {} |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
27 |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
28 def __new__(cls, val): |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
29 return cls._instances.setdefault(val, str.__new__(cls, val)) |
1 | 30 |
31 | |
32 class Stream(object): | |
33 """Represents a stream of markup events. | |
34 | |
35 This class is basically an iterator over the events. | |
36 | |
37 Also provided are ways to serialize the stream to text. The `serialize()` | |
38 method will return an iterator over generated strings, while `render()` | |
39 returns the complete generated text at once. Both accept various parameters | |
40 that impact the way the stream is serialized. | |
41 | |
42 Stream events are tuples of the form: | |
43 | |
44 (kind, data, position) | |
45 | |
46 where `kind` is the event kind (such as `START`, `END`, `TEXT`, etc), `data` | |
172
ff4f0d89eef7
Fix for #30 (trouble using `py:def`inside a match template)
cmlenz
parents:
171
diff
changeset
|
47 depends on the kind of event, and `position` is a `(filename, line, offset)` |
ff4f0d89eef7
Fix for #30 (trouble using `py:def`inside a match template)
cmlenz
parents:
171
diff
changeset
|
48 tuple that contains the location of the original element or text in the |
ff4f0d89eef7
Fix for #30 (trouble using `py:def`inside a match template)
cmlenz
parents:
171
diff
changeset
|
49 input. If the original location is unknown, `position` is `(None, -1, -1)`. |
1 | 50 """ |
51 __slots__ = ['events'] | |
52 | |
17
74cc70129d04
Refactoring to address #6: all match templates are now processed by a single filter, which means that match templates added by included templates are properly applied. A side effect of this refactoring is that `Context` objects may not be reused across multiple template processing runs.
cmlenz
parents:
10
diff
changeset
|
53 START = StreamEventKind('START') # a start tag |
74cc70129d04
Refactoring to address #6: all match templates are now processed by a single filter, which means that match templates added by included templates are properly applied. A side effect of this refactoring is that `Context` objects may not be reused across multiple template processing runs.
cmlenz
parents:
10
diff
changeset
|
54 END = StreamEventKind('END') # an end tag |
74cc70129d04
Refactoring to address #6: all match templates are now processed by a single filter, which means that match templates added by included templates are properly applied. A side effect of this refactoring is that `Context` objects may not be reused across multiple template processing runs.
cmlenz
parents:
10
diff
changeset
|
55 TEXT = StreamEventKind('TEXT') # literal text |
74cc70129d04
Refactoring to address #6: all match templates are now processed by a single filter, which means that match templates added by included templates are properly applied. A side effect of this refactoring is that `Context` objects may not be reused across multiple template processing runs.
cmlenz
parents:
10
diff
changeset
|
56 DOCTYPE = StreamEventKind('DOCTYPE') # doctype declaration |
143
3d4c214c979a
CDATA sections in XML input now appear as CDATA sections in the output. This should address the problem with escaping the contents of `<style>` and `<script>` elements, which would only get interpreted correctly if the output was served as `application/xhtml+xml`. Closes #24.
cmlenz
parents:
141
diff
changeset
|
57 START_NS = StreamEventKind('START_NS') # start namespace mapping |
3d4c214c979a
CDATA sections in XML input now appear as CDATA sections in the output. This should address the problem with escaping the contents of `<style>` and `<script>` elements, which would only get interpreted correctly if the output was served as `application/xhtml+xml`. Closes #24.
cmlenz
parents:
141
diff
changeset
|
58 END_NS = StreamEventKind('END_NS') # end namespace mapping |
3d4c214c979a
CDATA sections in XML input now appear as CDATA sections in the output. This should address the problem with escaping the contents of `<style>` and `<script>` elements, which would only get interpreted correctly if the output was served as `application/xhtml+xml`. Closes #24.
cmlenz
parents:
141
diff
changeset
|
59 START_CDATA = StreamEventKind('START_CDATA') # start CDATA section |
3d4c214c979a
CDATA sections in XML input now appear as CDATA sections in the output. This should address the problem with escaping the contents of `<style>` and `<script>` elements, which would only get interpreted correctly if the output was served as `application/xhtml+xml`. Closes #24.
cmlenz
parents:
141
diff
changeset
|
60 END_CDATA = StreamEventKind('END_CDATA') # end CDATA section |
17
74cc70129d04
Refactoring to address #6: all match templates are now processed by a single filter, which means that match templates added by included templates are properly applied. A side effect of this refactoring is that `Context` objects may not be reused across multiple template processing runs.
cmlenz
parents:
10
diff
changeset
|
61 PI = StreamEventKind('PI') # processing instruction |
74cc70129d04
Refactoring to address #6: all match templates are now processed by a single filter, which means that match templates added by included templates are properly applied. A side effect of this refactoring is that `Context` objects may not be reused across multiple template processing runs.
cmlenz
parents:
10
diff
changeset
|
62 COMMENT = StreamEventKind('COMMENT') # comment |
1 | 63 |
64 def __init__(self, events): | |
65 """Initialize the stream with a sequence of markup events. | |
66 | |
27 | 67 @param events: a sequence or iterable providing the events |
1 | 68 """ |
69 self.events = events | |
70 | |
71 def __iter__(self): | |
72 return iter(self.events) | |
73 | |
204
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
74 def __or__(self, function): |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
75 """Override the "bitwise or" operator to apply filters or serializers |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
76 to the stream, providing a syntax similar to pipes on Unix shells. |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
77 |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
78 Assume the following stream produced by the `HTML` function: |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
79 |
230 | 80 >>> from genshi.input import HTML |
204
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
81 >>> html = HTML('''<p onclick="alert('Whoa')">Hello, world!</p>''') |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
82 >>> print html |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
83 <p onclick="alert('Whoa')">Hello, world!</p> |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
84 |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
85 A filter such as the HTML sanitizer can be applied to that stream using |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
86 the pipe notation as follows: |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
87 |
230 | 88 >>> from genshi.filters import HTMLSanitizer |
204
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
89 >>> sanitizer = HTMLSanitizer() |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
90 >>> print html | sanitizer |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
91 <p>Hello, world!</p> |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
92 |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
93 Filters can be any function that accepts and produces a stream (where |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
94 a stream is anything that iterators over events): |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
95 |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
96 >>> def uppercase(stream): |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
97 ... for kind, data, pos in stream: |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
98 ... if kind is TEXT: |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
99 ... data = data.upper() |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
100 ... yield kind, data, pos |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
101 >>> print html | sanitizer | uppercase |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
102 <p>HELLO, WORLD!</p> |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
103 |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
104 Serializers can also be used with this notation: |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
105 |
230 | 106 >>> from genshi.output import TextSerializer |
204
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
107 >>> output = TextSerializer() |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
108 >>> print html | sanitizer | uppercase | output |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
109 HELLO, WORLD! |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
110 |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
111 Commonly, serializers should be used at the end of the "pipeline"; |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
112 using them somewhere in the middle may produce unexpected results. |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
113 """ |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
114 return Stream(_ensure(function(self))) |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
115 |
123
10279d2eeec9
Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents:
116
diff
changeset
|
116 def filter(self, *filters): |
10279d2eeec9
Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents:
116
diff
changeset
|
117 """Apply filters to the stream. |
113
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
118 |
123
10279d2eeec9
Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents:
116
diff
changeset
|
119 This method returns a new stream with the given filters applied. The |
10279d2eeec9
Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents:
116
diff
changeset
|
120 filters must be callables that accept the stream object as parameter, |
10279d2eeec9
Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents:
116
diff
changeset
|
121 and return the filtered stream. |
204
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
122 |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
123 The call: |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
124 |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
125 stream.filter(filter1, filter2) |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
126 |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
127 is equivalent to: |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
128 |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
129 stream | filter1 | filter2 |
113
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
130 """ |
204
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
131 return reduce(operator.or_, (self,) + filters) |
113
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
132 |
123
10279d2eeec9
Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents:
116
diff
changeset
|
133 def render(self, method='xml', encoding='utf-8', **kwargs): |
1 | 134 """Return a string representation of the stream. |
135 | |
136 @param method: determines how the stream is serialized; can be either | |
200
5861f4446c26
Add serialization to plain text, based on cboos' patch. Closes #41.
cmlenz
parents:
182
diff
changeset
|
137 "xml", "xhtml", "html", "text", or a custom serializer |
5861f4446c26
Add serialization to plain text, based on cboos' patch. Closes #41.
cmlenz
parents:
182
diff
changeset
|
138 class |
1 | 139 @param encoding: how the output string should be encoded; if set to |
140 `None`, this method returns a `unicode` object | |
141 | |
142 Any additional keyword arguments are passed to the serializer, and thus | |
143 depend on the `method` parameter value. | |
144 """ | |
123
10279d2eeec9
Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents:
116
diff
changeset
|
145 generator = self.serialize(method=method, **kwargs) |
17
74cc70129d04
Refactoring to address #6: all match templates are now processed by a single filter, which means that match templates added by included templates are properly applied. A side effect of this refactoring is that `Context` objects may not be reused across multiple template processing runs.
cmlenz
parents:
10
diff
changeset
|
146 output = u''.join(list(generator)) |
1 | 147 if encoding is not None: |
200
5861f4446c26
Add serialization to plain text, based on cboos' patch. Closes #41.
cmlenz
parents:
182
diff
changeset
|
148 errors = 'replace' |
5861f4446c26
Add serialization to plain text, based on cboos' patch. Closes #41.
cmlenz
parents:
182
diff
changeset
|
149 if method != 'text': |
5861f4446c26
Add serialization to plain text, based on cboos' patch. Closes #41.
cmlenz
parents:
182
diff
changeset
|
150 errors = 'xmlcharrefreplace' |
5861f4446c26
Add serialization to plain text, based on cboos' patch. Closes #41.
cmlenz
parents:
182
diff
changeset
|
151 return output.encode(encoding, errors) |
8
3710e3d0d4a2
`Stream.render()` was masking `TypeError`s (fix based on suggestion by Matt Good).
cmlenz
parents:
6
diff
changeset
|
152 return output |
1 | 153 |
278
d6c58473a9d0
Fix the handling of namespace context for match templates.
cmlenz
parents:
230
diff
changeset
|
154 def select(self, path, namespaces=None, variables=None): |
1 | 155 """Return a new stream that contains the events matching the given |
156 XPath expression. | |
157 | |
158 @param path: a string containing the XPath expression | |
159 """ | |
230 | 160 from genshi.path import Path |
278
d6c58473a9d0
Fix the handling of namespace context for match templates.
cmlenz
parents:
230
diff
changeset
|
161 return Path(path).select(self, namespaces, variables) |
1 | 162 |
123
10279d2eeec9
Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents:
116
diff
changeset
|
163 def serialize(self, method='xml', **kwargs): |
1 | 164 """Generate strings corresponding to a specific serialization of the |
165 stream. | |
166 | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
167 Unlike the `render()` method, this method is a generator that returns |
1 | 168 the serialized output incrementally, as opposed to returning a single |
169 string. | |
170 | |
171 @param method: determines how the stream is serialized; can be either | |
200
5861f4446c26
Add serialization to plain text, based on cboos' patch. Closes #41.
cmlenz
parents:
182
diff
changeset
|
172 "xml", "xhtml", "html", "text", or a custom serializer |
5861f4446c26
Add serialization to plain text, based on cboos' patch. Closes #41.
cmlenz
parents:
182
diff
changeset
|
173 class |
147
a4a0ca41b6ad
Use `xmlcharrefreplace` when encoding the output in `Stream.render()`, so that encoding the output to legacy encodings such as ASCII or ISO-8859-1 should always work.
cmlenz
parents:
145
diff
changeset
|
174 |
a4a0ca41b6ad
Use `xmlcharrefreplace` when encoding the output in `Stream.render()`, so that encoding the output to legacy encodings such as ASCII or ISO-8859-1 should always work.
cmlenz
parents:
145
diff
changeset
|
175 Any additional keyword arguments are passed to the serializer, and thus |
a4a0ca41b6ad
Use `xmlcharrefreplace` when encoding the output in `Stream.render()`, so that encoding the output to legacy encodings such as ASCII or ISO-8859-1 should always work.
cmlenz
parents:
145
diff
changeset
|
176 depend on the `method` parameter value. |
1 | 177 """ |
230 | 178 from genshi import output |
1 | 179 cls = method |
180 if isinstance(method, basestring): | |
96
fa08aef181a2
Add an XHTML serialization method. Now really need to get rid of some code duplication in the `markup.output` module.
cmlenz
parents:
91
diff
changeset
|
181 cls = {'xml': output.XMLSerializer, |
fa08aef181a2
Add an XHTML serialization method. Now really need to get rid of some code duplication in the `markup.output` module.
cmlenz
parents:
91
diff
changeset
|
182 'xhtml': output.XHTMLSerializer, |
200
5861f4446c26
Add serialization to plain text, based on cboos' patch. Closes #41.
cmlenz
parents:
182
diff
changeset
|
183 'html': output.HTMLSerializer, |
5861f4446c26
Add serialization to plain text, based on cboos' patch. Closes #41.
cmlenz
parents:
182
diff
changeset
|
184 'text': output.TextSerializer}[method] |
204
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
185 return cls(**kwargs)(_ensure(self)) |
1 | 186 |
187 def __str__(self): | |
188 return self.render() | |
189 | |
190 def __unicode__(self): | |
191 return self.render(encoding=None) | |
192 | |
193 | |
69 | 194 START = Stream.START |
195 END = Stream.END | |
196 TEXT = Stream.TEXT | |
197 DOCTYPE = Stream.DOCTYPE | |
198 START_NS = Stream.START_NS | |
199 END_NS = Stream.END_NS | |
143
3d4c214c979a
CDATA sections in XML input now appear as CDATA sections in the output. This should address the problem with escaping the contents of `<style>` and `<script>` elements, which would only get interpreted correctly if the output was served as `application/xhtml+xml`. Closes #24.
cmlenz
parents:
141
diff
changeset
|
200 START_CDATA = Stream.START_CDATA |
3d4c214c979a
CDATA sections in XML input now appear as CDATA sections in the output. This should address the problem with escaping the contents of `<style>` and `<script>` elements, which would only get interpreted correctly if the output was served as `application/xhtml+xml`. Closes #24.
cmlenz
parents:
141
diff
changeset
|
201 END_CDATA = Stream.END_CDATA |
69 | 202 PI = Stream.PI |
203 COMMENT = Stream.COMMENT | |
204 | |
111
2368c3becc52
Some fixes and more unit tests for the XPath engine.
cmlenz
parents:
100
diff
changeset
|
205 def _ensure(stream): |
2368c3becc52
Some fixes and more unit tests for the XPath engine.
cmlenz
parents:
100
diff
changeset
|
206 """Ensure that every item on the stream is actually a markup event.""" |
2368c3becc52
Some fixes and more unit tests for the XPath engine.
cmlenz
parents:
100
diff
changeset
|
207 for event in stream: |
145
47bbd9d2a5af
* Fix error in expression evaluation when the expression evaluates to an iterable that does not produce event tuples.
cmlenz
parents:
143
diff
changeset
|
208 if type(event) is not tuple: |
47bbd9d2a5af
* Fix error in expression evaluation when the expression evaluates to an iterable that does not produce event tuples.
cmlenz
parents:
143
diff
changeset
|
209 if hasattr(event, 'totuple'): |
47bbd9d2a5af
* Fix error in expression evaluation when the expression evaluates to an iterable that does not produce event tuples.
cmlenz
parents:
143
diff
changeset
|
210 event = event.totuple() |
47bbd9d2a5af
* Fix error in expression evaluation when the expression evaluates to an iterable that does not produce event tuples.
cmlenz
parents:
143
diff
changeset
|
211 else: |
47bbd9d2a5af
* Fix error in expression evaluation when the expression evaluates to an iterable that does not produce event tuples.
cmlenz
parents:
143
diff
changeset
|
212 event = TEXT, unicode(event), (None, -1, -1) |
47bbd9d2a5af
* Fix error in expression evaluation when the expression evaluates to an iterable that does not produce event tuples.
cmlenz
parents:
143
diff
changeset
|
213 yield event |
111
2368c3becc52
Some fixes and more unit tests for the XPath engine.
cmlenz
parents:
100
diff
changeset
|
214 |
69 | 215 |
347 | 216 class Attrs(tuple): |
217 """Immutable sequence type that stores the attributes of an element. | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
218 |
347 | 219 Ordering of the attributes is preserved, while accessing by name is also |
220 supported. | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
221 |
182
2f30ce3fb85e
Renamed `Attributes` to `Attrs` to reduce the verbosity.
cmlenz
parents:
172
diff
changeset
|
222 >>> attrs = Attrs([('href', '#'), ('title', 'Foo')]) |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
223 >>> attrs |
326
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
224 Attrs([(QName(u'href'), '#'), (QName(u'title'), 'Foo')]) |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
225 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
226 >>> 'href' in attrs |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
227 True |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
228 >>> 'tabindex' in attrs |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
229 False |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
230 >>> attrs.get(u'title') |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
231 'Foo' |
347 | 232 |
233 Instances may not be manipulated directly. Instead, the operators `|` and | |
234 `-` can be used to produce new instances that have specific attributes | |
235 added, replaced or removed. | |
236 | |
237 To remove an attribute, use the `-` operator. The right hand side can be | |
238 either a string or a set/sequence of strings, identifying the name(s) of | |
239 the attribute(s) to remove: | |
240 | |
241 >>> attrs - 'title' | |
242 Attrs([(QName(u'href'), '#')]) | |
243 >>> attrs - ('title', 'href') | |
244 Attrs() | |
245 | |
246 The original instance is not modified, but the operator can of course be | |
247 used with an assignment: | |
248 | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
249 >>> attrs |
347 | 250 Attrs([(QName(u'href'), '#'), (QName(u'title'), 'Foo')]) |
251 >>> attrs -= 'title' | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
252 >>> attrs |
326
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
253 Attrs([(QName(u'href'), '#')]) |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
254 |
347 | 255 To add a new attribute, use the `|` operator, where the right hand value |
256 is a sequence of `(name, value)` tuples (which includes `Attrs` instances): | |
170
6b265e02d099
Allow initialization of `Attributes` with keyword arguments.
cmlenz
parents:
161
diff
changeset
|
257 |
347 | 258 >>> attrs | [(u'title', 'Bar')] |
259 Attrs([(QName(u'href'), '#'), (QName(u'title'), 'Bar')]) | |
171
7fcf8e04514e
Follow-up to [214]: allow initializing `Attributes` with attribute names that contain dashes or conflict with a reserved word (such as ?class?.)
cmlenz
parents:
170
diff
changeset
|
260 |
347 | 261 If the attributes already contain an attribute with a given name, the value |
262 of that attribute is replaced: | |
171
7fcf8e04514e
Follow-up to [214]: allow initializing `Attributes` with attribute names that contain dashes or conflict with a reserved word (such as ?class?.)
cmlenz
parents:
170
diff
changeset
|
263 |
347 | 264 >>> attrs | [(u'href', 'http://example.org/')] |
265 Attrs([(QName(u'href'), 'http://example.org/')]) | |
171
7fcf8e04514e
Follow-up to [214]: allow initializing `Attributes` with attribute names that contain dashes or conflict with a reserved word (such as ?class?.)
cmlenz
parents:
170
diff
changeset
|
266 |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
267 """ |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
268 __slots__ = [] |
1 | 269 |
347 | 270 def __new__(cls, items=()): |
182
2f30ce3fb85e
Renamed `Attributes` to `Attrs` to reduce the verbosity.
cmlenz
parents:
172
diff
changeset
|
271 """Create the `Attrs` instance. |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
272 |
347 | 273 If the `items` parameter is provided, it is expected to be a sequence |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
274 of `(name, value)` tuples. |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
275 """ |
347 | 276 return tuple.__new__(cls, [(QName(name), val) for name, val in items]) |
1 | 277 |
278 def __contains__(self, name): | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
279 """Return whether the list includes an attribute with the specified |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
280 name. |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
281 """ |
133
79f445396cd7
Minor cleanup and performance improvement for the builder module.
cmlenz
parents:
123
diff
changeset
|
282 for attr, _ in self: |
79f445396cd7
Minor cleanup and performance improvement for the builder module.
cmlenz
parents:
123
diff
changeset
|
283 if attr == name: |
79f445396cd7
Minor cleanup and performance improvement for the builder module.
cmlenz
parents:
123
diff
changeset
|
284 return True |
1 | 285 |
347 | 286 def __getslice__(self, i, j): |
287 return Attrs(tuple.__getslice__(self, i, j)) | |
288 | |
289 def __or__(self, attrs): | |
290 """Return a new instance that contains the attributes in `attrs` in | |
291 addition to any already existing attributes. | |
292 """ | |
293 repl = dict([(an, av) for an, av in attrs if an in self]) | |
294 return Attrs([(sn, repl.get(sn, sv)) for sn, sv in self] + | |
295 [(an, av) for an, av in attrs if an not in self]) | |
296 | |
326
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
297 def __repr__(self): |
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
298 if not self: |
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
299 return 'Attrs()' |
347 | 300 return 'Attrs([%s])' % ', '.join([repr(item) for item in self]) |
301 | |
302 def __sub__(self, names): | |
303 """Return a new instance with all attributes with a name in `names` are | |
304 removed. | |
305 """ | |
306 if isinstance(names, basestring): | |
307 names = (names,) | |
308 return Attrs([(name, val) for name, val in self if name not in names]) | |
326
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
309 |
1 | 310 def get(self, name, default=None): |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
311 """Return the value of the attribute with the specified name, or the |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
312 value of the `default` parameter if no such attribute is found. |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
313 """ |
1 | 314 for attr, value in self: |
315 if attr == name: | |
316 return value | |
317 return default | |
318 | |
77
f5ec6d4a61e4
* Simplify implementation of the individual XPath tests (use closures instead of callable classes)
cmlenz
parents:
73
diff
changeset
|
319 def totuple(self): |
161
7b1f07496bf7
Various docstring additions and other cosmetic changes.
cmlenz
parents:
147
diff
changeset
|
320 """Return the attributes as a markup event. |
7b1f07496bf7
Various docstring additions and other cosmetic changes.
cmlenz
parents:
147
diff
changeset
|
321 |
7b1f07496bf7
Various docstring additions and other cosmetic changes.
cmlenz
parents:
147
diff
changeset
|
322 The returned event is a TEXT event, the data is the value of all |
7b1f07496bf7
Various docstring additions and other cosmetic changes.
cmlenz
parents:
147
diff
changeset
|
323 attributes joined together. |
7b1f07496bf7
Various docstring additions and other cosmetic changes.
cmlenz
parents:
147
diff
changeset
|
324 """ |
77
f5ec6d4a61e4
* Simplify implementation of the individual XPath tests (use closures instead of callable classes)
cmlenz
parents:
73
diff
changeset
|
325 return TEXT, u''.join([x[1] for x in self]), (None, -1, -1) |
f5ec6d4a61e4
* Simplify implementation of the individual XPath tests (use closures instead of callable classes)
cmlenz
parents:
73
diff
changeset
|
326 |
1 | 327 |
116 | 328 def plaintext(text, keeplinebreaks=True): |
329 """Returns the text as a `unicode` string with all entities and tags | |
330 removed. | |
331 """ | |
332 text = stripentities(striptags(text)) | |
333 if not keeplinebreaks: | |
334 text = text.replace(u'\n', u' ') | |
335 return text | |
336 | |
113
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
337 def stripentities(text, keepxmlentities=False): |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
338 """Return a copy of the given text with any character or numeric entities |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
339 replaced by the equivalent UTF-8 characters. |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
340 |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
341 If the `keepxmlentities` parameter is provided and evaluates to `True`, |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
342 the core XML entities (&, ', >, < and ") are not |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
343 stripped. |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
344 """ |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
345 def _replace_entity(match): |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
346 if match.group(1): # numeric entity |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
347 ref = match.group(1) |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
348 if ref.startswith('x'): |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
349 ref = int(ref[1:], 16) |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
350 else: |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
351 ref = int(ref, 10) |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
352 return unichr(ref) |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
353 else: # character entity |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
354 ref = match.group(2) |
200
5861f4446c26
Add serialization to plain text, based on cboos' patch. Closes #41.
cmlenz
parents:
182
diff
changeset
|
355 if keepxmlentities and ref in ('amp', 'apos', 'gt', 'lt', 'quot'): |
113
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
356 return '&%s;' % ref |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
357 try: |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
358 codepoint = htmlentitydefs.name2codepoint[ref] |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
359 return unichr(codepoint) |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
360 except KeyError: |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
361 if keepxmlentities: |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
362 return '&%s;' % ref |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
363 else: |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
364 return ref |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
365 return re.sub(r'&(?:#((?:\d+)|(?:[xX][0-9a-fA-F]+));?|(\w+);)', |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
366 _replace_entity, text) |
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
367 |
116 | 368 def striptags(text): |
369 """Return a copy of the text with all XML/HTML tags removed.""" | |
370 return re.sub(r'<[^>]*?>', '', text) | |
371 | |
113
d10fbba1d5e0
Removed the `sanitize()` method from the `Markup` class, and migrate the existing unit tests to `markup.tests.filters`. Provide a `Stream.filter()` method instead which can be used to conveniently apply a filter to a stream.
cmlenz
parents:
111
diff
changeset
|
372 |
1 | 373 class Markup(unicode): |
374 """Marks a string as being safe for inclusion in HTML/XML output without | |
375 needing to be escaped. | |
376 """ | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
377 __slots__ = [] |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
378 |
27 | 379 def __new__(cls, text='', *args): |
1 | 380 if args: |
136 | 381 text %= tuple(map(escape, args)) |
27 | 382 return unicode.__new__(cls, text) |
1 | 383 |
384 def __add__(self, other): | |
204
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
385 return Markup(unicode(self) + unicode(escape(other))) |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
386 |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
387 def __radd__(self, other): |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
388 return Markup(unicode(escape(other)) + unicode(self)) |
1 | 389 |
390 def __mod__(self, args): | |
391 if not isinstance(args, (list, tuple)): | |
392 args = [args] | |
136 | 393 return Markup(unicode.__mod__(self, tuple(map(escape, args)))) |
1 | 394 |
395 def __mul__(self, num): | |
396 return Markup(unicode(self) * num) | |
397 | |
204
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
398 def __rmul__(self, num): |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
399 return Markup(num * unicode(self)) |
51d4101f49ca
* Implement reverse add/mul operators for `Markup` class, so that the result is also a `Markup` instance.
cmlenz
parents:
200
diff
changeset
|
400 |
17
74cc70129d04
Refactoring to address #6: all match templates are now processed by a single filter, which means that match templates added by included templates are properly applied. A side effect of this refactoring is that `Context` objects may not be reused across multiple template processing runs.
cmlenz
parents:
10
diff
changeset
|
401 def __repr__(self): |
74cc70129d04
Refactoring to address #6: all match templates are now processed by a single filter, which means that match templates added by included templates are properly applied. A side effect of this refactoring is that `Context` objects may not be reused across multiple template processing runs.
cmlenz
parents:
10
diff
changeset
|
402 return '<%s "%s">' % (self.__class__.__name__, self) |
74cc70129d04
Refactoring to address #6: all match templates are now processed by a single filter, which means that match templates added by included templates are properly applied. A side effect of this refactoring is that `Context` objects may not be reused across multiple template processing runs.
cmlenz
parents:
10
diff
changeset
|
403 |
54 | 404 def join(self, seq, escape_quotes=True): |
405 return Markup(unicode(self).join([escape(item, quotes=escape_quotes) | |
34 | 406 for item in seq])) |
1 | 407 |
408 def escape(cls, text, quotes=True): | |
409 """Create a Markup instance from a string and escape special characters | |
410 it may contain (<, >, & and \"). | |
411 | |
412 If the `quotes` parameter is set to `False`, the \" character is left | |
413 as is. Escaping quotes is generally only required for strings that are | |
414 to be used in attribute values. | |
415 """ | |
73 | 416 if not text: |
417 return cls() | |
418 if type(text) is cls: | |
1 | 419 return text |
73 | 420 text = unicode(text).replace('&', '&') \ |
421 .replace('<', '<') \ | |
422 .replace('>', '>') | |
1 | 423 if quotes: |
424 text = text.replace('"', '"') | |
425 return cls(text) | |
426 escape = classmethod(escape) | |
427 | |
428 def unescape(self): | |
429 """Reverse-escapes &, <, > and \" and returns a `unicode` object.""" | |
430 if not self: | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
431 return u'' |
1 | 432 return unicode(self).replace('"', '"') \ |
433 .replace('>', '>') \ | |
434 .replace('<', '<') \ | |
435 .replace('&', '&') | |
436 | |
116 | 437 def stripentities(self, keepxmlentities=False): |
438 """Return a copy of the text with any character or numeric entities | |
439 replaced by the equivalent UTF-8 characters. | |
440 | |
441 If the `keepxmlentities` parameter is provided and evaluates to `True`, | |
442 the core XML entities (&, ', >, < and ") are not | |
443 stripped. | |
6 | 444 """ |
116 | 445 return Markup(stripentities(self, keepxmlentities=keepxmlentities)) |
446 | |
447 def striptags(self): | |
448 """Return a copy of the text with all XML/HTML tags removed.""" | |
449 return Markup(striptags(self)) | |
1 | 450 |
451 | |
452 escape = Markup.escape | |
453 | |
454 def unescape(text): | |
455 """Reverse-escapes &, <, > and \" and returns a `unicode` object.""" | |
456 if not isinstance(text, Markup): | |
457 return text | |
458 return text.unescape() | |
459 | |
460 | |
461 class Namespace(object): | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
462 """Utility class creating and testing elements with a namespace. |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
463 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
464 Internally, namespace URIs are encoded in the `QName` of any element or |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
465 attribute, the namespace URI being enclosed in curly braces. This class |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
466 helps create and test these strings. |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
467 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
468 A `Namespace` object is instantiated with the namespace URI. |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
469 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
470 >>> html = Namespace('http://www.w3.org/1999/xhtml') |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
471 >>> html |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
472 <Namespace "http://www.w3.org/1999/xhtml"> |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
473 >>> html.uri |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
474 u'http://www.w3.org/1999/xhtml' |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
475 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
476 The `Namespace` object can than be used to generate `QName` objects with |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
477 that namespace: |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
478 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
479 >>> html.body |
326
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
480 QName(u'http://www.w3.org/1999/xhtml}body') |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
481 >>> html.body.localname |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
482 u'body' |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
483 >>> html.body.namespace |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
484 u'http://www.w3.org/1999/xhtml' |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
485 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
486 The same works using item access notation, which is useful for element or |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
487 attribute names that are not valid Python identifiers: |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
488 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
489 >>> html['body'] |
326
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
490 QName(u'http://www.w3.org/1999/xhtml}body') |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
491 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
492 A `Namespace` object can also be used to test whether a specific `QName` |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
493 belongs to that namespace using the `in` operator: |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
494 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
495 >>> qname = html.body |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
496 >>> qname in html |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
497 True |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
498 >>> qname in Namespace('http://www.w3.org/2002/06/xhtml2') |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
499 False |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
500 """ |
224
90d62225f411
Implement support for namespace prefixes in XPath expressions.
cmlenz
parents:
204
diff
changeset
|
501 def __new__(cls, uri): |
90d62225f411
Implement support for namespace prefixes in XPath expressions.
cmlenz
parents:
204
diff
changeset
|
502 if type(uri) is cls: |
90d62225f411
Implement support for namespace prefixes in XPath expressions.
cmlenz
parents:
204
diff
changeset
|
503 return uri |
90d62225f411
Implement support for namespace prefixes in XPath expressions.
cmlenz
parents:
204
diff
changeset
|
504 return object.__new__(cls, uri) |
90d62225f411
Implement support for namespace prefixes in XPath expressions.
cmlenz
parents:
204
diff
changeset
|
505 |
279
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
506 def __getnewargs__(self): |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
507 return (self.uri,) |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
508 |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
509 def __getstate__(self): |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
510 return self.uri |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
511 |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
512 def __setstate__(self, uri): |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
513 self.uri = uri |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
514 |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
515 def __init__(self, uri): |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
516 self.uri = unicode(uri) |
1 | 517 |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
518 def __contains__(self, qname): |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
519 return qname.namespace == self.uri |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
520 |
278
d6c58473a9d0
Fix the handling of namespace context for match templates.
cmlenz
parents:
230
diff
changeset
|
521 def __ne__(self, other): |
d6c58473a9d0
Fix the handling of namespace context for match templates.
cmlenz
parents:
230
diff
changeset
|
522 return not self == other |
d6c58473a9d0
Fix the handling of namespace context for match templates.
cmlenz
parents:
230
diff
changeset
|
523 |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
524 def __eq__(self, other): |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
525 if isinstance(other, Namespace): |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
526 return self.uri == other.uri |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
527 return self.uri == other |
1 | 528 |
529 def __getitem__(self, name): | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
530 return QName(self.uri + u'}' + name) |
1 | 531 __getattr__ = __getitem__ |
532 | |
533 def __repr__(self): | |
534 return '<Namespace "%s">' % self.uri | |
535 | |
536 def __str__(self): | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
537 return self.uri.encode('utf-8') |
1 | 538 |
539 def __unicode__(self): | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
540 return self.uri |
1 | 541 |
542 | |
147
a4a0ca41b6ad
Use `xmlcharrefreplace` when encoding the output in `Stream.render()`, so that encoding the output to legacy encodings such as ASCII or ISO-8859-1 should always work.
cmlenz
parents:
145
diff
changeset
|
543 # The namespace used by attributes such as xml:lang and xml:space |
141
520a5b7dd6d2
* No escaping of `<script>` or `<style>` tags in HTML output (see #24)
cmlenz
parents:
140
diff
changeset
|
544 XML_NAMESPACE = Namespace('http://www.w3.org/XML/1998/namespace') |
520a5b7dd6d2
* No escaping of `<script>` or `<style>` tags in HTML output (see #24)
cmlenz
parents:
140
diff
changeset
|
545 |
520a5b7dd6d2
* No escaping of `<script>` or `<style>` tags in HTML output (see #24)
cmlenz
parents:
140
diff
changeset
|
546 |
1 | 547 class QName(unicode): |
548 """A qualified element or attribute name. | |
549 | |
550 The unicode value of instances of this class contains the qualified name of | |
551 the element or attribute, in the form `{namespace}localname`. The namespace | |
552 URI can be obtained through the additional `namespace` attribute, while the | |
553 local name can be accessed through the `localname` attribute. | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
554 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
555 >>> qname = QName('foo') |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
556 >>> qname |
326
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
557 QName(u'foo') |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
558 >>> qname.localname |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
559 u'foo' |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
560 >>> qname.namespace |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
561 |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
562 >>> qname = QName('http://www.w3.org/1999/xhtml}body') |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
563 >>> qname |
326
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
564 QName(u'http://www.w3.org/1999/xhtml}body') |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
565 >>> qname.localname |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
566 u'body' |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
567 >>> qname.namespace |
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
568 u'http://www.w3.org/1999/xhtml' |
1 | 569 """ |
570 __slots__ = ['namespace', 'localname'] | |
571 | |
572 def __new__(cls, qname): | |
100 | 573 if type(qname) is cls: |
1 | 574 return qname |
575 | |
18
5420cfe42d36
Actually make use of the `markup.core.Namespace` class, and add a couple of doctests.
cmlenz
parents:
17
diff
changeset
|
576 parts = qname.split(u'}', 1) |
100 | 577 if len(parts) > 1: |
136 | 578 self = unicode.__new__(cls, u'{%s' % qname) |
579 self.namespace, self.localname = map(unicode, parts) | |
1 | 580 else: |
581 self = unicode.__new__(cls, qname) | |
136 | 582 self.namespace, self.localname = None, unicode(qname) |
1 | 583 return self |
279
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
584 |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
585 def __getnewargs__(self): |
a99666402b12
Some adjustments to make core data structures picklable (requires protocol 2).
cmlenz
parents:
278
diff
changeset
|
586 return (self.lstrip('{'),) |
326
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
587 |
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
588 def __repr__(self): |
f999da894391
Fixed `__repr__` of the `QName`, `Attrs`, and `Expression` classes so that the output can be used as code to instantiate the object again.
cmlenz
parents:
279
diff
changeset
|
589 return 'QName(%s)' % unicode.__repr__(self.lstrip('{')) |