|
226
|
1 .. -*- mode: rst; encoding: utf-8 -*-
|
|
|
2
|
|
|
3 ==============
|
|
|
4 Markup Streams
|
|
|
5 ==============
|
|
|
6
|
|
|
7 A stream is the common representation of markup as a *stream of events*.
|
|
|
8
|
|
|
9
|
|
|
10 .. contents:: Contents
|
|
|
11 :depth: 2
|
|
|
12 .. sectnum::
|
|
|
13
|
|
|
14
|
|
|
15 Basics
|
|
|
16 ======
|
|
|
17
|
|
|
18 A stream can be attained in a number of ways. It can be:
|
|
|
19
|
|
|
20 * the result of parsing XML or HTML text, or
|
|
|
21 * programmatically generated, or
|
|
|
22 * the result of selecting a subset of another stream filtered by an XPath
|
|
|
23 expression.
|
|
|
24
|
|
|
25 For example, the functions ``XML()`` and ``HTML()`` can be used to convert
|
|
|
26 literal XML or HTML text to a markup stream::
|
|
|
27
|
|
230
|
28 >>> from genshi import XML
|
|
226
|
29 >>> stream = XML('<p class="intro">Some text and '
|
|
|
30 ... '<a href="http://example.org/">a link</a>.'
|
|
|
31 ... '<br/></p>')
|
|
|
32 >>> stream
|
|
230
|
33 <genshi.core.Stream object at 0x6bef0>
|
|
226
|
34
|
|
|
35 The stream is the result of parsing the text into events. Each event is a tuple
|
|
|
36 of the form ``(kind, data, pos)``, where:
|
|
|
37
|
|
|
38 * ``kind`` defines what kind of event it is (such as the start of an element,
|
|
|
39 text, a comment, etc).
|
|
|
40 * ``data`` is the actual data associated with the event. How this looks depends
|
|
|
41 on the event kind.
|
|
|
42 * ``pos`` is a ``(filename, lineno, column)`` tuple that describes where the
|
|
|
43 event “comes from”.
|
|
|
44
|
|
|
45 ::
|
|
|
46
|
|
|
47 >>> for kind, data, pos in stream:
|
|
|
48 ... print kind, `data`, pos
|
|
|
49 ...
|
|
|
50 START (u'p', [(u'class', u'intro')]) ('<string>', 1, 0)
|
|
|
51 TEXT u'Some text and ' ('<string>', 1, 31)
|
|
|
52 START (u'a', [(u'href', u'http://example.org/')]) ('<string>', 1, 31)
|
|
|
53 TEXT u'a link' ('<string>', 1, 67)
|
|
|
54 END u'a' ('<string>', 1, 67)
|
|
|
55 TEXT u'.' ('<string>', 1, 72)
|
|
|
56 START (u'br', []) ('<string>', 1, 72)
|
|
|
57 END u'br' ('<string>', 1, 77)
|
|
|
58 END u'p' ('<string>', 1, 77)
|
|
|
59
|
|
|
60
|
|
|
61 Filtering
|
|
|
62 =========
|
|
|
63
|
|
|
64 One important feature of markup streams is that you can apply *filters* to the
|
|
230
|
65 stream, either filters that come with Genshi, or your own custom filters.
|
|
226
|
66
|
|
|
67 A filter is simply a callable that accepts the stream as parameter, and returns
|
|
|
68 the filtered stream::
|
|
|
69
|
|
|
70 def noop(stream):
|
|
|
71 """A filter that doesn't actually do anything with the stream."""
|
|
|
72 for kind, data, pos in stream:
|
|
|
73 yield kind, data, pos
|
|
|
74
|
|
|
75 Filters can be applied in a number of ways. The simplest is to just call the
|
|
|
76 filter directly::
|
|
|
77
|
|
|
78 stream = noop(stream)
|
|
|
79
|
|
|
80 The ``Stream`` class also provides a ``filter()`` method, which takes an
|
|
|
81 arbitrary number of filter callables and applies them all::
|
|
|
82
|
|
|
83 stream = stream.filter(noop)
|
|
|
84
|
|
|
85 Finally, filters can also be applied using the *bitwise or* operator (``|``),
|
|
|
86 which allows a syntax similar to pipes on Unix shells::
|
|
|
87
|
|
|
88 stream = stream | noop
|
|
|
89
|
|
230
|
90 One example of a filter included with Genshi is the ``HTMLSanitizer`` in
|
|
|
91 ``genshi.filters``. It processes a stream of HTML markup, and strips out any
|
|
226
|
92 potentially dangerous constructs, such as Javascript event handlers.
|
|
|
93 ``HTMLSanitizer`` is not a function, but rather a class that implements
|
|
|
94 ``__call__``, which means instances of the class are callable.
|
|
|
95
|
|
|
96 Both the ``filter()`` method and the pipe operator allow easy chaining of
|
|
|
97 filters::
|
|
|
98
|
|
230
|
99 from genshi.filters import HTMLSanitizer
|
|
226
|
100 stream = stream.filter(noop, HTMLSanitizer())
|
|
|
101
|
|
|
102 That is equivalent to::
|
|
|
103
|
|
|
104 stream = stream | noop | HTMLSanitizer()
|
|
|
105
|
|
|
106
|
|
|
107 Serialization
|
|
|
108 =============
|
|
|
109
|
|
|
110 The ``Stream`` class provides two methods for serializing this list of events:
|
|
|
111 ``serialize()`` and ``render()``. The former is a generator that yields chunks
|
|
230
|
112 of ``Markup`` objects (which are basically unicode strings that are considered
|
|
|
113 safe for output on the web). The latter returns a single string, by default
|
|
|
114 UTF-8 encoded.
|
|
226
|
115
|
|
|
116 Here's the output from ``serialize()``::
|
|
|
117
|
|
|
118 >>> for output in stream.serialize():
|
|
|
119 ... print `output`
|
|
|
120 ...
|
|
|
121 <Markup u'<p class="intro">'>
|
|
|
122 <Markup u'Some text and '>
|
|
|
123 <Markup u'<a href="http://example.org/">'>
|
|
|
124 <Markup u'a link'>
|
|
|
125 <Markup u'</a>'>
|
|
|
126 <Markup u'.'>
|
|
|
127 <Markup u'<br/>'>
|
|
|
128 <Markup u'</p>'>
|
|
|
129
|
|
|
130 And here's the output from ``render()``::
|
|
|
131
|
|
|
132 >>> print stream.render()
|
|
|
133 <p class="intro">Some text and <a href="http://example.org/">a link</a>.<br/></p>
|
|
|
134
|
|
|
135 Both methods can be passed a ``method`` parameter that determines how exactly
|
|
|
136 the events are serialzed to text. This parameter can be either “xml” (the
|
|
|
137 default), “xhtml”, “html”, “text”, or a custom serializer class::
|
|
|
138
|
|
|
139 >>> print stream.render('html')
|
|
|
140 <p class="intro">Some text and <a href="http://example.org/">a link</a>.<br></p>
|
|
|
141
|
|
|
142 Note how the `<br>` element isn't closed, which is the right thing to do for
|
|
|
143 HTML.
|
|
|
144
|
|
|
145 In addition, the ``render()`` method takes an ``encoding`` parameter, which
|
|
|
146 defaults to “UTF-8”. If set to ``None``, the result will be a unicode string.
|
|
|
147
|
|
230
|
148 The different serializer classes in ``genshi.output`` can also be used
|
|
226
|
149 directly::
|
|
|
150
|
|
230
|
151 >>> from genshi.filters import HTMLSanitizer
|
|
|
152 >>> from genshi.output import TextSerializer
|
|
226
|
153 >>> print TextSerializer()(HTMLSanitizer()(stream))
|
|
|
154 Some text and a link.
|
|
|
155
|
|
|
156 The pipe operator allows a nicer syntax::
|
|
|
157
|
|
|
158 >>> print stream | HTMLSanitizer() | TextSerializer()
|
|
|
159 Some text and a link.
|
|
|
160
|
|
|
161 Using XPath
|
|
|
162 ===========
|
|
|
163
|
|
|
164 XPath can be used to extract a specific subset of the stream via the
|
|
|
165 ``select()`` method::
|
|
|
166
|
|
|
167 >>> substream = stream.select('a')
|
|
|
168 >>> substream
|
|
230
|
169 <genshi.core.Stream object at 0x7118b0>
|
|
226
|
170 >>> print substream
|
|
|
171 <a href="http://example.org/">a link</a>
|
|
|
172
|
|
|
173 Often, streams cannot be reused: in the above example, the sub-stream is based
|
|
|
174 on a generator. Once it has been serialized, it will have been fully consumed,
|
|
|
175 and cannot be rendered again. To work around this, you can wrap such a stream
|
|
|
176 in a ``list``::
|
|
|
177
|
|
230
|
178 >>> from genshi import Stream
|
|
226
|
179 >>> substream = Stream(list(stream.select('a')))
|
|
|
180 >>> substream
|
|
230
|
181 <genshi.core.Stream object at 0x7118b0>
|
|
226
|
182 >>> print substream
|
|
|
183 <a href="http://example.org/">a link</a>
|
|
|
184 >>> print substream.select('@href')
|
|
|
185 http://example.org/
|
|
|
186 >>> print substream.select('text()')
|
|
|
187 a link
|
