Mercurial > genshi > mirror

annotate genshi/filters/html.py @ 576:b00765a115a5 trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Improve docs on `Stream.select()` for #135.
author cmlenz
date Mon, 23 Jul 2007 09:50:44 +0000
parents f0461dc3939a
children 94f719af686d
rev   line source
1
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
1 # -*- coding: utf-8 -*-
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
2 #
408
4675d5cf6c67 Update copyright year for files modified this year.
cmlenz
parents: 403
diff changeset
3 # Copyright (C) 2006-2007 Edgewall Software
1
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
4 # All rights reserved.
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
5 #
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
6 # This software is licensed as described in the file COPYING, which
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
7 # you should have received as part of this distribution. The terms
230
84168828b074 Renamed Markup to Genshi in repository.
cmlenz
parents: 182
diff changeset
8 # are also available at http://genshi.edgewall.org/wiki/License.
1
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
9 #
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
10 # This software consists of voluntary contributions made by many
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
11 # individuals. For the exact contribution history, see the revision
230
84168828b074 Renamed Markup to Genshi in repository.
cmlenz
parents: 182
diff changeset
12 # history and logs, available at http://genshi.edgewall.org/log/.
1
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
13
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
14 """Implementation of a number of stream filters."""
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
15
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
16 try:
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
17 frozenset
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
18 except NameError:
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
19 from sets import ImmutableSet as frozenset
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
20 import re
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
21
403
228907abb726 Remove some magic/overhead from `Attrs` creation and manipulation by not automatically wrapping attribute names in `QName`.
cmlenz
parents: 363
diff changeset
22 from genshi.core import Attrs, QName, stripentities
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
23 from genshi.core import END, START, TEXT, COMMENT
1
5479aae32f5a Initial import.
cmlenz
parents:
diff changeset
24
363
37e4b4bb0b53 Parse template includes at parse time to avoid some runtime overhead.
cmlenz
parents: 345
diff changeset
25 __all__ = ['HTMLFormFiller', 'HTMLSanitizer']
425
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
26 __docformat__ = 'restructuredtext en'
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
27
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
28
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
29 class HTMLFormFiller(object):
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
30 """A stream filter that can populate HTML forms from a dictionary of values.
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
31
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
32 >>> from genshi.input import HTML
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
33 >>> html = HTML('''<form>
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
34 ... <p><input type="text" name="foo" /></p>
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
35 ... </form>''')
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
36 >>> filler = HTMLFormFiller(data={'foo': 'bar'})
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
37 >>> print html | filler
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
38 <form>
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
39 <p><input type="text" name="foo" value="bar"/></p>
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
40 </form>
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
41 """
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
42 # TODO: only select the first radio button, and the first select option
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
43 # (if not in a multiple-select)
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
44 # TODO: only apply to elements in the XHTML namespace (or no namespace)?
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
45
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
46 def __init__(self, name=None, id=None, data=None):
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
47 """Create the filter.
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
48
425
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
49 :param name: The name of the form that should be populated. If this
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
50 parameter is given, only forms where the ``name`` attribute
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
51 value matches the parameter are processed.
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
52 :param id: The ID of the form that should be populated. If this
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
53 parameter is given, only forms where the ``id`` attribute
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
54 value matches the parameter are processed.
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
55 :param data: The dictionary of form values, where the keys are the names
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
56 of the form fields, and the values are the values to fill
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
57 in.
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
58 """
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
59 self.name = name
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
60 self.id = id
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
61 if data is None:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
62 data = {}
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
63 self.data = data
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
64
439
9f11c745fac9 Add support for adding custom template filters by passing a custom callback function to the `TemplateLoader`. Closes #89 (see added unit test).
cmlenz
parents: 431
diff changeset
65 def __call__(self, stream):
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
66 """Apply the filter to the given stream.
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
67
425
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
68 :param stream: the markup event stream to filter
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
69 """
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
70 in_form = in_select = in_option = in_textarea = False
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
71 select_value = option_value = textarea_value = None
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
72 option_start = option_text = None
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
73
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
74 for kind, data, pos in stream:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
75
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
76 if kind is START:
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
77 tag, attrs = data
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
78 tagname = tag.localname
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
79
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
80 if tagname == 'form' and (
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
81 self.name and attrs.get('name') == self.name or
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
82 self.id and attrs.get('id') == self.id or
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
83 not (self.id or self.name)):
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
84 in_form = True
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
85
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
86 elif in_form:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
87 if tagname == 'input':
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
88 type = attrs.get('type')
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
89 if type in ('checkbox', 'radio'):
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
90 name = attrs.get('name')
471
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
91 if name and name in self.data:
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
92 value = self.data[name]
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
93 declval = attrs.get('value')
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
94 checked = False
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
95 if isinstance(value, (list, tuple)):
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
96 if declval:
415
b9f9a22484f0 `HTMLFormFiller` now correctly deals with non-string values in the data dictionary for select/checkbox/radio controls.
cmlenz
parents: 408
diff changeset
97 checked = declval in [str(v) for v
b9f9a22484f0 `HTMLFormFiller` now correctly deals with non-string values in the data dictionary for select/checkbox/radio controls.
cmlenz
parents: 408
diff changeset
98 in value]
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
99 else:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
100 checked = bool(filter(None, value))
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
101 else:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
102 if declval:
415
b9f9a22484f0 `HTMLFormFiller` now correctly deals with non-string values in the data dictionary for select/checkbox/radio controls.
cmlenz
parents: 408
diff changeset
103 checked = declval == str(value)
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
104 elif type == 'checkbox':
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
105 checked = bool(value)
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
106 if checked:
403
228907abb726 Remove some magic/overhead from `Attrs` creation and manipulation by not automatically wrapping attribute names in `QName`.
cmlenz
parents: 363
diff changeset
107 attrs |= [(QName('checked'), 'checked')]
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
108 elif 'checked' in attrs:
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
109 attrs -= 'checked'
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
110 elif type in (None, 'hidden', 'text'):
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
111 name = attrs.get('name')
471
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
112 if name and name in self.data:
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
113 value = self.data[name]
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
114 if isinstance(value, (list, tuple)):
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
115 value = value[0]
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
116 if value is not None:
403
228907abb726 Remove some magic/overhead from `Attrs` creation and manipulation by not automatically wrapping attribute names in `QName`.
cmlenz
parents: 363
diff changeset
117 attrs |= [(QName('value'), unicode(value))]
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
118 elif tagname == 'select':
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
119 name = attrs.get('name')
471
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
120 if name in self.data:
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
121 select_value = self.data[name]
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
122 in_select = True
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
123 elif tagname == 'textarea':
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
124 name = attrs.get('name')
471
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
125 if name in self.data:
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
126 textarea_value = self.data.get(name)
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
127 if isinstance(textarea_value, (list, tuple)):
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
128 textarea_value = textarea_value[0]
76a0ec32835d The `HTMLFormFiller` stream filter no longer alters form elements for which the data element contains no corresponding item.
cmlenz
parents: 446
diff changeset
129 in_textarea = True
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
130 elif in_select and tagname == 'option':
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
131 option_start = kind, data, pos
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
132 option_value = attrs.get('value')
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
133 in_option = True
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
134 continue
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
135 yield kind, (tag, attrs), pos
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
136
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
137 elif in_form and kind is TEXT:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
138 if in_select and in_option:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
139 if option_value is None:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
140 option_value = data
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
141 option_text = kind, data, pos
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
142 continue
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
143 elif in_textarea:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
144 continue
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
145 yield kind, data, pos
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
146
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
147 elif in_form and kind is END:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
148 tagname = data.localname
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
149 if tagname == 'form':
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
150 in_form = False
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
151 elif tagname == 'select':
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
152 in_select = False
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
153 select_value = None
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
154 elif in_select and tagname == 'option':
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
155 if isinstance(select_value, (tuple, list)):
415
b9f9a22484f0 `HTMLFormFiller` now correctly deals with non-string values in the data dictionary for select/checkbox/radio controls.
cmlenz
parents: 408
diff changeset
156 selected = option_value in [str(v) for v
b9f9a22484f0 `HTMLFormFiller` now correctly deals with non-string values in the data dictionary for select/checkbox/radio controls.
cmlenz
parents: 408
diff changeset
157 in select_value]
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
158 else:
415
b9f9a22484f0 `HTMLFormFiller` now correctly deals with non-string values in the data dictionary for select/checkbox/radio controls.
cmlenz
parents: 408
diff changeset
159 selected = option_value == str(select_value)
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
160 okind, (tag, attrs), opos = option_start
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
161 if selected:
403
228907abb726 Remove some magic/overhead from `Attrs` creation and manipulation by not automatically wrapping attribute names in `QName`.
cmlenz
parents: 363
diff changeset
162 attrs |= [(QName('selected'), 'selected')]
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
163 elif 'selected' in attrs:
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
164 attrs -= 'selected'
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
165 yield okind, (tag, attrs), opos
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
166 if option_text:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
167 yield option_text
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
168 in_option = False
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
169 option_start = option_text = option_value = None
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
170 elif tagname == 'textarea':
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
171 if textarea_value:
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
172 yield TEXT, unicode(textarea_value), pos
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
173 in_textarea = False
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
174 yield kind, data, pos
275
d91cbdeb75e9 Integrated `HTMLFormFiller` filter initially presented as a [wiki:FormFilling#Usingatemplatefilter recipe].
cmlenz
parents: 230
diff changeset
175
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
176 else:
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
177 yield kind, data, pos
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
178
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
179
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
180 class HTMLSanitizer(object):
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
181 """A filter that removes potentially dangerous HTML tags and attributes
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
182 from the stream.
431
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
183
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
184 >>> from genshi import HTML
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
185 >>> html = HTML('<div><script>alert(document.cookie)</script></div>')
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
186 >>> print html | HTMLSanitizer()
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
187 <div/>
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
188
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
189 The default set of safe tags and attributes can be modified when the filter
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
190 is instantiated. For example, to allow inline ``style`` attributes, the
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
191 following instantation would work:
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
192
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
193 >>> html = HTML('<div style="background: #000"></div>')
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
194 >>> sanitizer = HTMLSanitizer(safe_attrs=HTMLSanitizer.SAFE_ATTRS | set(['style']))
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
195 >>> print html | sanitizer
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
196 <div style="background: #000"/>
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
197
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
198 Note that even in this case, the filter *does* attempt to remove dangerous
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
199 constructs from style attributes:
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
200
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
201 >>> html = HTML('<div style="background: url(javascript:void); color: #000"></div>')
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
202 >>> print html | sanitizer
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
203 <div style="color: #000"/>
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
204
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
205 This handles HTML entities, unicode escapes in CSS and Javascript text, as
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
206 well as a lot of other things. However, the style tag is still excluded by
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
207 default because it is very hard for such sanitizing to be completely safe,
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
208 especially considering how much error recovery current web browsers perform.
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
209
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
210 :warn: Note that this special processing of CSS is currently only applied to
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
211 style attributes, **not** style elements.
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
212 """
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
213
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
214 SAFE_TAGS = frozenset(['a', 'abbr', 'acronym', 'address', 'area', 'b',
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
215 'big', 'blockquote', 'br', 'button', 'caption', 'center', 'cite',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
216 'code', 'col', 'colgroup', 'dd', 'del', 'dfn', 'dir', 'div', 'dl', 'dt',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
217 'em', 'fieldset', 'font', 'form', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
218 'hr', 'i', 'img', 'input', 'ins', 'kbd', 'label', 'legend', 'li', 'map',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
219 'menu', 'ol', 'optgroup', 'option', 'p', 'pre', 'q', 's', 'samp',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
220 'select', 'small', 'span', 'strike', 'strong', 'sub', 'sup', 'table',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
221 'tbody', 'td', 'textarea', 'tfoot', 'th', 'thead', 'tr', 'tt', 'u',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
222 'ul', 'var'])
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
223
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
224 SAFE_ATTRS = frozenset(['abbr', 'accept', 'accept-charset', 'accesskey',
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
225 'action', 'align', 'alt', 'axis', 'bgcolor', 'border', 'cellpadding',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
226 'cellspacing', 'char', 'charoff', 'charset', 'checked', 'cite', 'class',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
227 'clear', 'cols', 'colspan', 'color', 'compact', 'coords', 'datetime',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
228 'dir', 'disabled', 'enctype', 'for', 'frame', 'headers', 'height',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
229 'href', 'hreflang', 'hspace', 'id', 'ismap', 'label', 'lang',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
230 'longdesc', 'maxlength', 'media', 'method', 'multiple', 'name',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
231 'nohref', 'noshade', 'nowrap', 'prompt', 'readonly', 'rel', 'rev',
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
232 'rows', 'rowspan', 'rules', 'scope', 'selected', 'shape', 'size',
431
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
233 'span', 'src', 'start', 'summary', 'tabindex', 'target', 'title',
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
234 'type', 'usemap', 'valign', 'value', 'vspace', 'width'])
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
235
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
236 SAFE_SCHEMES = frozenset(['file', 'ftp', 'http', 'https', 'mailto', None])
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
237
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
238 URI_ATTRS = frozenset(['action', 'background', 'dynsrc', 'href', 'lowsrc',
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
239 'src'])
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
240
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
241 def __init__(self, safe_tags=SAFE_TAGS, safe_attrs=SAFE_ATTRS,
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
242 safe_schemes=SAFE_SCHEMES, uri_attrs=URI_ATTRS):
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
243 """Create the sanitizer.
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
244
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
245 The exact set of allowed elements and attributes can be configured.
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
246
425
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
247 :param safe_tags: a set of tag names that are considered safe
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
248 :param safe_attrs: a set of attribute names that are considered safe
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
249 :param safe_schemes: a set of URI schemes that are considered safe
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
250 :param uri_attrs: a set of names of attributes that contain URIs
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
251 """
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
252 self.safe_tags = safe_tags
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
253 "The set of tag names that are considered safe."
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
254 self.safe_attrs = safe_attrs
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
255 "The set of attribute names that are considered safe."
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
256 self.uri_attrs = uri_attrs
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
257 "The set of names of attributes that may contain URIs."
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
258 self.safe_schemes = safe_schemes
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
259 "The set of URI schemes that are considered safe."
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
260
439
9f11c745fac9 Add support for adding custom template filters by passing a custom callback function to the `TemplateLoader`. Closes #89 (see added unit test).
cmlenz
parents: 431
diff changeset
261 def __call__(self, stream):
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
262 """Apply the filter to the given stream.
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
263
425
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 415
diff changeset
264 :param stream: the markup event stream to filter
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
265 """
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
266 waiting_for = None
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
267
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
268 for kind, data, pos in stream:
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
269 if kind is START:
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
270 if waiting_for:
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
271 continue
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
272 tag, attrs = data
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
273 if tag not in self.safe_tags:
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
274 waiting_for = tag
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
275 continue
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
276
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
277 new_attrs = []
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
278 for attr, value in attrs:
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
279 value = stripentities(value)
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
280 if attr not in self.safe_attrs:
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
281 continue
277
7e30bfa966ab The `HTMLSanitizer` now lets you override the default set of tag and attribute names that are considered safe.
cmlenz
parents: 275
diff changeset
282 elif attr in self.uri_attrs:
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
283 # Don't allow URI schemes such as "javascript:"
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
284 if not self.is_safe_uri(value):
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
285 continue
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
286 elif attr == 'style':
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
287 # Remove dangerous CSS declarations from inline styles
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
288 decls = self.sanitize_css(value)
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
289 if not decls:
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
290 continue
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
291 value = '; '.join(decls)
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
292 new_attrs.append((attr, value))
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
293
345
2aa7ca37ae6a Make `Attrs` instances immutable.
cmlenz
parents: 305
diff changeset
294 yield kind, (tag, Attrs(new_attrs)), pos
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
295
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
296 elif kind is END:
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
297 tag = data
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
298 if waiting_for:
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
299 if waiting_for == tag:
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
300 waiting_for = None
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
301 else:
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
302 yield kind, data, pos
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
303
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
304 elif kind is not COMMENT:
123
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
305 if not waiting_for:
10279d2eeec9 Fix for #18: whitespace in space-sensitive elements such as `<pre>` and `<textarea>` is now preserved.
cmlenz
parents: 113
diff changeset
306 yield kind, data, pos
431
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
307
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
308 def is_safe_uri(self, uri):
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
309 """Determine whether the given URI is to be considered safe for
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
310 inclusion in the output.
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
311
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
312 The default implementation checks whether the scheme of the URI is in
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
313 the set of allowed URIs (`safe_schemes`).
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
314
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
315 >>> sanitizer = HTMLSanitizer()
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
316 >>> sanitizer.is_safe_uri('http://example.org/')
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
317 True
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
318 >>> sanitizer.is_safe_uri('javascript:alert(document.cookie)')
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
319 False
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
320
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
321 :param uri: the URI to check
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
322 :return: `True` if the URI can be considered safe, `False` otherwise
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
323 :rtype: `bool`
576
b00765a115a5 Improve docs on `Stream.select()` for #135.
cmlenz
parents: 571
diff changeset
324 :since: version 0.4.3
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
325 """
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
326 if ':' not in uri:
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
327 return True # This is a relative URI
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
328 chars = [char for char in uri.split(':', 1)[0] if char.isalnum()]
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
329 return ''.join(chars).lower() in self.safe_schemes
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
330
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
331 def sanitize_css(self, text):
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
332 """Remove potentially dangerous property declarations from CSS code.
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
333
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
334 In particular, properties using the CSS ``url()`` function with a scheme
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
335 that is not considered safe are removed:
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
336
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
337 >>> sanitizer = HTMLSanitizer()
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
338 >>> sanitizer.sanitize_css(u'''
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
339 ... background: url(javascript:alert("foo"));
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
340 ... color: #000;
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
341 ... ''')
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
342 [u'color: #000']
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
343
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
344 Also, the proprietary Internet Explorer function ``expression()`` is
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
345 always stripped:
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
346
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
347 >>> sanitizer.sanitize_css(u'''
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
348 ... background: #fff;
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
349 ... color: #000;
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
350 ... width: e/**/xpression(alert("foo"));
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
351 ... ''')
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
352 [u'background: #fff', u'color: #000']
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
353
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
354 :param text: the CSS text; this is expected to be `unicode` and to not
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
355 contain any character or numeric references
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
356 :return: a list of declarations that are considered safe
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
357 :rtype: `list`
576
b00765a115a5 Improve docs on `Stream.select()` for #135.
cmlenz
parents: 571
diff changeset
358 :since: version 0.4.3
571
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
359 """
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
360 decls = []
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
361 text = self._strip_css_comments(self._replace_unicode_escapes(text))
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
362 for decl in filter(None, text.split(';')):
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
363 decl = decl.strip()
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
364 if not decl:
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
365 continue
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
366 is_evil = False
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
367 if 'expression' in decl:
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
368 is_evil = True
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
369 for match in re.finditer(r'url\s*\(([^)]+)', decl):
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
370 if not self.is_safe_uri(match.group(1)):
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
371 is_evil = True
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
372 break
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
373 if not is_evil:
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
374 decls.append(decl.strip())
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
375 return decls
f0461dc3939a * Cleaned up the implementation of the `HTMLSanitizer`.
cmlenz
parents: 556
diff changeset
376
431
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
377 _NORMALIZE_NEWLINES = re.compile(r'\r\n').sub
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
378 _UNICODE_ESCAPE = re.compile(r'\\([0-9a-fA-F]{1,6})\s?').sub
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
379
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
380 def _replace_unicode_escapes(self, text):
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
381 def _repl(match):
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
382 return unichr(int(match.group(1), 16))
ad01564e87f2 * Don't allow `style` attributes by default in the `HTMLSanitizer`. Closes #97.
cmlenz
parents: 425
diff changeset
383 return self._UNICODE_ESCAPE(_repl, self._NORMALIZE_NEWLINES('\n', text))
556
0d98569eaced The HTML sanitizer now strips any CSS comments in style attributes, which could previously be used to hide malicious property values.
cmlenz
parents: 471
diff changeset
384
0d98569eaced The HTML sanitizer now strips any CSS comments in style attributes, which could previously be used to hide malicious property values.
cmlenz
parents: 471
diff changeset
385 _CSS_COMMENTS = re.compile(r'/\*.*?\*/').sub
0d98569eaced The HTML sanitizer now strips any CSS comments in style attributes, which could previously be used to hide malicious property values.
cmlenz
parents: 471
diff changeset
386
0d98569eaced The HTML sanitizer now strips any CSS comments in style attributes, which could previously be used to hide malicious property values.
cmlenz
parents: 471
diff changeset
387 def _strip_css_comments(self, text):
0d98569eaced The HTML sanitizer now strips any CSS comments in style attributes, which could previously be used to hide malicious property values.
cmlenz
parents: 471
diff changeset
388 return self._CSS_COMMENTS('', text)
Copyright (C) 2012-2017 Edgewall Software