Mercurial > genshi > mirror
annotate genshi/filters/i18n.py @ 856:21308bd343b8 trunk
Add a couple of fallback imports for Python 3.0.
author | cmlenz |
---|---|
date | Thu, 12 Nov 2009 11:36:14 +0000 |
parents | 4d9bef447df9 |
children | 013623773357 |
rev | line source |
---|---|
531 | 1 # -*- coding: utf-8 -*- |
2 # | |
850
e8f596c726c4
Update copyright years on files changed due to advanced-i18n merge.
cmlenz
parents:
849
diff
changeset
|
3 # Copyright (C) 2007-2009 Edgewall Software |
531 | 4 # All rights reserved. |
5 # | |
6 # This software is licensed as described in the file COPYING, which | |
7 # you should have received as part of this distribution. The terms | |
8 # are also available at http://genshi.edgewall.org/wiki/License. | |
9 # | |
10 # This software consists of voluntary contributions made by many | |
11 # individuals. For the exact contribution history, see the revision | |
12 # history and logs, available at http://genshi.edgewall.org/log/. | |
13 | |
849 | 14 """Directives and utilities for internationalization and localization of |
15 templates. | |
576 | 16 |
17 :since: version 0.4 | |
849 | 18 :note: Directives support added since version 0.6 |
576 | 19 """ |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
20 |
856 | 21 try: |
22 any | |
23 except NameError: | |
24 from genshi.util import any | |
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
25 from gettext import NullTranslations |
849 | 26 import os |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
27 import re |
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
28 from types import FunctionType |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
29 |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
30 from genshi.core import Attrs, Namespace, QName, START, END, TEXT, START_NS, \ |
849 | 31 END_NS, XML_NAMESPACE, _ensure, StreamEventKind |
794
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
32 from genshi.template.eval import _ast |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
33 from genshi.template.base import DirectiveFactory, EXPR, SUB, _apply_directives |
849 | 34 from genshi.template.directives import Directive, StripDirective |
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
35 from genshi.template.markup import MarkupTemplate, EXEC |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
36 |
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
37 __all__ = ['Translator', 'extract'] |
501
5e7604c2d60d
Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions.
cmlenz
parents:
493
diff
changeset
|
38 __docformat__ = 'restructuredtext en' |
5e7604c2d60d
Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions.
cmlenz
parents:
493
diff
changeset
|
39 |
849 | 40 |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
41 I18N_NAMESPACE = Namespace('http://genshi.edgewall.org/i18n') |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
42 |
849 | 43 MSGBUF = StreamEventKind('MSGBUF') |
44 SUB_START = StreamEventKind('SUB_START') | |
45 SUB_END = StreamEventKind('SUB_END') | |
46 | |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
47 |
849 | 48 class I18NDirective(Directive): |
49 """Simple interface for i18n directives to support messages extraction.""" | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
50 |
849 | 51 def __call__(self, stream, directives, ctxt, **vars): |
52 return _apply_directives(stream, directives, ctxt, vars) | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
53 |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
54 |
849 | 55 class ExtractableI18NDirective(I18NDirective): |
56 """Simple interface for directives to support messages extraction.""" | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
57 |
849 | 58 def extract(self, stream, comment_stack): |
59 raise NotImplementedError | |
60 | |
61 | |
62 class CommentDirective(I18NDirective): | |
63 """Implementation of the ``i18n:comment`` template directive which adds | |
64 translation comments. | |
65 | |
66 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
67 ... <p i18n:comment="As in Foo Bar">Foo</p> | |
68 ... </html>''') | |
69 >>> translator = Translator() | |
70 >>> translator.setup(tmpl) | |
71 >>> list(translator.extract(tmpl.stream)) | |
72 [(2, None, u'Foo', [u'As in Foo Bar'])] | |
73 """ | |
74 __slots__ = ['comment'] | |
75 | |
76 def __init__(self, value, template, hints=None, namespaces=None, | |
77 lineno=-1, offset=-1): | |
78 Directive.__init__(self, None, template, namespaces, lineno, offset) | |
79 self.comment = value | |
80 | |
81 | |
82 class MsgDirective(ExtractableI18NDirective): | |
83 r"""Implementation of the ``i18n:msg`` directive which marks inner content | |
84 as translatable. Consider the following examples: | |
85 | |
86 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
87 ... <div i18n:msg=""> | |
88 ... <p>Foo</p> | |
89 ... <p>Bar</p> | |
90 ... </div> | |
91 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
92 ... </html>''') | |
93 | |
94 >>> translator = Translator() | |
95 >>> translator.setup(tmpl) | |
96 >>> list(translator.extract(tmpl.stream)) | |
97 [(2, None, u'[1:Foo]\n [2:Bar]', []), (6, None, u'Foo [1:bar]!', [])] | |
853
f33ecf3c319e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
98 >>> print(tmpl.generate().render()) |
849 | 99 <html> |
100 <div><p>Foo</p> | |
101 <p>Bar</p></div> | |
102 <p>Foo <em>bar</em>!</p> | |
103 </html> | |
104 | |
105 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
106 ... <div i18n:msg="fname, lname"> | |
107 ... <p>First Name: ${fname}</p> | |
108 ... <p>Last Name: ${lname}</p> | |
109 ... </div> | |
110 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
111 ... </html>''') | |
112 >>> translator.setup(tmpl) | |
113 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
114 [(2, None, u'[1:First Name: %(fname)s]\n [2:Last Name: %(lname)s]', []), | |
115 (6, None, u'Foo [1:bar]!', [])] | |
116 | |
117 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
118 ... <div i18n:msg="fname, lname"> | |
119 ... <p>First Name: ${fname}</p> | |
120 ... <p>Last Name: ${lname}</p> | |
121 ... </div> | |
122 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
123 ... </html>''') | |
124 >>> translator.setup(tmpl) | |
853
f33ecf3c319e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
125 >>> print(tmpl.generate(fname='John', lname='Doe').render()) |
849 | 126 <html> |
127 <div><p>First Name: John</p> | |
128 <p>Last Name: Doe</p></div> | |
129 <p>Foo <em>bar</em>!</p> | |
130 </html> | |
131 | |
132 Starting and ending white-space is stripped of to make it simpler for | |
133 translators. Stripping it is not that important since it's on the html | |
134 source, the rendered output will remain the same. | |
135 """ | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
136 __slots__ = ['params'] |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
137 |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
138 def __init__(self, value, template, hints=None, namespaces=None, |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
139 lineno=-1, offset=-1): |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
140 Directive.__init__(self, None, template, namespaces, lineno, offset) |
849 | 141 self.params = [param.strip() for param in value.split(',') if param] |
142 | |
143 @classmethod | |
144 def attach(cls, template, stream, value, namespaces, pos): | |
145 if type(value) is dict: | |
146 value = value.get('params', '').strip() | |
147 return super(MsgDirective, cls).attach(template, stream, value.strip(), | |
148 namespaces, pos) | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
149 |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
150 def __call__(self, stream, directives, ctxt, **vars): |
849 | 151 gettext = ctxt.get('_i18n.gettext') |
152 dgettext = ctxt.get('_i18n.dgettext') | |
153 if ctxt.get('_i18n.domain'): | |
854
4d9bef447df9
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
154 assert hasattr(dgettext, '__call__'), \ |
4d9bef447df9
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
155 'No domain gettext function passed' |
849 | 156 gettext = lambda msg: dgettext(ctxt.get('_i18n.domain'), msg) |
157 | |
158 def _generate(): | |
159 msgbuf = MessageBuffer(self) | |
160 previous = stream.next() | |
161 if previous[0] is START: | |
162 yield previous | |
163 else: | |
164 msgbuf.append(*previous) | |
165 previous = stream.next() | |
166 for kind, data, pos in stream: | |
167 msgbuf.append(*previous) | |
168 previous = kind, data, pos | |
169 if previous[0] is not END: | |
170 msgbuf.append(*previous) | |
171 previous = None | |
172 for event in msgbuf.translate(gettext(msgbuf.format())): | |
173 yield event | |
174 if previous: | |
175 yield previous | |
176 | |
177 return _apply_directives(_generate(), directives, ctxt, vars) | |
178 | |
179 def extract(self, stream, comment_stack): | |
180 msgbuf = MessageBuffer(self) | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
181 |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
182 stream = iter(stream) |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
183 previous = stream.next() |
849 | 184 if previous[0] is START: |
185 previous = stream.next() | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
186 for event in stream: |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
187 msgbuf.append(*previous) |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
188 previous = event |
849 | 189 if previous[0] is not END: |
190 msgbuf.append(*previous) | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
191 |
849 | 192 yield None, msgbuf.format(), comment_stack[-1:] |
193 | |
194 | |
195 class ChooseBranchDirective(I18NDirective): | |
196 __slots__ = ['params'] | |
197 | |
198 def __call__(self, stream, directives, ctxt, **vars): | |
199 self.params = ctxt.get('_i18n.choose.params', [])[:] | |
200 msgbuf = MessageBuffer(self) | |
201 | |
202 stream = iter(_apply_directives(stream, directives, ctxt, vars)) | |
203 yield stream.next() # the outer start tag | |
204 previous = stream.next() | |
205 for kind, data, pos in stream: | |
206 msgbuf.append(*previous) | |
207 previous = kind, data, pos | |
208 yield MSGBUF, (), -1 # the place holder for msgbuf output | |
209 yield previous # the outer end tag | |
210 ctxt['_i18n.choose.%s' % type(self).__name__] = msgbuf | |
211 | |
212 | |
213 def extract(self, stream, comment_stack, msgbuf): | |
214 stream = iter(stream) | |
215 previous = stream.next() | |
216 if previous[0] is START: | |
217 previous = stream.next() | |
218 for event in stream: | |
219 msgbuf.append(*previous) | |
220 previous = event | |
221 if previous[0] is not END: | |
222 msgbuf.append(*previous) | |
223 return msgbuf | |
224 | |
225 | |
226 class SingularDirective(ChooseBranchDirective): | |
227 """Implementation of the ``i18n:singular`` directive to be used with the | |
228 ``i18n:choose`` directive.""" | |
229 | |
230 | |
231 class PluralDirective(ChooseBranchDirective): | |
232 """Implementation of the ``i18n:plural`` directive to be used with the | |
233 ``i18n:choose`` directive.""" | |
234 | |
235 | |
236 class ChooseDirective(ExtractableI18NDirective): | |
237 """Implementation of the ``i18n:choose`` directive which provides plural | |
238 internationalisation of strings. | |
239 | |
240 This directive requires at least one parameter, the one which evaluates to | |
241 an integer which will allow to choose the plural/singular form. If you also | |
242 have expressions inside the singular and plural version of the string you | |
243 also need to pass a name for those parameters. Consider the following | |
244 examples: | |
245 | |
246 >>> tmpl = MarkupTemplate('''\ | |
247 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
248 ... <div i18n:choose="num; num"> | |
249 ... <p i18n:singular="">There is $num coin</p> | |
250 ... <p i18n:plural="">There are $num coins</p> | |
251 ... </div> | |
252 ... </html>''') | |
253 >>> translator = Translator() | |
254 >>> translator.setup(tmpl) | |
255 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
256 [(2, 'ngettext', (u'There is %(num)s coin', | |
257 u'There are %(num)s coins'), [])] | |
258 | |
259 >>> tmpl = MarkupTemplate('''\ | |
260 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
261 ... <div i18n:choose="num; num"> | |
262 ... <p i18n:singular="">There is $num coin</p> | |
263 ... <p i18n:plural="">There are $num coins</p> | |
264 ... </div> | |
265 ... </html>''') | |
266 >>> translator.setup(tmpl) | |
853
f33ecf3c319e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
267 >>> print(tmpl.generate(num=1).render()) |
849 | 268 <html> |
269 <div> | |
270 <p>There is 1 coin</p> | |
271 </div> | |
272 </html> | |
853
f33ecf3c319e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
273 >>> print(tmpl.generate(num=2).render()) |
849 | 274 <html> |
275 <div> | |
276 <p>There are 2 coins</p> | |
277 </div> | |
278 </html> | |
279 | |
280 When used as a directive and not as an attribute: | |
281 | |
282 >>> tmpl = MarkupTemplate('''\ | |
283 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
284 ... <i18n:choose numeral="num" params="num"> | |
285 ... <p i18n:singular="">There is $num coin</p> | |
286 ... <p i18n:plural="">There are $num coins</p> | |
287 ... </i18n:choose> | |
288 ... </html>''') | |
289 >>> translator.setup(tmpl) | |
290 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
291 [(2, 'ngettext', (u'There is %(num)s coin', | |
292 u'There are %(num)s coins'), [])] | |
293 """ | |
294 __slots__ = ['numeral', 'params'] | |
295 | |
296 def __init__(self, value, template, hints=None, namespaces=None, | |
297 lineno=-1, offset=-1): | |
298 Directive.__init__(self, None, template, namespaces, lineno, offset) | |
299 params = [v.strip() for v in value.split(';')] | |
300 self.numeral = self._parse_expr(params.pop(0), template, lineno, offset) | |
301 self.params = params and [name.strip() for name in | |
302 params[0].split(',') if name] or [] | |
303 | |
304 @classmethod | |
305 def attach(cls, template, stream, value, namespaces, pos): | |
306 if type(value) is dict: | |
307 numeral = value.get('numeral', '').strip() | |
308 assert numeral is not '', "at least pass the numeral param" | |
309 params = [v.strip() for v in value.get('params', '').split(',')] | |
310 value = '%s; ' % numeral + ', '.join(params) | |
311 return super(ChooseDirective, cls).attach(template, stream, value, | |
312 namespaces, pos) | |
313 | |
314 def __call__(self, stream, directives, ctxt, **vars): | |
315 ctxt.push({'_i18n.choose.params': self.params, | |
316 '_i18n.choose.SingularDirective': None, | |
317 '_i18n.choose.PluralDirective': None}) | |
318 | |
319 new_stream = [] | |
320 singular_stream = None | |
321 singular_msgbuf = None | |
322 plural_stream = None | |
323 plural_msgbuf = None | |
324 | |
325 ngettext = ctxt.get('_i18n.ungettext') | |
854
4d9bef447df9
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
326 assert hasattr(ngettext, '__call__'), 'No ngettext function available' |
849 | 327 dngettext = ctxt.get('_i18n.dngettext') |
328 if not dngettext: | |
329 dngettext = lambda d, s, p, n: ngettext(s, p, n) | |
330 | |
331 for kind, event, pos in stream: | |
332 if kind is SUB: | |
333 subdirectives, substream = event | |
334 if isinstance(subdirectives[0], | |
335 SingularDirective) and not singular_stream: | |
336 # Apply directives to update context | |
337 singular_stream = list(_apply_directives(substream, | |
338 subdirectives, | |
339 ctxt, vars)) | |
340 new_stream.append((MSGBUF, (), ('', -1))) # msgbuf place holder | |
341 singular_msgbuf = ctxt.get('_i18n.choose.SingularDirective') | |
342 elif isinstance(subdirectives[0], | |
343 PluralDirective) and not plural_stream: | |
344 # Apply directives to update context | |
345 plural_stream = list(_apply_directives(substream, | |
346 subdirectives, | |
347 ctxt, vars)) | |
348 plural_msgbuf = ctxt.get('_i18n.choose.PluralDirective') | |
349 else: | |
350 new_stream.append((kind, event, pos)) | |
351 else: | |
352 new_stream.append((kind, event, pos)) | |
353 | |
354 if ctxt.get('_i18n.domain'): | |
355 ngettext = lambda s, p, n: dngettext(ctxt.get('_i18n.domain'), | |
356 s, p, n) | |
357 | |
358 for kind, data, pos in new_stream: | |
359 if kind is MSGBUF: | |
360 for skind, sdata, spos in singular_stream: | |
361 if skind is MSGBUF: | |
362 translation = ngettext(singular_msgbuf.format(), | |
363 plural_msgbuf.format(), | |
364 self.numeral.evaluate(ctxt)) | |
365 for event in singular_msgbuf.translate(translation): | |
366 yield event | |
367 else: | |
368 yield skind, sdata, spos | |
369 else: | |
370 yield kind, data, pos | |
371 | |
372 ctxt.pop() | |
373 | |
374 def extract(self, stream, comment_stack): | |
375 stream = iter(stream) | |
376 previous = stream.next() | |
377 if previous is START: | |
378 stream.next() | |
379 | |
380 singular_msgbuf = MessageBuffer(self) | |
381 plural_msgbuf = MessageBuffer(self) | |
382 | |
383 for kind, event, pos in stream: | |
384 if kind is SUB: | |
385 subdirectives, substream = event | |
386 for subdirective in subdirectives: | |
387 if isinstance(subdirective, SingularDirective): | |
388 singular_msgbuf = subdirective.extract(substream, comment_stack, | |
389 singular_msgbuf) | |
390 elif isinstance(subdirective, PluralDirective): | |
391 plural_msgbuf = subdirective.extract(substream, comment_stack, | |
392 plural_msgbuf) | |
393 elif not isinstance(subdirective, StripDirective): | |
394 singular_msgbuf.append(kind, event, pos) | |
395 plural_msgbuf.append(kind, event, pos) | |
396 else: | |
397 singular_msgbuf.append(kind, event, pos) | |
398 plural_msgbuf.append(kind, event, pos) | |
399 | |
400 yield 'ngettext', \ | |
401 (singular_msgbuf.format(), plural_msgbuf.format()), \ | |
402 comment_stack[-1:] | |
403 | |
404 | |
405 class DomainDirective(I18NDirective): | |
406 """Implementation of the ``i18n:domain`` directive which allows choosing | |
407 another i18n domain(catalog) to translate from. | |
408 | |
409 >>> from genshi.filters.tests.i18n import DummyTranslations | |
410 >>> tmpl = MarkupTemplate('''\ | |
411 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
412 ... <p i18n:msg="">Bar</p> | |
413 ... <div i18n:domain="foo"> | |
414 ... <p i18n:msg="">FooBar</p> | |
415 ... <p>Bar</p> | |
416 ... <p i18n:domain="bar" i18n:msg="">Bar</p> | |
417 ... <p i18n:domain="">Bar</p> | |
418 ... </div> | |
419 ... <p>Bar</p> | |
420 ... </html>''') | |
421 | |
422 >>> translations = DummyTranslations({'Bar': 'Voh'}) | |
423 >>> translations.add_domain('foo', {'FooBar': 'BarFoo', 'Bar': 'foo_Bar'}) | |
424 >>> translations.add_domain('bar', {'Bar': 'bar_Bar'}) | |
425 >>> translator = Translator(translations) | |
426 >>> translator.setup(tmpl) | |
427 | |
853
f33ecf3c319e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
428 >>> print(tmpl.generate().render()) |
849 | 429 <html> |
430 <p>Voh</p> | |
431 <div> | |
432 <p>BarFoo</p> | |
433 <p>foo_Bar</p> | |
434 <p>bar_Bar</p> | |
435 <p>Voh</p> | |
436 </div> | |
437 <p>Voh</p> | |
438 </html> | |
439 """ | |
440 __slots__ = ['domain'] | |
441 | |
442 def __init__(self, value, template, hints=None, namespaces=None, | |
443 lineno=-1, offset=-1): | |
444 Directive.__init__(self, None, template, namespaces, lineno, offset) | |
445 self.domain = value and value.strip() or '__DEFAULT__' | |
446 | |
447 @classmethod | |
448 def attach(cls, template, stream, value, namespaces, pos): | |
449 if type(value) is dict: | |
450 value = value.get('name') | |
451 return super(DomainDirective, cls).attach(template, stream, value, | |
452 namespaces, pos) | |
453 | |
454 def __call__(self, stream, directives, ctxt, **vars): | |
455 ctxt.push({'_i18n.domain': self.domain}) | |
456 for event in _apply_directives(stream, directives, ctxt, vars): | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
457 yield event |
849 | 458 ctxt.pop() |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
459 |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
460 |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
461 class Translator(DirectiveFactory): |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
462 """Can extract and translate localizable strings from markup streams and |
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
463 templates. |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
464 |
849 | 465 For example, assume the following template: |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
466 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
467 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/"> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
468 ... <head> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
469 ... <title>Example</title> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
470 ... </head> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
471 ... <body> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
472 ... <h1>Example</h1> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
473 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
474 ... </body> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
475 ... </html>''', filename='example.html') |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
476 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
477 For demonstration, we define a dummy ``gettext``-style function with a |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
478 hard-coded translation table, and pass that to the `Translator` initializer: |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
479 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
480 >>> def pseudo_gettext(string): |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
481 ... return { |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
482 ... 'Example': 'Beispiel', |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
483 ... 'Hello, %(name)s': 'Hallo, %(name)s' |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
484 ... }[string] |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
485 >>> translator = Translator(pseudo_gettext) |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
486 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
487 Next, the translator needs to be prepended to any already defined filters |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
488 on the template: |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
489 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
490 >>> tmpl.filters.insert(0, translator) |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
491 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
492 When generating the template output, our hard-coded translations should be |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
493 applied as expected: |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
494 |
853
f33ecf3c319e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
495 >>> print(tmpl.generate(username='Hans', _=pseudo_gettext)) |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
496 <html> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
497 <head> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
498 <title>Beispiel</title> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
499 </head> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
500 <body> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
501 <h1>Beispiel</h1> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
502 <p>Hallo, Hans</p> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
503 </body> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
504 </html> |
849 | 505 |
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
506 Note that elements defining ``xml:lang`` attributes that do not contain |
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
507 variable expressions are ignored by this filter. That can be used to |
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
508 exclude specific parts of a template from being extracted and translated. |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
509 """ |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
510 |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
511 directives = [ |
849 | 512 ('domain', DomainDirective), |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
513 ('comment', CommentDirective), |
849 | 514 ('msg', MsgDirective), |
515 ('choose', ChooseDirective), | |
516 ('singular', SingularDirective), | |
517 ('plural', PluralDirective) | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
518 ] |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
519 |
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
520 IGNORE_TAGS = frozenset([ |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
521 QName('script'), QName('http://www.w3.org/1999/xhtml}script'), |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
522 QName('style'), QName('http://www.w3.org/1999/xhtml}style') |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
523 ]) |
849 | 524 INCLUDE_ATTRS = frozenset([ |
525 'abbr', 'alt', 'label', 'prompt', 'standby', 'summary', 'title' | |
526 ]) | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
527 NAMESPACE = I18N_NAMESPACE |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
528 |
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
529 def __init__(self, translate=NullTranslations(), ignore_tags=IGNORE_TAGS, |
594
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
530 include_attrs=INCLUDE_ATTRS, extract_text=True): |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
531 """Initialize the translator. |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
532 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
533 :param translate: the translation function, for example ``gettext`` or |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
534 ``ugettext``. |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
535 :param ignore_tags: a set of tag names that should not be localized |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
536 :param include_attrs: a set of attribute names should be localized |
594
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
537 :param extract_text: whether the content of text nodes should be |
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
538 extracted, or only text in explicit ``gettext`` |
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
539 function calls |
849 | 540 |
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
541 :note: Changed in 0.6: the `translate` parameter can now be either |
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
542 a ``gettext``-style function, or an object compatible with the |
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
543 ``NullTransalations`` or ``GNUTranslations`` interface |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
544 """ |
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
545 self.translate = translate |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
546 self.ignore_tags = ignore_tags |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
547 self.include_attrs = include_attrs |
594
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
548 self.extract_text = extract_text |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
549 |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
550 def __call__(self, stream, ctxt=None, search_text=True): |
448 | 551 """Translate any localizable strings in the given stream. |
552 | |
553 This function shouldn't be called directly. Instead, an instance of | |
554 the `Translator` class should be registered as a filter with the | |
555 `Template` or the `TemplateLoader`, or applied as a regular stream | |
556 filter. If used as a template filter, it should be inserted in front of | |
557 all the default filters. | |
558 | |
559 :param stream: the markup event stream | |
560 :param ctxt: the template context (not used) | |
561 :param search_text: whether text nodes should be translated (used | |
562 internally) | |
563 :return: the localized stream | |
564 """ | |
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
565 ignore_tags = self.ignore_tags |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
566 include_attrs = self.include_attrs |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
567 skip = 0 |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
568 xml_lang = XML_NAMESPACE['lang'] |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
569 |
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
570 if type(self.translate) is FunctionType: |
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
571 gettext = self.translate |
849 | 572 if ctxt: |
573 ctxt['_i18n.gettext'] = gettext | |
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
574 else: |
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
575 gettext = self.translate.ugettext |
849 | 576 try: |
577 dgettext = self.translate.dugettext | |
578 except AttributeError: | |
579 dgettext = lambda x, y: gettext(y) | |
580 ngettext = self.translate.ungettext | |
581 try: | |
582 dngettext = self.translate.dungettext | |
583 except AttributeError: | |
584 dngettext = lambda d, s, p, n: ngettext(s, p, n) | |
585 | |
586 if ctxt: | |
587 ctxt['_i18n.gettext'] = gettext | |
588 ctxt['_i18n.ugettext'] = gettext | |
589 ctxt['_i18n.dgettext'] = dgettext | |
590 ctxt['_i18n.ngettext'] = ngettext | |
591 ctxt['_i18n.ungettext'] = ngettext | |
592 ctxt['_i18n.dngettext'] = dngettext | |
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
593 |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
594 extract_text = self.extract_text |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
595 if not extract_text: |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
596 search_text = False |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
597 |
849 | 598 if ctxt and ctxt.get('_i18n.domain'): |
599 old_gettext = gettext | |
600 gettext = lambda msg: dgettext(ctxt.get('_i18n.domain'), msg) | |
601 | |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
602 for kind, data, pos in stream: |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
603 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
604 # skip chunks that should not be localized |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
605 if skip: |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
606 if kind is START: |
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
607 skip += 1 |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
608 elif kind is END: |
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
609 skip -= 1 |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
610 yield kind, data, pos |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
611 continue |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
612 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
613 # handle different events that can be localized |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
614 if kind is START: |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
615 tag, attrs = data |
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
616 if tag in self.ignore_tags or \ |
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
617 isinstance(attrs.get(xml_lang), basestring): |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
618 skip += 1 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
619 yield kind, data, pos |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
620 continue |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
621 |
493
3f6582a5a4a5
Fix another bug in the translation filter: translated attributes were getting added instead of replaced.
cmlenz
parents:
485
diff
changeset
|
622 new_attrs = [] |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
623 changed = False |
849 | 624 |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
625 for name, value in attrs: |
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
626 newval = value |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
627 if extract_text and isinstance(value, basestring): |
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
628 if name in include_attrs: |
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
629 newval = gettext(value) |
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
630 else: |
849 | 631 newval = list( |
632 self(_ensure(value), ctxt, search_text=False) | |
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
633 ) |
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
634 if newval != value: |
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
635 value = newval |
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
636 changed = True |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
637 new_attrs.append((name, value)) |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
638 if changed: |
667
c9a084ffaee6
Fix case where attributes weren't properly wrapped in an `Attrs` instance if one or more of them were translated by the I18n filter, potentially breaking things further down the chain. Closes #162.
cmlenz
parents:
600
diff
changeset
|
639 attrs = Attrs(new_attrs) |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
640 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
641 yield kind, (tag, attrs), pos |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
642 |
485
fb66fb3e4b49
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
643 elif search_text and kind is TEXT: |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
644 text = data.strip() |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
645 if text: |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
646 data = data.replace(text, unicode(gettext(text))) |
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
647 yield kind, data, pos |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
648 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
649 elif kind is SUB: |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
650 directives, substream = data |
849 | 651 current_domain = None |
652 for idx, directive in enumerate(directives): | |
653 # Organize directives to make everything work | |
654 if isinstance(directive, DomainDirective): | |
655 # Grab current domain and update context | |
656 current_domain = directive.domain | |
657 ctxt.push({'_i18n.domain': current_domain}) | |
658 # Put domain directive as the first one in order to | |
659 # update context before any other directives evaluation | |
660 directives.insert(0, directives.pop(idx)) | |
661 | |
662 # If this is an i18n directive, no need to translate text | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
663 # nodes here |
856 | 664 is_i18n_directive = any([ |
665 isinstance(d, ExtractableI18NDirective) | |
666 for d in directives | |
667 ]) | |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
668 substream = list(self(substream, ctxt, |
849 | 669 search_text=not is_i18n_directive)) |
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
670 yield kind, (directives, substream), pos |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
671 |
849 | 672 if current_domain: |
673 ctxt.pop() | |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
674 else: |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
675 yield kind, data, pos |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
676 |
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
677 GETTEXT_FUNCTIONS = ('_', 'gettext', 'ngettext', 'dgettext', 'dngettext', |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
678 'ugettext', 'ungettext') |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
679 |
485
fb66fb3e4b49
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
680 def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS, |
849 | 681 search_text=True, msgbuf=None, comment_stack=None): |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
682 """Extract localizable strings from the given template stream. |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
683 |
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
684 For every string found, this function yields a ``(lineno, function, |
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
685 message, comments)`` tuple, where: |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
686 |
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
687 * ``lineno`` is the number of the line on which the string was found, |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
688 * ``function`` is the name of the ``gettext`` function used (if the |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
689 string was extracted from embedded Python code), and |
469
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
690 * ``message`` is the string itself (a ``unicode`` object, or a tuple |
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
691 of ``unicode`` objects for functions with multiple string |
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
692 arguments). |
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
693 * ``comments`` is a list of comments related to the message, extracted |
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
694 from ``i18n:comment`` attributes found in the markup |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
695 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
696 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/"> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
697 ... <head> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
698 ... <title>Example</title> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
699 ... </head> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
700 ... <body> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
701 ... <h1>Example</h1> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
702 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p> |
469
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
703 ... <p>${ngettext("You have %d item", "You have %d items", num)}</p> |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
704 ... </body> |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
705 ... </html>''', filename='example.html') |
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
706 >>> for line, func, msg, comments in Translator().extract(tmpl.stream): |
853
f33ecf3c319e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
707 ... print('%d, %r, %r' % (line, func, msg)) |
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
708 3, None, u'Example' |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
709 6, None, u'Example' |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
710 7, '_', u'Hello, %(name)s' |
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
711 8, 'ngettext', (u'You have %d item', u'You have %d items', None) |
469
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
712 |
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
713 :param stream: the event stream to extract strings from; can be a |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
714 regular stream or a template stream |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
715 :param gettext_functions: a sequence of function names that should be |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
716 treated as gettext-style localization |
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
717 functions |
485
fb66fb3e4b49
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
718 :param search_text: whether the content of text nodes should be |
fb66fb3e4b49
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
719 extracted (used internally) |
469
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
720 |
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
721 :note: Changed in 0.4.1: For a function with multiple string arguments |
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
722 (such as ``ngettext``), a single item with a tuple of strings is |
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
723 yielded, instead an item for each string argument. |
849 | 724 :note: Changed in 0.6: The returned tuples now include a fourth |
725 element, which is a list of comments for the translator. | |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
726 """ |
594
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
727 if not self.extract_text: |
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
728 search_text = False |
849 | 729 if comment_stack is None: |
730 comment_stack = [] | |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
731 skip = 0 |
849 | 732 |
733 # Un-comment bellow to extract messages without adding directives | |
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
734 xml_lang = XML_NAMESPACE['lang'] |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
735 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
736 for kind, data, pos in stream: |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
737 if skip: |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
738 if kind is START: |
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
739 skip += 1 |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
740 if kind is END: |
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
741 skip -= 1 |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
742 |
549
7214c1bdb383
The I18n filter now extracts text from translation functions in ignored tags. Fixes #132.
cmlenz
parents:
535
diff
changeset
|
743 if kind is START and not skip: |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
744 tag, attrs = data |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
745 |
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
746 if tag in self.ignore_tags or \ |
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
747 isinstance(attrs.get(xml_lang), basestring): |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
748 skip += 1 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
749 continue |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
750 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
751 for name, value in attrs: |
594
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
752 if search_text and isinstance(value, basestring): |
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
753 if name in self.include_attrs: |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
754 text = value.strip() |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
755 if text: |
849 | 756 # XXX: Do we need to grab i18n:comment from comment_stack ??? |
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
757 yield pos[1], None, text, [] |
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
758 else: |
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
759 for lineno, funcname, text, comments in self.extract( |
485
fb66fb3e4b49
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
760 _ensure(value), gettext_functions, |
535
35a413f3f1dd
The I18n filter no longer extracts or translates literal strings in attribute values that also contain expressions.
cmlenz
parents:
531
diff
changeset
|
761 search_text=False): |
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
762 yield lineno, funcname, text, comments |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
763 |
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
764 if msgbuf: |
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
765 msgbuf.append(kind, data, pos) |
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
766 |
549
7214c1bdb383
The I18n filter now extracts text from translation functions in ignored tags. Fixes #132.
cmlenz
parents:
535
diff
changeset
|
767 elif not skip and search_text and kind is TEXT: |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
768 if not msgbuf: |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
769 text = data.strip() |
854
4d9bef447df9
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
770 if text and [ch for ch in text if ch.isalpha()]: |
849 | 771 yield pos[1], None, text, comment_stack[-1:] |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
772 else: |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
773 msgbuf.append(kind, data, pos) |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
774 |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
775 elif not skip and msgbuf and kind is END: |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
776 msgbuf.append(kind, data, pos) |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
777 if not msgbuf.depth: |
854
4d9bef447df9
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
778 yield msgbuf.lineno, None, msgbuf.format(), [ |
4d9bef447df9
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
779 c for c in msgbuf.comment if c |
4d9bef447df9
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
780 ] |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
781 msgbuf = None |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
782 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
783 elif kind is EXPR or kind is EXEC: |
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
784 if msgbuf: |
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
785 msgbuf.append(kind, data, pos) |
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
786 for funcname, strings in extract_from_code(data, |
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
787 gettext_functions): |
849 | 788 # XXX: Do we need to grab i18n:comment from comment_stack ??? |
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
789 yield pos[1], funcname, strings, [] |
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
790 |
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
791 elif kind is SUB: |
849 | 792 directives, substream = data |
793 in_comment = False | |
794 | |
795 for idx, directive in enumerate(directives): | |
796 # Do a first loop to see if there's a comment directive | |
797 # If there is update context and pop it from directives | |
798 if isinstance(directive, CommentDirective): | |
799 in_comment = True | |
800 comment_stack.append(directive.comment) | |
801 if len(directives) == 1: | |
802 # in case we're in the presence of something like: | |
803 # <p i18n:comment="foo">Foo</p> | |
804 messages = self.extract( | |
805 substream, gettext_functions, | |
806 search_text=search_text and not skip, | |
807 msgbuf=msgbuf, comment_stack=comment_stack) | |
808 for lineno, funcname, text, comments in messages: | |
809 yield lineno, funcname, text, comments | |
810 directives.pop(idx) | |
811 elif not isinstance(directive, I18NDirective): | |
812 # Remove all other non i18n directives from the process | |
813 directives.pop(idx) | |
814 | |
815 if not directives and not in_comment: | |
816 # Extract content if there's no directives because | |
817 # strip was pop'ed and not because comment was pop'ed. | |
818 # Extraction in this case has been taken care of. | |
819 messages = self.extract( | |
820 substream, gettext_functions, | |
821 search_text=search_text and not skip, msgbuf=msgbuf) | |
822 for lineno, funcname, text, comments in messages: | |
823 yield lineno, funcname, text, comments | |
824 | |
825 for directive in directives: | |
826 if isinstance(directive, ExtractableI18NDirective): | |
827 messages = directive.extract(substream, comment_stack) | |
828 for funcname, text, comments in messages: | |
829 yield pos[1], funcname, text, comments | |
830 else: | |
831 messages = self.extract( | |
832 substream, gettext_functions, | |
833 search_text=search_text and not skip, msgbuf=msgbuf) | |
834 for lineno, funcname, text, comments in messages: | |
835 yield lineno, funcname, text, comments | |
836 | |
837 if in_comment: | |
838 comment_stack.pop() | |
839 | |
840 def get_directive_index(self, dir_cls): | |
841 total = len(self._dir_order) | |
842 if dir_cls in self._dir_order: | |
843 return self._dir_order.index(dir_cls) - total | |
844 return total | |
845 | |
846 def setup(self, template): | |
847 """Convenience function to register the `Translator` filter and the | |
848 related directives with the given template. | |
849 | |
850 :param template: a `Template` instance | |
851 """ | |
852 template.filters.insert(0, self) | |
853 if hasattr(template, 'add_directives'): | |
854 template.add_directives(Translator.NAMESPACE, self) | |
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
855 |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
856 |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
857 class MessageBuffer(object): |
738 | 858 """Helper class for managing internationalized mixed content. |
576 | 859 |
860 :since: version 0.5 | |
861 """ | |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
862 |
849 | 863 def __init__(self, directive=None): |
738 | 864 """Initialize the message buffer. |
865 | |
776
ddb58d74c8ee
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
866 :param params: comma-separated list of parameter names |
ddb58d74c8ee
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
867 :type params: `basestring` |
738 | 868 :param lineno: the line number on which the first stream event |
869 belonging to the message was found | |
870 """ | |
849 | 871 # params list needs to be copied so that directives can be evaluated |
872 # more than once | |
873 self.orig_params = self.params = directive.params[:] | |
874 self.directive = directive | |
738 | 875 self.string = [] |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
876 self.events = {} |
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
877 self.values = {} |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
878 self.depth = 1 |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
879 self.order = 1 |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
880 self.stack = [0] |
849 | 881 self.subdirectives = {} |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
882 |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
883 def append(self, kind, data, pos): |
738 | 884 """Append a stream event to the buffer. |
885 | |
886 :param kind: the stream event kind | |
887 :param data: the event data | |
888 :param pos: the position of the event in the source | |
889 """ | |
849 | 890 if kind is SUB: |
891 # The order needs to be +1 because a new START kind event will | |
892 # happen and we we need to wrap those events into our custom kind(s) | |
893 order = self.stack[-1] + 1 | |
894 subdirectives, substream = data | |
895 # Store the directives that should be applied after translation | |
896 self.subdirectives.setdefault(order, []).extend(subdirectives) | |
897 self.events.setdefault(order, []).append((SUB_START, None, pos)) | |
898 for skind, sdata, spos in substream: | |
899 self.append(skind, sdata, spos) | |
900 self.events.setdefault(order, []).append((SUB_END, None, pos)) | |
901 elif kind is TEXT: | |
902 if '[' in data or ']' in data: | |
903 # Quote [ and ] if it ain't us adding it, ie, if the user is | |
904 # using those chars in his templates, escape them | |
905 data = data.replace('[', '\[').replace(']', '\]') | |
738 | 906 self.string.append(data) |
849 | 907 self.events.setdefault(self.stack[-1], []).append((kind, data, pos)) |
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
908 elif kind is EXPR: |
849 | 909 if self.params: |
910 param = self.params.pop(0) | |
911 else: | |
912 params = ', '.join(['"%s"' % p for p in self.orig_params if p]) | |
913 if params: | |
914 params = "(%s)" % params | |
915 raise IndexError("%d parameters%s given to 'i18n:%s' but " | |
916 "%d or more expressions used in '%s', line %s" | |
917 % (len(self.orig_params), params, | |
918 self.directive.tagname, | |
919 len(self.orig_params)+1, | |
920 os.path.basename(pos[0] or | |
921 'In Memmory Template'), | |
922 pos[1])) | |
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
923 self.string.append('%%(%s)s' % param) |
849 | 924 self.events.setdefault(self.stack[-1], []).append((kind, data, pos)) |
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
925 self.values[param] = (kind, data, pos) |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
926 else: |
849 | 927 if kind is START: |
854
4d9bef447df9
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
928 self.string.append('[%d:' % self.order) |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
929 self.stack.append(self.order) |
849 | 930 self.events.setdefault(self.stack[-1], |
931 []).append((kind, data, pos)) | |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
932 self.depth += 1 |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
933 self.order += 1 |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
934 elif kind is END: |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
935 self.depth -= 1 |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
936 if self.depth: |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
937 self.events[self.stack[-1]].append((kind, data, pos)) |
854
4d9bef447df9
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
938 self.string.append(']') |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
939 self.stack.pop() |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
940 |
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
941 def format(self): |
738 | 942 """Return a message identifier representing the content in the |
943 buffer. | |
944 """ | |
854
4d9bef447df9
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
945 return ''.join(self.string).strip() |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
946 |
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
947 def translate(self, string, regex=re.compile(r'%\((\w+)\)s')): |
738 | 948 """Interpolate the given message translation with the events in the |
949 buffer and return the translated stream. | |
950 | |
951 :param string: the translated message string | |
952 """ | |
849 | 953 substream = None |
954 | |
955 def yield_parts(string): | |
956 for idx, part in enumerate(regex.split(string)): | |
957 if idx % 2: | |
958 yield self.values[part] | |
959 elif part: | |
960 yield (TEXT, | |
961 part.replace('\[', '[').replace('\]', ']'), | |
962 (None, -1, -1) | |
963 ) | |
964 | |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
965 parts = parse_msg(string) |
849 | 966 parts_counter = {} |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
967 for order, string in parts: |
849 | 968 parts_counter.setdefault(order, []).append(None) |
969 | |
970 while parts: | |
971 order, string = parts.pop(0) | |
972 if len(parts_counter[order]) == 1: | |
973 events = self.events[order] | |
974 else: | |
975 events = [self.events[order].pop(0)] | |
976 parts_counter[order].pop() | |
977 | |
978 for event in events: | |
979 if event[0] is SUB_START: | |
980 substream = [] | |
981 elif event[0] is SUB_END: | |
982 # Yield a substream which might have directives to be | |
983 # applied to it (after translation events) | |
984 yield SUB, (self.subdirectives[order], substream), event[2] | |
985 substream = None | |
986 elif event[0] is TEXT: | |
987 if string: | |
988 for part in yield_parts(string): | |
989 if substream is not None: | |
990 substream.append(part) | |
991 else: | |
992 yield part | |
993 # String handled, reset it | |
994 string = None | |
995 elif event[0] is START: | |
996 if substream is not None: | |
997 substream.append(event) | |
998 else: | |
999 yield event | |
1000 if string: | |
1001 for part in yield_parts(string): | |
1002 if substream is not None: | |
1003 substream.append(part) | |
1004 else: | |
1005 yield part | |
1006 # String handled, reset it | |
1007 string = None | |
1008 elif event[0] is END: | |
1009 if string: | |
1010 for part in yield_parts(string): | |
1011 if substream is not None: | |
1012 substream.append(part) | |
1013 else: | |
1014 yield part | |
1015 # String handled, reset it | |
1016 string = None | |
1017 if substream is not None: | |
1018 substream.append(event) | |
1019 else: | |
1020 yield event | |
1021 elif event[0] is EXPR: | |
1022 # These are handled on the strings itself | |
1023 continue | |
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
1024 else: |
849 | 1025 if string: |
1026 for part in yield_parts(string): | |
1027 if substream is not None: | |
1028 substream.append(part) | |
1029 else: | |
1030 yield part | |
1031 # String handled, reset it | |
1032 string = None | |
1033 if substream is not None: | |
1034 substream.append(event) | |
1035 else: | |
1036 yield event | |
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1037 |
849 | 1038 def parse_msg(string, regex=re.compile(r'(?:\[(\d+)\:)|(?<!\\)\]')): |
738 | 1039 """Parse a translated message using Genshi mixed content message |
1040 formatting. | |
849 | 1041 |
738 | 1042 >>> parse_msg("See [1:Help].") |
1043 [(0, 'See '), (1, 'Help'), (0, '.')] | |
849 | 1044 |
738 | 1045 >>> parse_msg("See [1:our [2:Help] page] for details.") |
1046 [(0, 'See '), (1, 'our '), (2, 'Help'), (1, ' page'), (0, ' for details.')] | |
849 | 1047 |
738 | 1048 >>> parse_msg("[2:Details] finden Sie in [1:Hilfe].") |
1049 [(2, 'Details'), (0, ' finden Sie in '), (1, 'Hilfe'), (0, '.')] | |
849 | 1050 |
738 | 1051 >>> parse_msg("[1:] Bilder pro Seite anzeigen.") |
1052 [(1, ''), (0, ' Bilder pro Seite anzeigen.')] | |
849 | 1053 |
738 | 1054 :param string: the translated message string |
1055 :return: a list of ``(order, string)`` tuples | |
1056 :rtype: `list` | |
1057 """ | |
1058 parts = [] | |
1059 stack = [0] | |
1060 while True: | |
1061 mo = regex.search(string) | |
1062 if not mo: | |
1063 break | |
1064 | |
1065 if mo.start() or stack[-1]: | |
1066 parts.append((stack[-1], string[:mo.start()])) | |
1067 string = string[mo.end():] | |
1068 | |
1069 orderno = mo.group(1) | |
1070 if orderno is not None: | |
1071 stack.append(int(orderno)) | |
1072 else: | |
1073 stack.pop() | |
1074 if not stack: | |
1075 break | |
1076 | |
1077 if string: | |
1078 parts.append((stack[-1], string)) | |
1079 | |
1080 return parts | |
1081 | |
776
ddb58d74c8ee
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
1082 |
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1083 def extract_from_code(code, gettext_functions): |
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1084 """Extract strings from Python bytecode. |
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1085 |
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1086 >>> from genshi.template.eval import Expression |
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1087 >>> expr = Expression('_("Hello")') |
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1088 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS)) |
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1089 [('_', u'Hello')] |
849 | 1090 |
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1091 >>> expr = Expression('ngettext("You have %(num)s item", ' |
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1092 ... '"You have %(num)s items", num)') |
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1093 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS)) |
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1094 [('ngettext', (u'You have %(num)s item', u'You have %(num)s items', None))] |
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1095 |
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1096 :param code: the `Code` object |
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1097 :type code: `genshi.template.eval.Code` |
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1098 :param gettext_functions: a sequence of function names |
576 | 1099 :since: version 0.5 |
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1100 """ |
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1101 def _walk(node): |
794
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1102 if isinstance(node, _ast.Call) and isinstance(node.func, _ast.Name) \ |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1103 and node.func.id in gettext_functions: |
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1104 strings = [] |
600
23a4784203ae
Handle starargs and dstarargs in the I18n extraction code.
cmlenz
parents:
596
diff
changeset
|
1105 def _add(arg): |
794
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1106 if isinstance(arg, _ast.Str) and isinstance(arg.s, basestring): |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1107 strings.append(unicode(arg.s, 'utf-8')) |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1108 elif arg: |
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1109 strings.append(None) |
600
23a4784203ae
Handle starargs and dstarargs in the I18n extraction code.
cmlenz
parents:
596
diff
changeset
|
1110 [_add(arg) for arg in node.args] |
794
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1111 _add(node.starargs) |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1112 _add(node.kwargs) |
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1113 if len(strings) == 1: |
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1114 strings = strings[0] |
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1115 else: |
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1116 strings = tuple(strings) |
794
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1117 yield node.func.id, strings |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1118 elif node._fields: |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1119 children = [] |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1120 for field in node._fields: |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1121 child = getattr(node, field, None) |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1122 if isinstance(child, list): |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1123 for elem in child: |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1124 children.append(elem) |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1125 elif isinstance(child, _ast.AST): |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1126 children.append(child) |
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1127 for child in children: |
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1128 for funcname, strings in _walk(child): |
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1129 yield funcname, strings |
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1130 return _walk(code.ast) |
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1131 |
776
ddb58d74c8ee
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
1132 |
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1133 def extract(fileobj, keywords, comment_tags, options): |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1134 """Babel extraction method for Genshi templates. |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1135 |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1136 :param fileobj: the file-like object the messages should be extracted from |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1137 :param keywords: a list of keywords (i.e. function names) that should be |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1138 recognized as translation functions |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1139 :param comment_tags: a list of translator tags to search for and include |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1140 in the results |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1141 :param options: a dictionary of additional options (optional) |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1142 :return: an iterator over ``(lineno, funcname, message, comments)`` tuples |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1143 :rtype: ``iterator`` |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1144 """ |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1145 template_class = options.get('template_class', MarkupTemplate) |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1146 if isinstance(template_class, basestring): |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1147 module, clsname = template_class.split(':', 1) |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1148 template_class = getattr(__import__(module, {}, {}, [clsname]), clsname) |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1149 encoding = options.get('encoding', None) |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1150 |
596
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1151 extract_text = options.get('extract_text', True) |
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1152 if isinstance(extract_text, basestring): |
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1153 extract_text = extract_text.lower() in ('1', 'on', 'yes', 'true') |
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1154 |
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1155 ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS) |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1156 if isinstance(ignore_tags, basestring): |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1157 ignore_tags = ignore_tags.split() |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1158 ignore_tags = [QName(tag) for tag in ignore_tags] |
596
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1159 |
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1160 include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS) |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1161 if isinstance(include_attrs, basestring): |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1162 include_attrs = include_attrs.split() |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1163 include_attrs = [QName(attr) for attr in include_attrs] |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1164 |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1165 tmpl = template_class(fileobj, filename=getattr(fileobj, 'name', None), |
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1166 encoding=encoding) |
849 | 1167 |
596
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1168 translator = Translator(None, ignore_tags, include_attrs, extract_text) |
849 | 1169 if hasattr(tmpl, 'add_directives'): |
1170 tmpl.add_directives(Translator.NAMESPACE, translator) | |
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
1171 for message in translator.extract(tmpl.stream, gettext_functions=keywords): |
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
1172 yield message |