Mercurial > genshi > mirror
annotate genshi/filters/i18n.py @ 850:e8f596c726c4 trunk
Update copyright years on files changed due to advanced-i18n merge.
| author | cmlenz |
|---|---|
| date | Tue, 10 Nov 2009 20:57:32 +0000 |
| parents | 878c4313c7d5 |
| children | f33ecf3c319e |
| rev | line source |
|---|---|
| 531 | 1 # -*- coding: utf-8 -*- |
| 2 # | |
|
850
e8f596c726c4
Update copyright years on files changed due to advanced-i18n merge.
cmlenz
parents:
849
diff
changeset
|
3 # Copyright (C) 2007-2009 Edgewall Software |
| 531 | 4 # All rights reserved. |
| 5 # | |
| 6 # This software is licensed as described in the file COPYING, which | |
| 7 # you should have received as part of this distribution. The terms | |
| 8 # are also available at http://genshi.edgewall.org/wiki/License. | |
| 9 # | |
| 10 # This software consists of voluntary contributions made by many | |
| 11 # individuals. For the exact contribution history, see the revision | |
| 12 # history and logs, available at http://genshi.edgewall.org/log/. | |
| 13 | |
| 849 | 14 """Directives and utilities for internationalization and localization of |
| 15 templates. | |
| 576 | 16 |
| 17 :since: version 0.4 | |
| 849 | 18 :note: Directives support added since version 0.6 |
| 576 | 19 """ |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
20 |
|
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
21 from gettext import NullTranslations |
| 849 | 22 import os |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
23 import re |
|
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
24 from types import FunctionType |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
25 |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
26 from genshi.core import Attrs, Namespace, QName, START, END, TEXT, START_NS, \ |
| 849 | 27 END_NS, XML_NAMESPACE, _ensure, StreamEventKind |
|
794
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
28 from genshi.template.eval import _ast |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
29 from genshi.template.base import DirectiveFactory, EXPR, SUB, _apply_directives |
| 849 | 30 from genshi.template.directives import Directive, StripDirective |
|
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
31 from genshi.template.markup import MarkupTemplate, EXEC |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
32 |
|
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
33 __all__ = ['Translator', 'extract'] |
|
501
5e7604c2d60d
Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions.
cmlenz
parents:
493
diff
changeset
|
34 __docformat__ = 'restructuredtext en' |
|
5e7604c2d60d
Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions.
cmlenz
parents:
493
diff
changeset
|
35 |
| 849 | 36 |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
37 I18N_NAMESPACE = Namespace('http://genshi.edgewall.org/i18n') |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
38 |
| 849 | 39 MSGBUF = StreamEventKind('MSGBUF') |
| 40 SUB_START = StreamEventKind('SUB_START') | |
| 41 SUB_END = StreamEventKind('SUB_END') | |
| 42 | |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
43 |
| 849 | 44 class I18NDirective(Directive): |
| 45 """Simple interface for i18n directives to support messages extraction.""" | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
46 |
| 849 | 47 def __call__(self, stream, directives, ctxt, **vars): |
| 48 return _apply_directives(stream, directives, ctxt, vars) | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
49 |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
50 |
| 849 | 51 class ExtractableI18NDirective(I18NDirective): |
| 52 """Simple interface for directives to support messages extraction.""" | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
53 |
| 849 | 54 def extract(self, stream, comment_stack): |
| 55 raise NotImplementedError | |
| 56 | |
| 57 | |
| 58 class CommentDirective(I18NDirective): | |
| 59 """Implementation of the ``i18n:comment`` template directive which adds | |
| 60 translation comments. | |
| 61 | |
| 62 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 63 ... <p i18n:comment="As in Foo Bar">Foo</p> | |
| 64 ... </html>''') | |
| 65 >>> translator = Translator() | |
| 66 >>> translator.setup(tmpl) | |
| 67 >>> list(translator.extract(tmpl.stream)) | |
| 68 [(2, None, u'Foo', [u'As in Foo Bar'])] | |
| 69 """ | |
| 70 __slots__ = ['comment'] | |
| 71 | |
| 72 def __init__(self, value, template, hints=None, namespaces=None, | |
| 73 lineno=-1, offset=-1): | |
| 74 Directive.__init__(self, None, template, namespaces, lineno, offset) | |
| 75 self.comment = value | |
| 76 | |
| 77 | |
| 78 class MsgDirective(ExtractableI18NDirective): | |
| 79 r"""Implementation of the ``i18n:msg`` directive which marks inner content | |
| 80 as translatable. Consider the following examples: | |
| 81 | |
| 82 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 83 ... <div i18n:msg=""> | |
| 84 ... <p>Foo</p> | |
| 85 ... <p>Bar</p> | |
| 86 ... </div> | |
| 87 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
| 88 ... </html>''') | |
| 89 | |
| 90 >>> translator = Translator() | |
| 91 >>> translator.setup(tmpl) | |
| 92 >>> list(translator.extract(tmpl.stream)) | |
| 93 [(2, None, u'[1:Foo]\n [2:Bar]', []), (6, None, u'Foo [1:bar]!', [])] | |
| 94 >>> print tmpl.generate().render() | |
| 95 <html> | |
| 96 <div><p>Foo</p> | |
| 97 <p>Bar</p></div> | |
| 98 <p>Foo <em>bar</em>!</p> | |
| 99 </html> | |
| 100 | |
| 101 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 102 ... <div i18n:msg="fname, lname"> | |
| 103 ... <p>First Name: ${fname}</p> | |
| 104 ... <p>Last Name: ${lname}</p> | |
| 105 ... </div> | |
| 106 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
| 107 ... </html>''') | |
| 108 >>> translator.setup(tmpl) | |
| 109 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
| 110 [(2, None, u'[1:First Name: %(fname)s]\n [2:Last Name: %(lname)s]', []), | |
| 111 (6, None, u'Foo [1:bar]!', [])] | |
| 112 | |
| 113 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 114 ... <div i18n:msg="fname, lname"> | |
| 115 ... <p>First Name: ${fname}</p> | |
| 116 ... <p>Last Name: ${lname}</p> | |
| 117 ... </div> | |
| 118 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
| 119 ... </html>''') | |
| 120 >>> translator.setup(tmpl) | |
| 121 >>> print tmpl.generate(fname='John', lname='Doe').render() | |
| 122 <html> | |
| 123 <div><p>First Name: John</p> | |
| 124 <p>Last Name: Doe</p></div> | |
| 125 <p>Foo <em>bar</em>!</p> | |
| 126 </html> | |
| 127 | |
| 128 Starting and ending white-space is stripped of to make it simpler for | |
| 129 translators. Stripping it is not that important since it's on the html | |
| 130 source, the rendered output will remain the same. | |
| 131 """ | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
132 __slots__ = ['params'] |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
133 |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
134 def __init__(self, value, template, hints=None, namespaces=None, |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
135 lineno=-1, offset=-1): |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
136 Directive.__init__(self, None, template, namespaces, lineno, offset) |
| 849 | 137 self.params = [param.strip() for param in value.split(',') if param] |
| 138 | |
| 139 @classmethod | |
| 140 def attach(cls, template, stream, value, namespaces, pos): | |
| 141 if type(value) is dict: | |
| 142 value = value.get('params', '').strip() | |
| 143 return super(MsgDirective, cls).attach(template, stream, value.strip(), | |
| 144 namespaces, pos) | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
145 |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
146 def __call__(self, stream, directives, ctxt, **vars): |
| 849 | 147 gettext = ctxt.get('_i18n.gettext') |
| 148 dgettext = ctxt.get('_i18n.dgettext') | |
| 149 if ctxt.get('_i18n.domain'): | |
| 150 assert callable(dgettext), "No domain gettext function passed" | |
| 151 gettext = lambda msg: dgettext(ctxt.get('_i18n.domain'), msg) | |
| 152 | |
| 153 def _generate(): | |
| 154 msgbuf = MessageBuffer(self) | |
| 155 previous = stream.next() | |
| 156 if previous[0] is START: | |
| 157 yield previous | |
| 158 else: | |
| 159 msgbuf.append(*previous) | |
| 160 previous = stream.next() | |
| 161 for kind, data, pos in stream: | |
| 162 msgbuf.append(*previous) | |
| 163 previous = kind, data, pos | |
| 164 if previous[0] is not END: | |
| 165 msgbuf.append(*previous) | |
| 166 previous = None | |
| 167 for event in msgbuf.translate(gettext(msgbuf.format())): | |
| 168 yield event | |
| 169 if previous: | |
| 170 yield previous | |
| 171 | |
| 172 return _apply_directives(_generate(), directives, ctxt, vars) | |
| 173 | |
| 174 def extract(self, stream, comment_stack): | |
| 175 msgbuf = MessageBuffer(self) | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
176 |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
177 stream = iter(stream) |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
178 previous = stream.next() |
| 849 | 179 if previous[0] is START: |
| 180 previous = stream.next() | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
181 for event in stream: |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
182 msgbuf.append(*previous) |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
183 previous = event |
| 849 | 184 if previous[0] is not END: |
| 185 msgbuf.append(*previous) | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
186 |
| 849 | 187 yield None, msgbuf.format(), comment_stack[-1:] |
| 188 | |
| 189 | |
| 190 class ChooseBranchDirective(I18NDirective): | |
| 191 __slots__ = ['params'] | |
| 192 | |
| 193 def __call__(self, stream, directives, ctxt, **vars): | |
| 194 self.params = ctxt.get('_i18n.choose.params', [])[:] | |
| 195 msgbuf = MessageBuffer(self) | |
| 196 | |
| 197 stream = iter(_apply_directives(stream, directives, ctxt, vars)) | |
| 198 yield stream.next() # the outer start tag | |
| 199 previous = stream.next() | |
| 200 for kind, data, pos in stream: | |
| 201 msgbuf.append(*previous) | |
| 202 previous = kind, data, pos | |
| 203 yield MSGBUF, (), -1 # the place holder for msgbuf output | |
| 204 yield previous # the outer end tag | |
| 205 ctxt['_i18n.choose.%s' % type(self).__name__] = msgbuf | |
| 206 | |
| 207 | |
| 208 def extract(self, stream, comment_stack, msgbuf): | |
| 209 stream = iter(stream) | |
| 210 previous = stream.next() | |
| 211 if previous[0] is START: | |
| 212 previous = stream.next() | |
| 213 for event in stream: | |
| 214 msgbuf.append(*previous) | |
| 215 previous = event | |
| 216 if previous[0] is not END: | |
| 217 msgbuf.append(*previous) | |
| 218 return msgbuf | |
| 219 | |
| 220 | |
| 221 class SingularDirective(ChooseBranchDirective): | |
| 222 """Implementation of the ``i18n:singular`` directive to be used with the | |
| 223 ``i18n:choose`` directive.""" | |
| 224 | |
| 225 | |
| 226 class PluralDirective(ChooseBranchDirective): | |
| 227 """Implementation of the ``i18n:plural`` directive to be used with the | |
| 228 ``i18n:choose`` directive.""" | |
| 229 | |
| 230 | |
| 231 class ChooseDirective(ExtractableI18NDirective): | |
| 232 """Implementation of the ``i18n:choose`` directive which provides plural | |
| 233 internationalisation of strings. | |
| 234 | |
| 235 This directive requires at least one parameter, the one which evaluates to | |
| 236 an integer which will allow to choose the plural/singular form. If you also | |
| 237 have expressions inside the singular and plural version of the string you | |
| 238 also need to pass a name for those parameters. Consider the following | |
| 239 examples: | |
| 240 | |
| 241 >>> tmpl = MarkupTemplate('''\ | |
| 242 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 243 ... <div i18n:choose="num; num"> | |
| 244 ... <p i18n:singular="">There is $num coin</p> | |
| 245 ... <p i18n:plural="">There are $num coins</p> | |
| 246 ... </div> | |
| 247 ... </html>''') | |
| 248 >>> translator = Translator() | |
| 249 >>> translator.setup(tmpl) | |
| 250 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
| 251 [(2, 'ngettext', (u'There is %(num)s coin', | |
| 252 u'There are %(num)s coins'), [])] | |
| 253 | |
| 254 >>> tmpl = MarkupTemplate('''\ | |
| 255 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 256 ... <div i18n:choose="num; num"> | |
| 257 ... <p i18n:singular="">There is $num coin</p> | |
| 258 ... <p i18n:plural="">There are $num coins</p> | |
| 259 ... </div> | |
| 260 ... </html>''') | |
| 261 >>> translator.setup(tmpl) | |
| 262 >>> print tmpl.generate(num=1).render() | |
| 263 <html> | |
| 264 <div> | |
| 265 <p>There is 1 coin</p> | |
| 266 </div> | |
| 267 </html> | |
| 268 >>> print tmpl.generate(num=2).render() | |
| 269 <html> | |
| 270 <div> | |
| 271 <p>There are 2 coins</p> | |
| 272 </div> | |
| 273 </html> | |
| 274 | |
| 275 When used as a directive and not as an attribute: | |
| 276 | |
| 277 >>> tmpl = MarkupTemplate('''\ | |
| 278 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 279 ... <i18n:choose numeral="num" params="num"> | |
| 280 ... <p i18n:singular="">There is $num coin</p> | |
| 281 ... <p i18n:plural="">There are $num coins</p> | |
| 282 ... </i18n:choose> | |
| 283 ... </html>''') | |
| 284 >>> translator.setup(tmpl) | |
| 285 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
| 286 [(2, 'ngettext', (u'There is %(num)s coin', | |
| 287 u'There are %(num)s coins'), [])] | |
| 288 """ | |
| 289 __slots__ = ['numeral', 'params'] | |
| 290 | |
| 291 def __init__(self, value, template, hints=None, namespaces=None, | |
| 292 lineno=-1, offset=-1): | |
| 293 Directive.__init__(self, None, template, namespaces, lineno, offset) | |
| 294 params = [v.strip() for v in value.split(';')] | |
| 295 self.numeral = self._parse_expr(params.pop(0), template, lineno, offset) | |
| 296 self.params = params and [name.strip() for name in | |
| 297 params[0].split(',') if name] or [] | |
| 298 | |
| 299 @classmethod | |
| 300 def attach(cls, template, stream, value, namespaces, pos): | |
| 301 if type(value) is dict: | |
| 302 numeral = value.get('numeral', '').strip() | |
| 303 assert numeral is not '', "at least pass the numeral param" | |
| 304 params = [v.strip() for v in value.get('params', '').split(',')] | |
| 305 value = '%s; ' % numeral + ', '.join(params) | |
| 306 return super(ChooseDirective, cls).attach(template, stream, value, | |
| 307 namespaces, pos) | |
| 308 | |
| 309 def __call__(self, stream, directives, ctxt, **vars): | |
| 310 ctxt.push({'_i18n.choose.params': self.params, | |
| 311 '_i18n.choose.SingularDirective': None, | |
| 312 '_i18n.choose.PluralDirective': None}) | |
| 313 | |
| 314 new_stream = [] | |
| 315 singular_stream = None | |
| 316 singular_msgbuf = None | |
| 317 plural_stream = None | |
| 318 plural_msgbuf = None | |
| 319 | |
| 320 ngettext = ctxt.get('_i18n.ungettext') | |
| 321 assert callable(ngettext), "No ngettext function available" | |
| 322 dngettext = ctxt.get('_i18n.dngettext') | |
| 323 if not dngettext: | |
| 324 dngettext = lambda d, s, p, n: ngettext(s, p, n) | |
| 325 | |
| 326 for kind, event, pos in stream: | |
| 327 if kind is SUB: | |
| 328 subdirectives, substream = event | |
| 329 if isinstance(subdirectives[0], | |
| 330 SingularDirective) and not singular_stream: | |
| 331 # Apply directives to update context | |
| 332 singular_stream = list(_apply_directives(substream, | |
| 333 subdirectives, | |
| 334 ctxt, vars)) | |
| 335 new_stream.append((MSGBUF, (), ('', -1))) # msgbuf place holder | |
| 336 singular_msgbuf = ctxt.get('_i18n.choose.SingularDirective') | |
| 337 elif isinstance(subdirectives[0], | |
| 338 PluralDirective) and not plural_stream: | |
| 339 # Apply directives to update context | |
| 340 plural_stream = list(_apply_directives(substream, | |
| 341 subdirectives, | |
| 342 ctxt, vars)) | |
| 343 plural_msgbuf = ctxt.get('_i18n.choose.PluralDirective') | |
| 344 else: | |
| 345 new_stream.append((kind, event, pos)) | |
| 346 else: | |
| 347 new_stream.append((kind, event, pos)) | |
| 348 | |
| 349 if ctxt.get('_i18n.domain'): | |
| 350 ngettext = lambda s, p, n: dngettext(ctxt.get('_i18n.domain'), | |
| 351 s, p, n) | |
| 352 | |
| 353 for kind, data, pos in new_stream: | |
| 354 if kind is MSGBUF: | |
| 355 for skind, sdata, spos in singular_stream: | |
| 356 if skind is MSGBUF: | |
| 357 translation = ngettext(singular_msgbuf.format(), | |
| 358 plural_msgbuf.format(), | |
| 359 self.numeral.evaluate(ctxt)) | |
| 360 for event in singular_msgbuf.translate(translation): | |
| 361 yield event | |
| 362 else: | |
| 363 yield skind, sdata, spos | |
| 364 else: | |
| 365 yield kind, data, pos | |
| 366 | |
| 367 ctxt.pop() | |
| 368 | |
| 369 def extract(self, stream, comment_stack): | |
| 370 stream = iter(stream) | |
| 371 previous = stream.next() | |
| 372 if previous is START: | |
| 373 stream.next() | |
| 374 | |
| 375 singular_msgbuf = MessageBuffer(self) | |
| 376 plural_msgbuf = MessageBuffer(self) | |
| 377 | |
| 378 for kind, event, pos in stream: | |
| 379 if kind is SUB: | |
| 380 subdirectives, substream = event | |
| 381 for subdirective in subdirectives: | |
| 382 if isinstance(subdirective, SingularDirective): | |
| 383 singular_msgbuf = subdirective.extract(substream, comment_stack, | |
| 384 singular_msgbuf) | |
| 385 elif isinstance(subdirective, PluralDirective): | |
| 386 plural_msgbuf = subdirective.extract(substream, comment_stack, | |
| 387 plural_msgbuf) | |
| 388 elif not isinstance(subdirective, StripDirective): | |
| 389 singular_msgbuf.append(kind, event, pos) | |
| 390 plural_msgbuf.append(kind, event, pos) | |
| 391 else: | |
| 392 singular_msgbuf.append(kind, event, pos) | |
| 393 plural_msgbuf.append(kind, event, pos) | |
| 394 | |
| 395 yield 'ngettext', \ | |
| 396 (singular_msgbuf.format(), plural_msgbuf.format()), \ | |
| 397 comment_stack[-1:] | |
| 398 | |
| 399 | |
| 400 class DomainDirective(I18NDirective): | |
| 401 """Implementation of the ``i18n:domain`` directive which allows choosing | |
| 402 another i18n domain(catalog) to translate from. | |
| 403 | |
| 404 >>> from genshi.filters.tests.i18n import DummyTranslations | |
| 405 >>> tmpl = MarkupTemplate('''\ | |
| 406 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 407 ... <p i18n:msg="">Bar</p> | |
| 408 ... <div i18n:domain="foo"> | |
| 409 ... <p i18n:msg="">FooBar</p> | |
| 410 ... <p>Bar</p> | |
| 411 ... <p i18n:domain="bar" i18n:msg="">Bar</p> | |
| 412 ... <p i18n:domain="">Bar</p> | |
| 413 ... </div> | |
| 414 ... <p>Bar</p> | |
| 415 ... </html>''') | |
| 416 | |
| 417 >>> translations = DummyTranslations({'Bar': 'Voh'}) | |
| 418 >>> translations.add_domain('foo', {'FooBar': 'BarFoo', 'Bar': 'foo_Bar'}) | |
| 419 >>> translations.add_domain('bar', {'Bar': 'bar_Bar'}) | |
| 420 >>> translator = Translator(translations) | |
| 421 >>> translator.setup(tmpl) | |
| 422 | |
| 423 >>> print tmpl.generate().render() | |
| 424 <html> | |
| 425 <p>Voh</p> | |
| 426 <div> | |
| 427 <p>BarFoo</p> | |
| 428 <p>foo_Bar</p> | |
| 429 <p>bar_Bar</p> | |
| 430 <p>Voh</p> | |
| 431 </div> | |
| 432 <p>Voh</p> | |
| 433 </html> | |
| 434 """ | |
| 435 __slots__ = ['domain'] | |
| 436 | |
| 437 def __init__(self, value, template, hints=None, namespaces=None, | |
| 438 lineno=-1, offset=-1): | |
| 439 Directive.__init__(self, None, template, namespaces, lineno, offset) | |
| 440 self.domain = value and value.strip() or '__DEFAULT__' | |
| 441 | |
| 442 @classmethod | |
| 443 def attach(cls, template, stream, value, namespaces, pos): | |
| 444 if type(value) is dict: | |
| 445 value = value.get('name') | |
| 446 return super(DomainDirective, cls).attach(template, stream, value, | |
| 447 namespaces, pos) | |
| 448 | |
| 449 def __call__(self, stream, directives, ctxt, **vars): | |
| 450 ctxt.push({'_i18n.domain': self.domain}) | |
| 451 for event in _apply_directives(stream, directives, ctxt, vars): | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
452 yield event |
| 849 | 453 ctxt.pop() |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
454 |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
455 |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
456 class Translator(DirectiveFactory): |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
457 """Can extract and translate localizable strings from markup streams and |
|
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
458 templates. |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
459 |
| 849 | 460 For example, assume the following template: |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
461 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
462 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/"> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
463 ... <head> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
464 ... <title>Example</title> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
465 ... </head> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
466 ... <body> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
467 ... <h1>Example</h1> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
468 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
469 ... </body> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
470 ... </html>''', filename='example.html') |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
471 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
472 For demonstration, we define a dummy ``gettext``-style function with a |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
473 hard-coded translation table, and pass that to the `Translator` initializer: |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
474 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
475 >>> def pseudo_gettext(string): |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
476 ... return { |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
477 ... 'Example': 'Beispiel', |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
478 ... 'Hello, %(name)s': 'Hallo, %(name)s' |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
479 ... }[string] |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
480 >>> translator = Translator(pseudo_gettext) |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
481 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
482 Next, the translator needs to be prepended to any already defined filters |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
483 on the template: |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
484 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
485 >>> tmpl.filters.insert(0, translator) |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
486 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
487 When generating the template output, our hard-coded translations should be |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
488 applied as expected: |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
489 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
490 >>> print tmpl.generate(username='Hans', _=pseudo_gettext) |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
491 <html> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
492 <head> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
493 <title>Beispiel</title> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
494 </head> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
495 <body> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
496 <h1>Beispiel</h1> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
497 <p>Hallo, Hans</p> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
498 </body> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
499 </html> |
| 849 | 500 |
|
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
501 Note that elements defining ``xml:lang`` attributes that do not contain |
|
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
502 variable expressions are ignored by this filter. That can be used to |
|
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
503 exclude specific parts of a template from being extracted and translated. |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
504 """ |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
505 |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
506 directives = [ |
| 849 | 507 ('domain', DomainDirective), |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
508 ('comment', CommentDirective), |
| 849 | 509 ('msg', MsgDirective), |
| 510 ('choose', ChooseDirective), | |
| 511 ('singular', SingularDirective), | |
| 512 ('plural', PluralDirective) | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
513 ] |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
514 |
|
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
515 IGNORE_TAGS = frozenset([ |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
516 QName('script'), QName('http://www.w3.org/1999/xhtml}script'), |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
517 QName('style'), QName('http://www.w3.org/1999/xhtml}style') |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
518 ]) |
| 849 | 519 INCLUDE_ATTRS = frozenset([ |
| 520 'abbr', 'alt', 'label', 'prompt', 'standby', 'summary', 'title' | |
| 521 ]) | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
522 NAMESPACE = I18N_NAMESPACE |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
523 |
|
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
524 def __init__(self, translate=NullTranslations(), ignore_tags=IGNORE_TAGS, |
|
594
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
525 include_attrs=INCLUDE_ATTRS, extract_text=True): |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
526 """Initialize the translator. |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
527 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
528 :param translate: the translation function, for example ``gettext`` or |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
529 ``ugettext``. |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
530 :param ignore_tags: a set of tag names that should not be localized |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
531 :param include_attrs: a set of attribute names should be localized |
|
594
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
532 :param extract_text: whether the content of text nodes should be |
|
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
533 extracted, or only text in explicit ``gettext`` |
|
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
534 function calls |
| 849 | 535 |
|
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
536 :note: Changed in 0.6: the `translate` parameter can now be either |
|
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
537 a ``gettext``-style function, or an object compatible with the |
|
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
538 ``NullTransalations`` or ``GNUTranslations`` interface |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
539 """ |
|
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
540 self.translate = translate |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
541 self.ignore_tags = ignore_tags |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
542 self.include_attrs = include_attrs |
|
594
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
543 self.extract_text = extract_text |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
544 |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
545 def __call__(self, stream, ctxt=None, search_text=True): |
| 448 | 546 """Translate any localizable strings in the given stream. |
| 547 | |
| 548 This function shouldn't be called directly. Instead, an instance of | |
| 549 the `Translator` class should be registered as a filter with the | |
| 550 `Template` or the `TemplateLoader`, or applied as a regular stream | |
| 551 filter. If used as a template filter, it should be inserted in front of | |
| 552 all the default filters. | |
| 553 | |
| 554 :param stream: the markup event stream | |
| 555 :param ctxt: the template context (not used) | |
| 556 :param search_text: whether text nodes should be translated (used | |
| 557 internally) | |
| 558 :return: the localized stream | |
| 559 """ | |
|
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
560 ignore_tags = self.ignore_tags |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
561 include_attrs = self.include_attrs |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
562 skip = 0 |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
563 xml_lang = XML_NAMESPACE['lang'] |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
564 |
|
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
565 if type(self.translate) is FunctionType: |
|
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
566 gettext = self.translate |
| 849 | 567 if ctxt: |
| 568 ctxt['_i18n.gettext'] = gettext | |
|
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
569 else: |
|
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
570 gettext = self.translate.ugettext |
| 849 | 571 try: |
| 572 dgettext = self.translate.dugettext | |
| 573 except AttributeError: | |
| 574 dgettext = lambda x, y: gettext(y) | |
| 575 ngettext = self.translate.ungettext | |
| 576 try: | |
| 577 dngettext = self.translate.dungettext | |
| 578 except AttributeError: | |
| 579 dngettext = lambda d, s, p, n: ngettext(s, p, n) | |
| 580 | |
| 581 if ctxt: | |
| 582 ctxt['_i18n.gettext'] = gettext | |
| 583 ctxt['_i18n.ugettext'] = gettext | |
| 584 ctxt['_i18n.dgettext'] = dgettext | |
| 585 ctxt['_i18n.ngettext'] = ngettext | |
| 586 ctxt['_i18n.ungettext'] = ngettext | |
| 587 ctxt['_i18n.dngettext'] = dngettext | |
|
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
588 |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
589 extract_text = self.extract_text |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
590 if not extract_text: |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
591 search_text = False |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
592 |
| 849 | 593 if ctxt and ctxt.get('_i18n.domain'): |
| 594 old_gettext = gettext | |
| 595 gettext = lambda msg: dgettext(ctxt.get('_i18n.domain'), msg) | |
| 596 | |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
597 for kind, data, pos in stream: |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
598 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
599 # skip chunks that should not be localized |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
600 if skip: |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
601 if kind is START: |
|
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
602 skip += 1 |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
603 elif kind is END: |
|
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
604 skip -= 1 |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
605 yield kind, data, pos |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
606 continue |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
607 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
608 # handle different events that can be localized |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
609 if kind is START: |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
610 tag, attrs = data |
|
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
611 if tag in self.ignore_tags or \ |
|
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
612 isinstance(attrs.get(xml_lang), basestring): |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
613 skip += 1 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
614 yield kind, data, pos |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
615 continue |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
616 |
|
493
3f6582a5a4a5
Fix another bug in the translation filter: translated attributes were getting added instead of replaced.
cmlenz
parents:
485
diff
changeset
|
617 new_attrs = [] |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
618 changed = False |
| 849 | 619 |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
620 for name, value in attrs: |
|
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
621 newval = value |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
622 if extract_text and isinstance(value, basestring): |
|
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
623 if name in include_attrs: |
|
788
31432f30a6fb
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
624 newval = gettext(value) |
|
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
625 else: |
| 849 | 626 newval = list( |
| 627 self(_ensure(value), ctxt, search_text=False) | |
|
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
628 ) |
|
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
629 if newval != value: |
|
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
630 value = newval |
|
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
631 changed = True |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
632 new_attrs.append((name, value)) |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
633 if changed: |
|
667
c9a084ffaee6
Fix case where attributes weren't properly wrapped in an `Attrs` instance if one or more of them were translated by the I18n filter, potentially breaking things further down the chain. Closes #162.
cmlenz
parents:
600
diff
changeset
|
634 attrs = Attrs(new_attrs) |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
635 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
636 yield kind, (tag, attrs), pos |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
637 |
|
485
fb66fb3e4b49
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
638 elif search_text and kind is TEXT: |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
639 text = data.strip() |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
640 if text: |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
641 data = data.replace(text, unicode(gettext(text))) |
|
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
642 yield kind, data, pos |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
643 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
644 elif kind is SUB: |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
645 directives, substream = data |
| 849 | 646 current_domain = None |
| 647 for idx, directive in enumerate(directives): | |
| 648 # Organize directives to make everything work | |
| 649 if isinstance(directive, DomainDirective): | |
| 650 # Grab current domain and update context | |
| 651 current_domain = directive.domain | |
| 652 ctxt.push({'_i18n.domain': current_domain}) | |
| 653 # Put domain directive as the first one in order to | |
| 654 # update context before any other directives evaluation | |
| 655 directives.insert(0, directives.pop(idx)) | |
| 656 | |
| 657 # If this is an i18n directive, no need to translate text | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
658 # nodes here |
| 849 | 659 is_i18n_directive = filter(None, |
| 660 [isinstance(d, ExtractableI18NDirective) | |
| 661 for d in directives]) | |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
662 substream = list(self(substream, ctxt, |
| 849 | 663 search_text=not is_i18n_directive)) |
|
790
da90cee22560
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
664 yield kind, (directives, substream), pos |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
665 |
| 849 | 666 if current_domain: |
| 667 ctxt.pop() | |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
668 else: |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
669 yield kind, data, pos |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
670 |
|
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
671 GETTEXT_FUNCTIONS = ('_', 'gettext', 'ngettext', 'dgettext', 'dngettext', |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
672 'ugettext', 'ungettext') |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
673 |
|
485
fb66fb3e4b49
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
674 def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS, |
| 849 | 675 search_text=True, msgbuf=None, comment_stack=None): |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
676 """Extract localizable strings from the given template stream. |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
677 |
|
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
678 For every string found, this function yields a ``(lineno, function, |
|
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
679 message, comments)`` tuple, where: |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
680 |
|
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
681 * ``lineno`` is the number of the line on which the string was found, |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
682 * ``function`` is the name of the ``gettext`` function used (if the |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
683 string was extracted from embedded Python code), and |
|
469
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
684 * ``message`` is the string itself (a ``unicode`` object, or a tuple |
|
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
685 of ``unicode`` objects for functions with multiple string |
|
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
686 arguments). |
|
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
687 * ``comments`` is a list of comments related to the message, extracted |
|
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
688 from ``i18n:comment`` attributes found in the markup |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
689 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
690 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/"> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
691 ... <head> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
692 ... <title>Example</title> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
693 ... </head> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
694 ... <body> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
695 ... <h1>Example</h1> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
696 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p> |
|
469
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
697 ... <p>${ngettext("You have %d item", "You have %d items", num)}</p> |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
698 ... </body> |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
699 ... </html>''', filename='example.html') |
|
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
700 >>> for line, func, msg, comments in Translator().extract(tmpl.stream): |
|
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
701 ... print "%d, %r, %r" % (line, func, msg) |
|
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
702 3, None, u'Example' |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
703 6, None, u'Example' |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
704 7, '_', u'Hello, %(name)s' |
|
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
705 8, 'ngettext', (u'You have %d item', u'You have %d items', None) |
|
469
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
706 |
|
450
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
707 :param stream: the event stream to extract strings from; can be a |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
708 regular stream or a template stream |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
709 :param gettext_functions: a sequence of function names that should be |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
710 treated as gettext-style localization |
|
94601511cd68
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
711 functions |
|
485
fb66fb3e4b49
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
712 :param search_text: whether the content of text nodes should be |
|
fb66fb3e4b49
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
713 extracted (used internally) |
|
469
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
714 |
|
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
715 :note: Changed in 0.4.1: For a function with multiple string arguments |
|
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
716 (such as ``ngettext``), a single item with a tuple of strings is |
|
2d3246f9ea54
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
717 yielded, instead an item for each string argument. |
| 849 | 718 :note: Changed in 0.6: The returned tuples now include a fourth |
| 719 element, which is a list of comments for the translator. | |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
720 """ |
|
594
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
721 if not self.extract_text: |
|
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
722 search_text = False |
| 849 | 723 if comment_stack is None: |
| 724 comment_stack = [] | |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
725 skip = 0 |
| 849 | 726 |
| 727 # Un-comment bellow to extract messages without adding directives | |
|
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
728 xml_lang = XML_NAMESPACE['lang'] |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
729 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
730 for kind, data, pos in stream: |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
731 if skip: |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
732 if kind is START: |
|
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
733 skip += 1 |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
734 if kind is END: |
|
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
735 skip -= 1 |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
736 |
|
549
7214c1bdb383
The I18n filter now extracts text from translation functions in ignored tags. Fixes #132.
cmlenz
parents:
535
diff
changeset
|
737 if kind is START and not skip: |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
738 tag, attrs = data |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
739 |
|
522
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
740 if tag in self.ignore_tags or \ |
|
082535e5087c
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
741 isinstance(attrs.get(xml_lang), basestring): |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
742 skip += 1 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
743 continue |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
744 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
745 for name, value in attrs: |
|
594
2bbaa61b328f
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
746 if search_text and isinstance(value, basestring): |
|
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
747 if name in self.include_attrs: |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
748 text = value.strip() |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
749 if text: |
| 849 | 750 # XXX: Do we need to grab i18n:comment from comment_stack ??? |
|
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
751 yield pos[1], None, text, [] |
|
483
5cc92db755c5
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
752 else: |
|
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
753 for lineno, funcname, text, comments in self.extract( |
|
485
fb66fb3e4b49
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
754 _ensure(value), gettext_functions, |
|
535
35a413f3f1dd
The I18n filter no longer extracts or translates literal strings in attribute values that also contain expressions.
cmlenz
parents:
531
diff
changeset
|
755 search_text=False): |
|
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
756 yield lineno, funcname, text, comments |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
757 |
|
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
758 if msgbuf: |
|
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
759 msgbuf.append(kind, data, pos) |
|
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
760 |
|
549
7214c1bdb383
The I18n filter now extracts text from translation functions in ignored tags. Fixes #132.
cmlenz
parents:
535
diff
changeset
|
761 elif not skip and search_text and kind is TEXT: |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
762 if not msgbuf: |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
763 text = data.strip() |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
764 if text and filter(None, [ch.isalpha() for ch in text]): |
| 849 | 765 yield pos[1], None, text, comment_stack[-1:] |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
766 else: |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
767 msgbuf.append(kind, data, pos) |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
768 |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
769 elif not skip and msgbuf and kind is END: |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
770 msgbuf.append(kind, data, pos) |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
771 if not msgbuf.depth: |
|
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
772 yield msgbuf.lineno, None, msgbuf.format(), \ |
| 849 | 773 filter(None, [msgbuf.comment]) |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
774 msgbuf = None |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
775 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
776 elif kind is EXPR or kind is EXEC: |
|
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
777 if msgbuf: |
|
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
778 msgbuf.append(kind, data, pos) |
|
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
779 for funcname, strings in extract_from_code(data, |
|
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
780 gettext_functions): |
| 849 | 781 # XXX: Do we need to grab i18n:comment from comment_stack ??? |
|
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
782 yield pos[1], funcname, strings, [] |
|
446
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
783 |
|
fd9c4f7a249a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
784 elif kind is SUB: |
| 849 | 785 directives, substream = data |
| 786 in_comment = False | |
| 787 | |
| 788 for idx, directive in enumerate(directives): | |
| 789 # Do a first loop to see if there's a comment directive | |
| 790 # If there is update context and pop it from directives | |
| 791 if isinstance(directive, CommentDirective): | |
| 792 in_comment = True | |
| 793 comment_stack.append(directive.comment) | |
| 794 if len(directives) == 1: | |
| 795 # in case we're in the presence of something like: | |
| 796 # <p i18n:comment="foo">Foo</p> | |
| 797 messages = self.extract( | |
| 798 substream, gettext_functions, | |
| 799 search_text=search_text and not skip, | |
| 800 msgbuf=msgbuf, comment_stack=comment_stack) | |
| 801 for lineno, funcname, text, comments in messages: | |
| 802 yield lineno, funcname, text, comments | |
| 803 directives.pop(idx) | |
| 804 elif not isinstance(directive, I18NDirective): | |
| 805 # Remove all other non i18n directives from the process | |
| 806 directives.pop(idx) | |
| 807 | |
| 808 if not directives and not in_comment: | |
| 809 # Extract content if there's no directives because | |
| 810 # strip was pop'ed and not because comment was pop'ed. | |
| 811 # Extraction in this case has been taken care of. | |
| 812 messages = self.extract( | |
| 813 substream, gettext_functions, | |
| 814 search_text=search_text and not skip, msgbuf=msgbuf) | |
| 815 for lineno, funcname, text, comments in messages: | |
| 816 yield lineno, funcname, text, comments | |
| 817 | |
| 818 for directive in directives: | |
| 819 if isinstance(directive, ExtractableI18NDirective): | |
| 820 messages = directive.extract(substream, comment_stack) | |
| 821 for funcname, text, comments in messages: | |
| 822 yield pos[1], funcname, text, comments | |
| 823 else: | |
| 824 messages = self.extract( | |
| 825 substream, gettext_functions, | |
| 826 search_text=search_text and not skip, msgbuf=msgbuf) | |
| 827 for lineno, funcname, text, comments in messages: | |
| 828 yield lineno, funcname, text, comments | |
| 829 | |
| 830 if in_comment: | |
| 831 comment_stack.pop() | |
| 832 | |
| 833 def get_directive_index(self, dir_cls): | |
| 834 total = len(self._dir_order) | |
| 835 if dir_cls in self._dir_order: | |
| 836 return self._dir_order.index(dir_cls) - total | |
| 837 return total | |
| 838 | |
| 839 def setup(self, template): | |
| 840 """Convenience function to register the `Translator` filter and the | |
| 841 related directives with the given template. | |
| 842 | |
| 843 :param template: a `Template` instance | |
| 844 """ | |
| 845 template.filters.insert(0, self) | |
| 846 if hasattr(template, 'add_directives'): | |
| 847 template.add_directives(Translator.NAMESPACE, self) | |
|
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
848 |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
849 |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
850 class MessageBuffer(object): |
| 738 | 851 """Helper class for managing internationalized mixed content. |
| 576 | 852 |
| 853 :since: version 0.5 | |
| 854 """ | |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
855 |
| 849 | 856 def __init__(self, directive=None): |
| 738 | 857 """Initialize the message buffer. |
| 858 | |
|
776
ddb58d74c8ee
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
859 :param params: comma-separated list of parameter names |
|
ddb58d74c8ee
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
860 :type params: `basestring` |
| 738 | 861 :param lineno: the line number on which the first stream event |
| 862 belonging to the message was found | |
| 863 """ | |
| 849 | 864 # params list needs to be copied so that directives can be evaluated |
| 865 # more than once | |
| 866 self.orig_params = self.params = directive.params[:] | |
| 867 self.directive = directive | |
| 738 | 868 self.string = [] |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
869 self.events = {} |
|
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
870 self.values = {} |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
871 self.depth = 1 |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
872 self.order = 1 |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
873 self.stack = [0] |
| 849 | 874 self.subdirectives = {} |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
875 |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
876 def append(self, kind, data, pos): |
| 738 | 877 """Append a stream event to the buffer. |
| 878 | |
| 879 :param kind: the stream event kind | |
| 880 :param data: the event data | |
| 881 :param pos: the position of the event in the source | |
| 882 """ | |
| 849 | 883 if kind is SUB: |
| 884 # The order needs to be +1 because a new START kind event will | |
| 885 # happen and we we need to wrap those events into our custom kind(s) | |
| 886 order = self.stack[-1] + 1 | |
| 887 subdirectives, substream = data | |
| 888 # Store the directives that should be applied after translation | |
| 889 self.subdirectives.setdefault(order, []).extend(subdirectives) | |
| 890 self.events.setdefault(order, []).append((SUB_START, None, pos)) | |
| 891 for skind, sdata, spos in substream: | |
| 892 self.append(skind, sdata, spos) | |
| 893 self.events.setdefault(order, []).append((SUB_END, None, pos)) | |
| 894 elif kind is TEXT: | |
| 895 if '[' in data or ']' in data: | |
| 896 # Quote [ and ] if it ain't us adding it, ie, if the user is | |
| 897 # using those chars in his templates, escape them | |
| 898 data = data.replace('[', '\[').replace(']', '\]') | |
| 738 | 899 self.string.append(data) |
| 849 | 900 self.events.setdefault(self.stack[-1], []).append((kind, data, pos)) |
|
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
901 elif kind is EXPR: |
| 849 | 902 if self.params: |
| 903 param = self.params.pop(0) | |
| 904 else: | |
| 905 params = ', '.join(['"%s"' % p for p in self.orig_params if p]) | |
| 906 if params: | |
| 907 params = "(%s)" % params | |
| 908 raise IndexError("%d parameters%s given to 'i18n:%s' but " | |
| 909 "%d or more expressions used in '%s', line %s" | |
| 910 % (len(self.orig_params), params, | |
| 911 self.directive.tagname, | |
| 912 len(self.orig_params)+1, | |
| 913 os.path.basename(pos[0] or | |
| 914 'In Memmory Template'), | |
| 915 pos[1])) | |
|
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
916 self.string.append('%%(%s)s' % param) |
| 849 | 917 self.events.setdefault(self.stack[-1], []).append((kind, data, pos)) |
|
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
918 self.values[param] = (kind, data, pos) |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
919 else: |
| 849 | 920 if kind is START: |
| 738 | 921 self.string.append(u'[%d:' % self.order) |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
922 self.stack.append(self.order) |
| 849 | 923 self.events.setdefault(self.stack[-1], |
| 924 []).append((kind, data, pos)) | |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
925 self.depth += 1 |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
926 self.order += 1 |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
927 elif kind is END: |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
928 self.depth -= 1 |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
929 if self.depth: |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
930 self.events[self.stack[-1]].append((kind, data, pos)) |
| 738 | 931 self.string.append(u']') |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
932 self.stack.pop() |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
933 |
|
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
934 def format(self): |
| 738 | 935 """Return a message identifier representing the content in the |
| 936 buffer. | |
| 937 """ | |
| 938 return u''.join(self.string).strip() | |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
939 |
|
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
940 def translate(self, string, regex=re.compile(r'%\((\w+)\)s')): |
| 738 | 941 """Interpolate the given message translation with the events in the |
| 942 buffer and return the translated stream. | |
| 943 | |
| 944 :param string: the translated message string | |
| 945 """ | |
| 849 | 946 substream = None |
| 947 | |
| 948 def yield_parts(string): | |
| 949 for idx, part in enumerate(regex.split(string)): | |
| 950 if idx % 2: | |
| 951 yield self.values[part] | |
| 952 elif part: | |
| 953 yield (TEXT, | |
| 954 part.replace('\[', '[').replace('\]', ']'), | |
| 955 (None, -1, -1) | |
| 956 ) | |
| 957 | |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
958 parts = parse_msg(string) |
| 849 | 959 parts_counter = {} |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
960 for order, string in parts: |
| 849 | 961 parts_counter.setdefault(order, []).append(None) |
| 962 | |
| 963 while parts: | |
| 964 order, string = parts.pop(0) | |
| 965 if len(parts_counter[order]) == 1: | |
| 966 events = self.events[order] | |
| 967 else: | |
| 968 events = [self.events[order].pop(0)] | |
| 969 parts_counter[order].pop() | |
| 970 | |
| 971 for event in events: | |
| 972 if event[0] is SUB_START: | |
| 973 substream = [] | |
| 974 elif event[0] is SUB_END: | |
| 975 # Yield a substream which might have directives to be | |
| 976 # applied to it (after translation events) | |
| 977 yield SUB, (self.subdirectives[order], substream), event[2] | |
| 978 substream = None | |
| 979 elif event[0] is TEXT: | |
| 980 if string: | |
| 981 for part in yield_parts(string): | |
| 982 if substream is not None: | |
| 983 substream.append(part) | |
| 984 else: | |
| 985 yield part | |
| 986 # String handled, reset it | |
| 987 string = None | |
| 988 elif event[0] is START: | |
| 989 if substream is not None: | |
| 990 substream.append(event) | |
| 991 else: | |
| 992 yield event | |
| 993 if string: | |
| 994 for part in yield_parts(string): | |
| 995 if substream is not None: | |
| 996 substream.append(part) | |
| 997 else: | |
| 998 yield part | |
| 999 # String handled, reset it | |
| 1000 string = None | |
| 1001 elif event[0] is END: | |
| 1002 if string: | |
| 1003 for part in yield_parts(string): | |
| 1004 if substream is not None: | |
| 1005 substream.append(part) | |
| 1006 else: | |
| 1007 yield part | |
| 1008 # String handled, reset it | |
| 1009 string = None | |
| 1010 if substream is not None: | |
| 1011 substream.append(event) | |
| 1012 else: | |
| 1013 yield event | |
| 1014 elif event[0] is EXPR: | |
| 1015 # These are handled on the strings itself | |
| 1016 continue | |
|
775
886934df7fea
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
1017 else: |
| 849 | 1018 if string: |
| 1019 for part in yield_parts(string): | |
| 1020 if substream is not None: | |
| 1021 substream.append(part) | |
| 1022 else: | |
| 1023 yield part | |
| 1024 # String handled, reset it | |
| 1025 string = None | |
| 1026 if substream is not None: | |
| 1027 substream.append(event) | |
| 1028 else: | |
| 1029 yield event | |
|
560
7e83be231f96
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1030 |
| 849 | 1031 def parse_msg(string, regex=re.compile(r'(?:\[(\d+)\:)|(?<!\\)\]')): |
| 738 | 1032 """Parse a translated message using Genshi mixed content message |
| 1033 formatting. | |
| 849 | 1034 |
| 738 | 1035 >>> parse_msg("See [1:Help].") |
| 1036 [(0, 'See '), (1, 'Help'), (0, '.')] | |
| 849 | 1037 |
| 738 | 1038 >>> parse_msg("See [1:our [2:Help] page] for details.") |
| 1039 [(0, 'See '), (1, 'our '), (2, 'Help'), (1, ' page'), (0, ' for details.')] | |
| 849 | 1040 |
| 738 | 1041 >>> parse_msg("[2:Details] finden Sie in [1:Hilfe].") |
| 1042 [(2, 'Details'), (0, ' finden Sie in '), (1, 'Hilfe'), (0, '.')] | |
| 849 | 1043 |
| 738 | 1044 >>> parse_msg("[1:] Bilder pro Seite anzeigen.") |
| 1045 [(1, ''), (0, ' Bilder pro Seite anzeigen.')] | |
| 849 | 1046 |
| 738 | 1047 :param string: the translated message string |
| 1048 :return: a list of ``(order, string)`` tuples | |
| 1049 :rtype: `list` | |
| 1050 """ | |
| 1051 parts = [] | |
| 1052 stack = [0] | |
| 1053 while True: | |
| 1054 mo = regex.search(string) | |
| 1055 if not mo: | |
| 1056 break | |
| 1057 | |
| 1058 if mo.start() or stack[-1]: | |
| 1059 parts.append((stack[-1], string[:mo.start()])) | |
| 1060 string = string[mo.end():] | |
| 1061 | |
| 1062 orderno = mo.group(1) | |
| 1063 if orderno is not None: | |
| 1064 stack.append(int(orderno)) | |
| 1065 else: | |
| 1066 stack.pop() | |
| 1067 if not stack: | |
| 1068 break | |
| 1069 | |
| 1070 if string: | |
| 1071 parts.append((stack[-1], string)) | |
| 1072 | |
| 1073 return parts | |
| 1074 | |
|
776
ddb58d74c8ee
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
1075 |
|
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1076 def extract_from_code(code, gettext_functions): |
|
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1077 """Extract strings from Python bytecode. |
|
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1078 |
|
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1079 >>> from genshi.template.eval import Expression |
|
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1080 >>> expr = Expression('_("Hello")') |
|
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1081 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS)) |
|
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1082 [('_', u'Hello')] |
| 849 | 1083 |
|
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1084 >>> expr = Expression('ngettext("You have %(num)s item", ' |
|
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1085 ... '"You have %(num)s items", num)') |
|
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1086 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS)) |
|
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1087 [('ngettext', (u'You have %(num)s item', u'You have %(num)s items', None))] |
|
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1088 |
|
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1089 :param code: the `Code` object |
|
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1090 :type code: `genshi.template.eval.Code` |
|
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1091 :param gettext_functions: a sequence of function names |
| 576 | 1092 :since: version 0.5 |
|
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1093 """ |
|
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1094 def _walk(node): |
|
794
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1095 if isinstance(node, _ast.Call) and isinstance(node.func, _ast.Name) \ |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1096 and node.func.id in gettext_functions: |
|
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1097 strings = [] |
|
600
23a4784203ae
Handle starargs and dstarargs in the I18n extraction code.
cmlenz
parents:
596
diff
changeset
|
1098 def _add(arg): |
|
794
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1099 if isinstance(arg, _ast.Str) and isinstance(arg.s, basestring): |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1100 strings.append(unicode(arg.s, 'utf-8')) |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1101 elif arg: |
|
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1102 strings.append(None) |
|
600
23a4784203ae
Handle starargs and dstarargs in the I18n extraction code.
cmlenz
parents:
596
diff
changeset
|
1103 [_add(arg) for arg in node.args] |
|
794
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1104 _add(node.starargs) |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1105 _add(node.kwargs) |
|
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1106 if len(strings) == 1: |
|
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1107 strings = strings[0] |
|
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1108 else: |
|
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1109 strings = tuple(strings) |
|
794
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1110 yield node.func.id, strings |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1111 elif node._fields: |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1112 children = [] |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1113 for field in node._fields: |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1114 child = getattr(node, field, None) |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1115 if isinstance(child, list): |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1116 for elem in child: |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1117 children.append(elem) |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1118 elif isinstance(child, _ast.AST): |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1119 children.append(child) |
|
f9e23d472a6e
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1120 for child in children: |
|
565
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1121 for funcname, strings in _walk(child): |
|
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1122 yield funcname, strings |
|
53b37e4f2921
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1123 return _walk(code.ast) |
|
561
21b473243e63
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1124 |
|
776
ddb58d74c8ee
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
1125 |
|
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1126 def extract(fileobj, keywords, comment_tags, options): |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1127 """Babel extraction method for Genshi templates. |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1128 |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1129 :param fileobj: the file-like object the messages should be extracted from |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1130 :param keywords: a list of keywords (i.e. function names) that should be |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1131 recognized as translation functions |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1132 :param comment_tags: a list of translator tags to search for and include |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1133 in the results |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1134 :param options: a dictionary of additional options (optional) |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1135 :return: an iterator over ``(lineno, funcname, message, comments)`` tuples |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1136 :rtype: ``iterator`` |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1137 """ |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1138 template_class = options.get('template_class', MarkupTemplate) |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1139 if isinstance(template_class, basestring): |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1140 module, clsname = template_class.split(':', 1) |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1141 template_class = getattr(__import__(module, {}, {}, [clsname]), clsname) |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1142 encoding = options.get('encoding', None) |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1143 |
|
596
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1144 extract_text = options.get('extract_text', True) |
|
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1145 if isinstance(extract_text, basestring): |
|
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1146 extract_text = extract_text.lower() in ('1', 'on', 'yes', 'true') |
|
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1147 |
|
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1148 ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS) |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1149 if isinstance(ignore_tags, basestring): |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1150 ignore_tags = ignore_tags.split() |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1151 ignore_tags = [QName(tag) for tag in ignore_tags] |
|
596
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1152 |
|
528
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1153 include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS) |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1154 if isinstance(include_attrs, basestring): |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1155 include_attrs = include_attrs.split() |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1156 include_attrs = [QName(attr) for attr in include_attrs] |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1157 |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1158 tmpl = template_class(fileobj, filename=getattr(fileobj, 'name', None), |
|
24df908da22d
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1159 encoding=encoding) |
| 849 | 1160 |
|
596
badb5e5b7bb9
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1161 translator = Translator(None, ignore_tags, include_attrs, extract_text) |
| 849 | 1162 if hasattr(tmpl, 'add_directives'): |
| 1163 tmpl.add_directives(Translator.NAMESPACE, translator) | |
|
787
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
1164 for message in translator.extract(tmpl.stream, gettext_functions=keywords): |
|
d7366797440f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
1165 yield message |