Mercurial > genshi > genshi-test
annotate genshi/filters/i18n.py @ 888:18dee397f8e1
More i18n doc improvements.
| author | cmlenz |
|---|---|
| date | Mon, 19 Apr 2010 11:45:40 +0000 |
| parents | 6f49c23045b1 |
| children | b40dbfee9ba6 |
| rev | line source |
|---|---|
| 531 | 1 # -*- coding: utf-8 -*- |
| 2 # | |
|
850
47297fd93363
Update copyright years on files changed due to advanced-i18n merge.
cmlenz
parents:
849
diff
changeset
|
3 # Copyright (C) 2007-2009 Edgewall Software |
| 531 | 4 # All rights reserved. |
| 5 # | |
| 6 # This software is licensed as described in the file COPYING, which | |
| 7 # you should have received as part of this distribution. The terms | |
| 8 # are also available at http://genshi.edgewall.org/wiki/License. | |
| 9 # | |
| 10 # This software consists of voluntary contributions made by many | |
| 11 # individuals. For the exact contribution history, see the revision | |
| 12 # history and logs, available at http://genshi.edgewall.org/log/. | |
| 13 | |
| 849 | 14 """Directives and utilities for internationalization and localization of |
| 15 templates. | |
| 576 | 16 |
| 17 :since: version 0.4 | |
| 849 | 18 :note: Directives support added since version 0.6 |
| 576 | 19 """ |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
20 |
| 856 | 21 try: |
| 22 any | |
| 23 except NameError: | |
| 24 from genshi.util import any | |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
25 from gettext import NullTranslations |
| 849 | 26 import os |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
27 import re |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
28 from types import FunctionType |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
29 |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
30 from genshi.core import Attrs, Namespace, QName, START, END, TEXT, START_NS, \ |
| 849 | 31 END_NS, XML_NAMESPACE, _ensure, StreamEventKind |
|
794
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
32 from genshi.template.eval import _ast |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
33 from genshi.template.base import DirectiveFactory, EXPR, SUB, _apply_directives |
| 849 | 34 from genshi.template.directives import Directive, StripDirective |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
35 from genshi.template.markup import MarkupTemplate, EXEC |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
36 |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
37 __all__ = ['Translator', 'extract'] |
|
501
3073ac688651
Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions.
cmlenz
parents:
493
diff
changeset
|
38 __docformat__ = 'restructuredtext en' |
|
3073ac688651
Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions.
cmlenz
parents:
493
diff
changeset
|
39 |
| 849 | 40 |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
41 I18N_NAMESPACE = Namespace('http://genshi.edgewall.org/i18n') |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
42 |
| 849 | 43 MSGBUF = StreamEventKind('MSGBUF') |
| 44 SUB_START = StreamEventKind('SUB_START') | |
| 45 SUB_END = StreamEventKind('SUB_END') | |
| 46 | |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
47 |
| 849 | 48 class I18NDirective(Directive): |
| 49 """Simple interface for i18n directives to support messages extraction.""" | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
50 |
| 849 | 51 def __call__(self, stream, directives, ctxt, **vars): |
| 52 return _apply_directives(stream, directives, ctxt, vars) | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
53 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
54 |
| 849 | 55 class ExtractableI18NDirective(I18NDirective): |
| 56 """Simple interface for directives to support messages extraction.""" | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
57 |
| 849 | 58 def extract(self, stream, comment_stack): |
| 59 raise NotImplementedError | |
| 60 | |
| 61 | |
| 62 class CommentDirective(I18NDirective): | |
| 63 """Implementation of the ``i18n:comment`` template directive which adds | |
| 64 translation comments. | |
| 65 | |
| 66 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 67 ... <p i18n:comment="As in Foo Bar">Foo</p> | |
| 68 ... </html>''') | |
| 69 >>> translator = Translator() | |
| 70 >>> translator.setup(tmpl) | |
| 71 >>> list(translator.extract(tmpl.stream)) | |
| 72 [(2, None, u'Foo', [u'As in Foo Bar'])] | |
| 73 """ | |
| 74 __slots__ = ['comment'] | |
| 75 | |
| 76 def __init__(self, value, template, hints=None, namespaces=None, | |
| 77 lineno=-1, offset=-1): | |
| 78 Directive.__init__(self, None, template, namespaces, lineno, offset) | |
| 79 self.comment = value | |
| 80 | |
| 81 | |
| 82 class MsgDirective(ExtractableI18NDirective): | |
| 83 r"""Implementation of the ``i18n:msg`` directive which marks inner content | |
| 84 as translatable. Consider the following examples: | |
| 85 | |
| 86 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 87 ... <div i18n:msg=""> | |
| 88 ... <p>Foo</p> | |
| 89 ... <p>Bar</p> | |
| 90 ... </div> | |
| 91 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
| 92 ... </html>''') | |
| 93 | |
| 94 >>> translator = Translator() | |
| 95 >>> translator.setup(tmpl) | |
| 96 >>> list(translator.extract(tmpl.stream)) | |
| 97 [(2, None, u'[1:Foo]\n [2:Bar]', []), (6, None, u'Foo [1:bar]!', [])] | |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
98 >>> print(tmpl.generate().render()) |
| 849 | 99 <html> |
| 100 <div><p>Foo</p> | |
| 101 <p>Bar</p></div> | |
| 102 <p>Foo <em>bar</em>!</p> | |
| 103 </html> | |
| 104 | |
| 105 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 106 ... <div i18n:msg="fname, lname"> | |
| 107 ... <p>First Name: ${fname}</p> | |
| 108 ... <p>Last Name: ${lname}</p> | |
| 109 ... </div> | |
| 110 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
| 111 ... </html>''') | |
| 112 >>> translator.setup(tmpl) | |
| 113 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
| 114 [(2, None, u'[1:First Name: %(fname)s]\n [2:Last Name: %(lname)s]', []), | |
| 115 (6, None, u'Foo [1:bar]!', [])] | |
| 116 | |
| 117 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 118 ... <div i18n:msg="fname, lname"> | |
| 119 ... <p>First Name: ${fname}</p> | |
| 120 ... <p>Last Name: ${lname}</p> | |
| 121 ... </div> | |
| 122 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
| 123 ... </html>''') | |
| 124 >>> translator.setup(tmpl) | |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
125 >>> print(tmpl.generate(fname='John', lname='Doe').render()) |
| 849 | 126 <html> |
| 127 <div><p>First Name: John</p> | |
| 128 <p>Last Name: Doe</p></div> | |
| 129 <p>Foo <em>bar</em>!</p> | |
| 130 </html> | |
| 131 | |
| 132 Starting and ending white-space is stripped of to make it simpler for | |
| 133 translators. Stripping it is not that important since it's on the html | |
| 134 source, the rendered output will remain the same. | |
| 135 """ | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
136 __slots__ = ['params'] |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
137 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
138 def __init__(self, value, template, hints=None, namespaces=None, |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
139 lineno=-1, offset=-1): |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
140 Directive.__init__(self, None, template, namespaces, lineno, offset) |
| 849 | 141 self.params = [param.strip() for param in value.split(',') if param] |
| 142 | |
| 143 @classmethod | |
| 144 def attach(cls, template, stream, value, namespaces, pos): | |
| 145 if type(value) is dict: | |
| 146 value = value.get('params', '').strip() | |
| 147 return super(MsgDirective, cls).attach(template, stream, value.strip(), | |
| 148 namespaces, pos) | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
149 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
150 def __call__(self, stream, directives, ctxt, **vars): |
| 849 | 151 gettext = ctxt.get('_i18n.gettext') |
| 152 dgettext = ctxt.get('_i18n.dgettext') | |
| 153 if ctxt.get('_i18n.domain'): | |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
154 assert hasattr(dgettext, '__call__'), \ |
|
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
155 'No domain gettext function passed' |
| 849 | 156 gettext = lambda msg: dgettext(ctxt.get('_i18n.domain'), msg) |
| 157 | |
| 158 def _generate(): | |
| 159 msgbuf = MessageBuffer(self) | |
| 160 previous = stream.next() | |
| 161 if previous[0] is START: | |
| 162 yield previous | |
| 163 else: | |
| 164 msgbuf.append(*previous) | |
| 165 previous = stream.next() | |
| 166 for kind, data, pos in stream: | |
| 167 msgbuf.append(*previous) | |
| 168 previous = kind, data, pos | |
| 169 if previous[0] is not END: | |
| 170 msgbuf.append(*previous) | |
| 171 previous = None | |
| 172 for event in msgbuf.translate(gettext(msgbuf.format())): | |
| 173 yield event | |
| 174 if previous: | |
| 175 yield previous | |
| 176 | |
| 177 return _apply_directives(_generate(), directives, ctxt, vars) | |
| 178 | |
| 179 def extract(self, stream, comment_stack): | |
| 180 msgbuf = MessageBuffer(self) | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
181 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
182 stream = iter(stream) |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
183 previous = stream.next() |
| 849 | 184 if previous[0] is START: |
| 185 previous = stream.next() | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
186 for event in stream: |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
187 msgbuf.append(*previous) |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
188 previous = event |
|
869
38c44e2f4232
Apply patch for I18n message extraction bug as reported in #358. Thanks to cboos for the patch, again.
cmlenz
parents:
856
diff
changeset
|
189 msgbuf.append(*previous) |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
190 |
| 849 | 191 yield None, msgbuf.format(), comment_stack[-1:] |
| 192 | |
| 193 | |
| 194 class ChooseBranchDirective(I18NDirective): | |
| 195 __slots__ = ['params'] | |
|
871
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
196 |
| 849 | 197 def __call__(self, stream, directives, ctxt, **vars): |
| 198 self.params = ctxt.get('_i18n.choose.params', [])[:] | |
| 199 msgbuf = MessageBuffer(self) | |
| 200 | |
| 201 stream = iter(_apply_directives(stream, directives, ctxt, vars)) | |
|
871
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
202 |
| 849 | 203 previous = stream.next() |
|
871
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
204 if previous[0] is START: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
205 yield previous |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
206 else: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
207 msgbuf.append(*previous) |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
208 |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
209 try: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
210 previous = stream.next() |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
211 except StopIteration: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
212 # For example <i18n:singular> or <i18n:plural> directives |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
213 yield MSGBUF, (), -1 # the place holder for msgbuf output |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
214 ctxt['_i18n.choose.%s' % type(self).__name__] = msgbuf |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
215 return |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
216 |
| 849 | 217 for kind, data, pos in stream: |
| 218 msgbuf.append(*previous) | |
| 219 previous = kind, data, pos | |
| 220 yield MSGBUF, (), -1 # the place holder for msgbuf output | |
|
871
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
221 |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
222 if previous[0] is END: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
223 yield previous # the outer end tag |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
224 else: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
225 msgbuf.append(*previous) |
| 849 | 226 ctxt['_i18n.choose.%s' % type(self).__name__] = msgbuf |
| 227 | |
| 228 | |
| 229 def extract(self, stream, comment_stack, msgbuf): | |
| 230 stream = iter(stream) | |
| 231 previous = stream.next() | |
| 232 if previous[0] is START: | |
| 233 previous = stream.next() | |
| 234 for event in stream: | |
| 235 msgbuf.append(*previous) | |
| 236 previous = event | |
| 237 if previous[0] is not END: | |
| 238 msgbuf.append(*previous) | |
| 239 return msgbuf | |
| 240 | |
| 241 | |
| 242 class SingularDirective(ChooseBranchDirective): | |
| 243 """Implementation of the ``i18n:singular`` directive to be used with the | |
| 244 ``i18n:choose`` directive.""" | |
| 245 | |
| 246 | |
| 247 class PluralDirective(ChooseBranchDirective): | |
| 248 """Implementation of the ``i18n:plural`` directive to be used with the | |
| 249 ``i18n:choose`` directive.""" | |
| 250 | |
| 251 | |
| 252 class ChooseDirective(ExtractableI18NDirective): | |
| 253 """Implementation of the ``i18n:choose`` directive which provides plural | |
| 254 internationalisation of strings. | |
| 255 | |
| 256 This directive requires at least one parameter, the one which evaluates to | |
| 257 an integer which will allow to choose the plural/singular form. If you also | |
| 258 have expressions inside the singular and plural version of the string you | |
| 259 also need to pass a name for those parameters. Consider the following | |
| 260 examples: | |
| 261 | |
| 262 >>> tmpl = MarkupTemplate('''\ | |
| 263 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 264 ... <div i18n:choose="num; num"> | |
| 265 ... <p i18n:singular="">There is $num coin</p> | |
| 266 ... <p i18n:plural="">There are $num coins</p> | |
| 267 ... </div> | |
| 268 ... </html>''') | |
| 269 >>> translator = Translator() | |
| 270 >>> translator.setup(tmpl) | |
| 271 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
| 272 [(2, 'ngettext', (u'There is %(num)s coin', | |
| 273 u'There are %(num)s coins'), [])] | |
| 274 | |
| 275 >>> tmpl = MarkupTemplate('''\ | |
| 276 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 277 ... <div i18n:choose="num; num"> | |
| 278 ... <p i18n:singular="">There is $num coin</p> | |
| 279 ... <p i18n:plural="">There are $num coins</p> | |
| 280 ... </div> | |
| 281 ... </html>''') | |
| 282 >>> translator.setup(tmpl) | |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
283 >>> print(tmpl.generate(num=1).render()) |
| 849 | 284 <html> |
| 285 <div> | |
| 286 <p>There is 1 coin</p> | |
| 287 </div> | |
| 288 </html> | |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
289 >>> print(tmpl.generate(num=2).render()) |
| 849 | 290 <html> |
| 291 <div> | |
| 292 <p>There are 2 coins</p> | |
| 293 </div> | |
| 294 </html> | |
| 295 | |
| 888 | 296 When used as a element and not as an attribute: |
| 849 | 297 |
| 298 >>> tmpl = MarkupTemplate('''\ | |
| 299 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 300 ... <i18n:choose numeral="num" params="num"> | |
| 301 ... <p i18n:singular="">There is $num coin</p> | |
| 302 ... <p i18n:plural="">There are $num coins</p> | |
| 303 ... </i18n:choose> | |
| 304 ... </html>''') | |
| 305 >>> translator.setup(tmpl) | |
| 306 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
| 307 [(2, 'ngettext', (u'There is %(num)s coin', | |
| 308 u'There are %(num)s coins'), [])] | |
| 309 """ | |
| 310 __slots__ = ['numeral', 'params'] | |
| 311 | |
| 312 def __init__(self, value, template, hints=None, namespaces=None, | |
| 313 lineno=-1, offset=-1): | |
| 314 Directive.__init__(self, None, template, namespaces, lineno, offset) | |
| 315 params = [v.strip() for v in value.split(';')] | |
| 316 self.numeral = self._parse_expr(params.pop(0), template, lineno, offset) | |
| 317 self.params = params and [name.strip() for name in | |
| 318 params[0].split(',') if name] or [] | |
| 319 | |
| 320 @classmethod | |
| 321 def attach(cls, template, stream, value, namespaces, pos): | |
| 322 if type(value) is dict: | |
| 323 numeral = value.get('numeral', '').strip() | |
| 324 assert numeral is not '', "at least pass the numeral param" | |
| 325 params = [v.strip() for v in value.get('params', '').split(',')] | |
| 326 value = '%s; ' % numeral + ', '.join(params) | |
| 327 return super(ChooseDirective, cls).attach(template, stream, value, | |
| 328 namespaces, pos) | |
| 329 | |
| 330 def __call__(self, stream, directives, ctxt, **vars): | |
| 331 ctxt.push({'_i18n.choose.params': self.params, | |
| 332 '_i18n.choose.SingularDirective': None, | |
| 333 '_i18n.choose.PluralDirective': None}) | |
| 334 | |
| 335 new_stream = [] | |
| 336 singular_stream = None | |
| 337 singular_msgbuf = None | |
| 338 plural_stream = None | |
| 339 plural_msgbuf = None | |
| 340 | |
| 341 ngettext = ctxt.get('_i18n.ungettext') | |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
342 assert hasattr(ngettext, '__call__'), 'No ngettext function available' |
| 849 | 343 dngettext = ctxt.get('_i18n.dngettext') |
| 344 if not dngettext: | |
| 345 dngettext = lambda d, s, p, n: ngettext(s, p, n) | |
| 346 for kind, event, pos in stream: | |
| 347 if kind is SUB: | |
| 348 subdirectives, substream = event | |
| 349 if isinstance(subdirectives[0], | |
| 350 SingularDirective) and not singular_stream: | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
351 strip_directive_present = [] |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
352 for idx, subdirective in enumerate(subdirectives): |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
353 if isinstance(subdirective, StripDirective): |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
354 # Any strip directive should be applied AFTER |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
355 # the event's have been translated. |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
356 strip_directive_present.append( |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
357 subdirectives.pop(idx) |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
358 ) |
| 849 | 359 # Apply directives to update context |
| 360 singular_stream = list(_apply_directives(substream, | |
| 361 subdirectives, | |
| 362 ctxt, vars)) | |
|
871
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
363 if strip_directive_present: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
364 singular_stream = list( |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
365 _apply_directives(singular_stream, |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
366 strip_directive_present, |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
367 ctxt, vars) |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
368 ) |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
369 del strip_directive_present |
| 849 | 370 new_stream.append((MSGBUF, (), ('', -1))) # msgbuf place holder |
| 371 singular_msgbuf = ctxt.get('_i18n.choose.SingularDirective') | |
| 372 elif isinstance(subdirectives[0], | |
| 373 PluralDirective) and not plural_stream: | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
374 strip_directive_present = [] |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
375 for idx, subdirective in enumerate(subdirectives): |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
376 if isinstance(subdirective, StripDirective): |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
377 # Any strip directive should be applied AFTER |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
378 # the event's have been translated. |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
379 strip_directive_present.append( |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
380 subdirectives.pop(idx) |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
381 ) |
| 849 | 382 # Apply directives to update context |
| 383 plural_stream = list(_apply_directives(substream, | |
| 384 subdirectives, | |
| 385 ctxt, vars)) | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
386 if strip_directive_present: |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
387 plural_stream = list( |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
388 _apply_directives(plural_stream, |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
389 strip_directive_present, |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
390 ctxt, vars) |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
391 ) |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
392 del strip_directive_present |
| 849 | 393 plural_msgbuf = ctxt.get('_i18n.choose.PluralDirective') |
| 394 else: | |
| 395 new_stream.append((kind, event, pos)) | |
| 396 else: | |
| 397 new_stream.append((kind, event, pos)) | |
| 398 | |
| 399 if ctxt.get('_i18n.domain'): | |
| 400 ngettext = lambda s, p, n: dngettext(ctxt.get('_i18n.domain'), | |
| 401 s, p, n) | |
| 402 | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
403 # XXX: should we test which form was chosen like this!?!?!? |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
404 # There should be no match in any catalogue for these singular and |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
405 # plural test strings |
|
882
6f49c23045b1
Fix regression in [1099]: templates must not have an implicit loader during message extraction.
cmlenz
parents:
873
diff
changeset
|
406 singular_test = u'O\x85\xbe\xa9\xa8az\xc3?\xe6\xa1\x02n\x84\x93' |
|
6f49c23045b1
Fix regression in [1099]: templates must not have an implicit loader during message extraction.
cmlenz
parents:
873
diff
changeset
|
407 plural_test = u'\xcc\xfb+\xd3Pn\x9d\tT\xec\x1d\xda\x1a\x88\x00' |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
408 translation = ngettext(singular_test, plural_test, |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
409 self.numeral.evaluate(ctxt)) |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
410 if translation==singular_test: |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
411 chosen_msgbuf = singular_msgbuf |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
412 chosen_stream = singular_stream |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
413 else: |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
414 chosen_msgbuf = plural_msgbuf |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
415 chosen_stream = plural_stream |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
416 del singular_test, plural_test, translation |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
417 |
| 849 | 418 for kind, data, pos in new_stream: |
| 419 if kind is MSGBUF: | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
420 for skind, sdata, spos in chosen_stream: |
| 849 | 421 if skind is MSGBUF: |
| 422 translation = ngettext(singular_msgbuf.format(), | |
| 423 plural_msgbuf.format(), | |
| 424 self.numeral.evaluate(ctxt)) | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
425 for event in chosen_msgbuf.translate(translation): |
| 849 | 426 yield event |
| 427 else: | |
| 428 yield skind, sdata, spos | |
| 429 else: | |
| 430 yield kind, data, pos | |
| 431 | |
| 432 ctxt.pop() | |
| 433 | |
| 434 def extract(self, stream, comment_stack): | |
| 435 stream = iter(stream) | |
| 436 previous = stream.next() | |
| 437 if previous is START: | |
| 438 stream.next() | |
| 439 | |
| 440 singular_msgbuf = MessageBuffer(self) | |
| 441 plural_msgbuf = MessageBuffer(self) | |
| 442 | |
| 443 for kind, event, pos in stream: | |
| 444 if kind is SUB: | |
| 445 subdirectives, substream = event | |
| 446 for subdirective in subdirectives: | |
| 447 if isinstance(subdirective, SingularDirective): | |
| 448 singular_msgbuf = subdirective.extract(substream, comment_stack, | |
| 449 singular_msgbuf) | |
| 450 elif isinstance(subdirective, PluralDirective): | |
| 451 plural_msgbuf = subdirective.extract(substream, comment_stack, | |
| 452 plural_msgbuf) | |
| 453 elif not isinstance(subdirective, StripDirective): | |
| 454 singular_msgbuf.append(kind, event, pos) | |
| 455 plural_msgbuf.append(kind, event, pos) | |
| 456 else: | |
| 457 singular_msgbuf.append(kind, event, pos) | |
| 458 plural_msgbuf.append(kind, event, pos) | |
| 459 | |
| 460 yield 'ngettext', \ | |
| 461 (singular_msgbuf.format(), plural_msgbuf.format()), \ | |
| 462 comment_stack[-1:] | |
| 463 | |
| 464 | |
| 465 class DomainDirective(I18NDirective): | |
| 466 """Implementation of the ``i18n:domain`` directive which allows choosing | |
| 467 another i18n domain(catalog) to translate from. | |
| 468 | |
| 469 >>> from genshi.filters.tests.i18n import DummyTranslations | |
| 470 >>> tmpl = MarkupTemplate('''\ | |
| 471 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 472 ... <p i18n:msg="">Bar</p> | |
| 473 ... <div i18n:domain="foo"> | |
| 474 ... <p i18n:msg="">FooBar</p> | |
| 475 ... <p>Bar</p> | |
| 476 ... <p i18n:domain="bar" i18n:msg="">Bar</p> | |
| 477 ... <p i18n:domain="">Bar</p> | |
| 478 ... </div> | |
| 479 ... <p>Bar</p> | |
| 480 ... </html>''') | |
| 481 | |
| 482 >>> translations = DummyTranslations({'Bar': 'Voh'}) | |
| 483 >>> translations.add_domain('foo', {'FooBar': 'BarFoo', 'Bar': 'foo_Bar'}) | |
| 484 >>> translations.add_domain('bar', {'Bar': 'bar_Bar'}) | |
| 485 >>> translator = Translator(translations) | |
| 486 >>> translator.setup(tmpl) | |
| 487 | |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
488 >>> print(tmpl.generate().render()) |
| 849 | 489 <html> |
| 490 <p>Voh</p> | |
| 491 <div> | |
| 492 <p>BarFoo</p> | |
| 493 <p>foo_Bar</p> | |
| 494 <p>bar_Bar</p> | |
| 495 <p>Voh</p> | |
| 496 </div> | |
| 497 <p>Voh</p> | |
| 498 </html> | |
| 499 """ | |
| 500 __slots__ = ['domain'] | |
| 501 | |
| 502 def __init__(self, value, template, hints=None, namespaces=None, | |
| 503 lineno=-1, offset=-1): | |
| 504 Directive.__init__(self, None, template, namespaces, lineno, offset) | |
| 505 self.domain = value and value.strip() or '__DEFAULT__' | |
| 506 | |
| 507 @classmethod | |
| 508 def attach(cls, template, stream, value, namespaces, pos): | |
| 509 if type(value) is dict: | |
| 510 value = value.get('name') | |
| 511 return super(DomainDirective, cls).attach(template, stream, value, | |
| 512 namespaces, pos) | |
| 513 | |
| 514 def __call__(self, stream, directives, ctxt, **vars): | |
| 515 ctxt.push({'_i18n.domain': self.domain}) | |
| 516 for event in _apply_directives(stream, directives, ctxt, vars): | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
517 yield event |
| 849 | 518 ctxt.pop() |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
519 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
520 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
521 class Translator(DirectiveFactory): |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
522 """Can extract and translate localizable strings from markup streams and |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
523 templates. |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
524 |
| 849 | 525 For example, assume the following template: |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
526 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
527 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/"> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
528 ... <head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
529 ... <title>Example</title> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
530 ... </head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
531 ... <body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
532 ... <h1>Example</h1> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
533 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
534 ... </body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
535 ... </html>''', filename='example.html') |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
536 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
537 For demonstration, we define a dummy ``gettext``-style function with a |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
538 hard-coded translation table, and pass that to the `Translator` initializer: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
539 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
540 >>> def pseudo_gettext(string): |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
541 ... return { |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
542 ... 'Example': 'Beispiel', |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
543 ... 'Hello, %(name)s': 'Hallo, %(name)s' |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
544 ... }[string] |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
545 >>> translator = Translator(pseudo_gettext) |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
546 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
547 Next, the translator needs to be prepended to any already defined filters |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
548 on the template: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
549 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
550 >>> tmpl.filters.insert(0, translator) |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
551 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
552 When generating the template output, our hard-coded translations should be |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
553 applied as expected: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
554 |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
555 >>> print(tmpl.generate(username='Hans', _=pseudo_gettext)) |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
556 <html> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
557 <head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
558 <title>Beispiel</title> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
559 </head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
560 <body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
561 <h1>Beispiel</h1> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
562 <p>Hallo, Hans</p> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
563 </body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
564 </html> |
| 849 | 565 |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
566 Note that elements defining ``xml:lang`` attributes that do not contain |
|
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
567 variable expressions are ignored by this filter. That can be used to |
|
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
568 exclude specific parts of a template from being extracted and translated. |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
569 """ |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
570 |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
571 directives = [ |
| 849 | 572 ('domain', DomainDirective), |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
573 ('comment', CommentDirective), |
| 849 | 574 ('msg', MsgDirective), |
| 575 ('choose', ChooseDirective), | |
| 576 ('singular', SingularDirective), | |
| 577 ('plural', PluralDirective) | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
578 ] |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
579 |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
580 IGNORE_TAGS = frozenset([ |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
581 QName('script'), QName('http://www.w3.org/1999/xhtml}script'), |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
582 QName('style'), QName('http://www.w3.org/1999/xhtml}style') |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
583 ]) |
| 849 | 584 INCLUDE_ATTRS = frozenset([ |
| 585 'abbr', 'alt', 'label', 'prompt', 'standby', 'summary', 'title' | |
| 586 ]) | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
587 NAMESPACE = I18N_NAMESPACE |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
588 |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
589 def __init__(self, translate=NullTranslations(), ignore_tags=IGNORE_TAGS, |
|
594
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
590 include_attrs=INCLUDE_ATTRS, extract_text=True): |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
591 """Initialize the translator. |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
592 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
593 :param translate: the translation function, for example ``gettext`` or |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
594 ``ugettext``. |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
595 :param ignore_tags: a set of tag names that should not be localized |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
596 :param include_attrs: a set of attribute names should be localized |
|
594
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
597 :param extract_text: whether the content of text nodes should be |
|
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
598 extracted, or only text in explicit ``gettext`` |
|
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
599 function calls |
| 849 | 600 |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
601 :note: Changed in 0.6: the `translate` parameter can now be either |
|
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
602 a ``gettext``-style function, or an object compatible with the |
|
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
603 ``NullTransalations`` or ``GNUTranslations`` interface |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
604 """ |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
605 self.translate = translate |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
606 self.ignore_tags = ignore_tags |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
607 self.include_attrs = include_attrs |
|
594
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
608 self.extract_text = extract_text |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
609 |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
610 def __call__(self, stream, ctxt=None, search_text=True): |
| 448 | 611 """Translate any localizable strings in the given stream. |
| 612 | |
| 613 This function shouldn't be called directly. Instead, an instance of | |
| 614 the `Translator` class should be registered as a filter with the | |
| 615 `Template` or the `TemplateLoader`, or applied as a regular stream | |
| 616 filter. If used as a template filter, it should be inserted in front of | |
| 617 all the default filters. | |
| 618 | |
| 619 :param stream: the markup event stream | |
| 620 :param ctxt: the template context (not used) | |
| 621 :param search_text: whether text nodes should be translated (used | |
| 622 internally) | |
| 623 :return: the localized stream | |
| 624 """ | |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
625 ignore_tags = self.ignore_tags |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
626 include_attrs = self.include_attrs |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
627 skip = 0 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
628 xml_lang = XML_NAMESPACE['lang'] |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
629 |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
630 if type(self.translate) is FunctionType: |
|
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
631 gettext = self.translate |
| 849 | 632 if ctxt: |
| 633 ctxt['_i18n.gettext'] = gettext | |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
634 else: |
|
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
635 gettext = self.translate.ugettext |
| 849 | 636 try: |
| 637 dgettext = self.translate.dugettext | |
| 638 except AttributeError: | |
| 639 dgettext = lambda x, y: gettext(y) | |
| 640 ngettext = self.translate.ungettext | |
| 641 try: | |
| 642 dngettext = self.translate.dungettext | |
| 643 except AttributeError: | |
| 644 dngettext = lambda d, s, p, n: ngettext(s, p, n) | |
| 645 | |
| 646 if ctxt: | |
| 647 ctxt['_i18n.gettext'] = gettext | |
| 648 ctxt['_i18n.ugettext'] = gettext | |
| 649 ctxt['_i18n.dgettext'] = dgettext | |
| 650 ctxt['_i18n.ngettext'] = ngettext | |
| 651 ctxt['_i18n.ungettext'] = ngettext | |
| 652 ctxt['_i18n.dngettext'] = dngettext | |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
653 |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
654 extract_text = self.extract_text |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
655 if not extract_text: |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
656 search_text = False |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
657 |
| 849 | 658 if ctxt and ctxt.get('_i18n.domain'): |
| 659 old_gettext = gettext | |
| 660 gettext = lambda msg: dgettext(ctxt.get('_i18n.domain'), msg) | |
| 661 | |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
662 for kind, data, pos in stream: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
663 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
664 # skip chunks that should not be localized |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
665 if skip: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
666 if kind is START: |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
667 skip += 1 |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
668 elif kind is END: |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
669 skip -= 1 |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
670 yield kind, data, pos |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
671 continue |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
672 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
673 # handle different events that can be localized |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
674 if kind is START: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
675 tag, attrs = data |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
676 if tag in self.ignore_tags or \ |
|
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
677 isinstance(attrs.get(xml_lang), basestring): |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
678 skip += 1 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
679 yield kind, data, pos |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
680 continue |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
681 |
|
493
abe6e4ab7ecf
Fix another bug in the translation filter: translated attributes were getting added instead of replaced.
cmlenz
parents:
485
diff
changeset
|
682 new_attrs = [] |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
683 changed = False |
| 849 | 684 |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
685 for name, value in attrs: |
|
483
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
686 newval = value |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
687 if extract_text and isinstance(value, basestring): |
|
483
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
688 if name in include_attrs: |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
689 newval = gettext(value) |
|
483
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
690 else: |
| 849 | 691 newval = list( |
| 692 self(_ensure(value), ctxt, search_text=False) | |
|
483
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
693 ) |
|
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
694 if newval != value: |
|
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
695 value = newval |
|
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
696 changed = True |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
697 new_attrs.append((name, value)) |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
698 if changed: |
|
667
077c9142dca0
Fix case where attributes weren't properly wrapped in an `Attrs` instance if one or more of them were translated by the I18n filter, potentially breaking things further down the chain. Closes #162.
cmlenz
parents:
600
diff
changeset
|
699 attrs = Attrs(new_attrs) |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
700 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
701 yield kind, (tag, attrs), pos |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
702 |
|
485
770ba8556940
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
703 elif search_text and kind is TEXT: |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
704 text = data.strip() |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
705 if text: |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
706 data = data.replace(text, unicode(gettext(text))) |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
707 yield kind, data, pos |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
708 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
709 elif kind is SUB: |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
710 directives, substream = data |
| 849 | 711 current_domain = None |
| 712 for idx, directive in enumerate(directives): | |
| 713 # Organize directives to make everything work | |
| 714 if isinstance(directive, DomainDirective): | |
| 715 # Grab current domain and update context | |
| 716 current_domain = directive.domain | |
| 717 ctxt.push({'_i18n.domain': current_domain}) | |
| 718 # Put domain directive as the first one in order to | |
| 719 # update context before any other directives evaluation | |
| 720 directives.insert(0, directives.pop(idx)) | |
| 721 | |
| 722 # If this is an i18n directive, no need to translate text | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
723 # nodes here |
| 856 | 724 is_i18n_directive = any([ |
| 725 isinstance(d, ExtractableI18NDirective) | |
| 726 for d in directives | |
| 727 ]) | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
728 substream = list(self(substream, ctxt, |
| 849 | 729 search_text=not is_i18n_directive)) |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
730 yield kind, (directives, substream), pos |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
731 |
| 849 | 732 if current_domain: |
| 733 ctxt.pop() | |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
734 else: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
735 yield kind, data, pos |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
736 |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
737 GETTEXT_FUNCTIONS = ('_', 'gettext', 'ngettext', 'dgettext', 'dngettext', |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
738 'ugettext', 'ungettext') |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
739 |
|
485
770ba8556940
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
740 def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS, |
| 849 | 741 search_text=True, msgbuf=None, comment_stack=None): |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
742 """Extract localizable strings from the given template stream. |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
743 |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
744 For every string found, this function yields a ``(lineno, function, |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
745 message, comments)`` tuple, where: |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
746 |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
747 * ``lineno`` is the number of the line on which the string was found, |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
748 * ``function`` is the name of the ``gettext`` function used (if the |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
749 string was extracted from embedded Python code), and |
|
469
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
750 * ``message`` is the string itself (a ``unicode`` object, or a tuple |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
751 of ``unicode`` objects for functions with multiple string |
|
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
752 arguments). |
|
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
753 * ``comments`` is a list of comments related to the message, extracted |
|
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
754 from ``i18n:comment`` attributes found in the markup |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
755 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
756 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/"> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
757 ... <head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
758 ... <title>Example</title> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
759 ... </head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
760 ... <body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
761 ... <h1>Example</h1> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
762 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p> |
|
469
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
763 ... <p>${ngettext("You have %d item", "You have %d items", num)}</p> |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
764 ... </body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
765 ... </html>''', filename='example.html') |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
766 >>> for line, func, msg, comments in Translator().extract(tmpl.stream): |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
767 ... print('%d, %r, %r' % (line, func, msg)) |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
768 3, None, u'Example' |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
769 6, None, u'Example' |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
770 7, '_', u'Hello, %(name)s' |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
771 8, 'ngettext', (u'You have %d item', u'You have %d items', None) |
|
469
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
772 |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
773 :param stream: the event stream to extract strings from; can be a |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
774 regular stream or a template stream |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
775 :param gettext_functions: a sequence of function names that should be |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
776 treated as gettext-style localization |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
777 functions |
|
485
770ba8556940
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
778 :param search_text: whether the content of text nodes should be |
|
770ba8556940
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
779 extracted (used internally) |
|
469
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
780 |
|
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
781 :note: Changed in 0.4.1: For a function with multiple string arguments |
|
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
782 (such as ``ngettext``), a single item with a tuple of strings is |
|
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
783 yielded, instead an item for each string argument. |
| 849 | 784 :note: Changed in 0.6: The returned tuples now include a fourth |
| 785 element, which is a list of comments for the translator. | |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
786 """ |
|
594
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
787 if not self.extract_text: |
|
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
788 search_text = False |
| 849 | 789 if comment_stack is None: |
| 790 comment_stack = [] | |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
791 skip = 0 |
| 849 | 792 |
| 793 # Un-comment bellow to extract messages without adding directives | |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
794 xml_lang = XML_NAMESPACE['lang'] |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
795 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
796 for kind, data, pos in stream: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
797 if skip: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
798 if kind is START: |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
799 skip += 1 |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
800 if kind is END: |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
801 skip -= 1 |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
802 |
|
549
da5cbf6d134d
The I18n filter now extracts text from translation functions in ignored tags. Fixes #132.
cmlenz
parents:
535
diff
changeset
|
803 if kind is START and not skip: |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
804 tag, attrs = data |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
805 |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
806 if tag in self.ignore_tags or \ |
|
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
807 isinstance(attrs.get(xml_lang), basestring): |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
808 skip += 1 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
809 continue |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
810 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
811 for name, value in attrs: |
|
594
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
812 if search_text and isinstance(value, basestring): |
|
483
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
813 if name in self.include_attrs: |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
814 text = value.strip() |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
815 if text: |
| 849 | 816 # XXX: Do we need to grab i18n:comment from comment_stack ??? |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
817 yield pos[1], None, text, [] |
|
483
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
818 else: |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
819 for lineno, funcname, text, comments in self.extract( |
|
485
770ba8556940
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
820 _ensure(value), gettext_functions, |
|
535
c8806216aec4
The I18n filter no longer extracts or translates literal strings in attribute values that also contain expressions.
cmlenz
parents:
531
diff
changeset
|
821 search_text=False): |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
822 yield lineno, funcname, text, comments |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
823 |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
824 if msgbuf: |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
825 msgbuf.append(kind, data, pos) |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
826 |
|
549
da5cbf6d134d
The I18n filter now extracts text from translation functions in ignored tags. Fixes #132.
cmlenz
parents:
535
diff
changeset
|
827 elif not skip and search_text and kind is TEXT: |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
828 if not msgbuf: |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
829 text = data.strip() |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
830 if text and [ch for ch in text if ch.isalpha()]: |
| 849 | 831 yield pos[1], None, text, comment_stack[-1:] |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
832 else: |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
833 msgbuf.append(kind, data, pos) |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
834 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
835 elif not skip and msgbuf and kind is END: |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
836 msgbuf.append(kind, data, pos) |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
837 if not msgbuf.depth: |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
838 yield msgbuf.lineno, None, msgbuf.format(), [ |
|
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
839 c for c in msgbuf.comment if c |
|
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
840 ] |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
841 msgbuf = None |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
842 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
843 elif kind is EXPR or kind is EXEC: |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
844 if msgbuf: |
|
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
845 msgbuf.append(kind, data, pos) |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
846 for funcname, strings in extract_from_code(data, |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
847 gettext_functions): |
| 849 | 848 # XXX: Do we need to grab i18n:comment from comment_stack ??? |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
849 yield pos[1], funcname, strings, [] |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
850 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
851 elif kind is SUB: |
| 849 | 852 directives, substream = data |
| 853 in_comment = False | |
| 854 | |
| 855 for idx, directive in enumerate(directives): | |
| 856 # Do a first loop to see if there's a comment directive | |
| 857 # If there is update context and pop it from directives | |
| 858 if isinstance(directive, CommentDirective): | |
| 859 in_comment = True | |
| 860 comment_stack.append(directive.comment) | |
| 861 if len(directives) == 1: | |
| 862 # in case we're in the presence of something like: | |
| 863 # <p i18n:comment="foo">Foo</p> | |
| 864 messages = self.extract( | |
| 865 substream, gettext_functions, | |
| 866 search_text=search_text and not skip, | |
| 867 msgbuf=msgbuf, comment_stack=comment_stack) | |
| 868 for lineno, funcname, text, comments in messages: | |
| 869 yield lineno, funcname, text, comments | |
| 870 directives.pop(idx) | |
| 871 elif not isinstance(directive, I18NDirective): | |
| 872 # Remove all other non i18n directives from the process | |
| 873 directives.pop(idx) | |
| 874 | |
| 875 if not directives and not in_comment: | |
| 876 # Extract content if there's no directives because | |
| 877 # strip was pop'ed and not because comment was pop'ed. | |
| 878 # Extraction in this case has been taken care of. | |
| 879 messages = self.extract( | |
| 880 substream, gettext_functions, | |
| 881 search_text=search_text and not skip, msgbuf=msgbuf) | |
| 882 for lineno, funcname, text, comments in messages: | |
| 883 yield lineno, funcname, text, comments | |
| 884 | |
| 885 for directive in directives: | |
| 886 if isinstance(directive, ExtractableI18NDirective): | |
| 887 messages = directive.extract(substream, comment_stack) | |
| 888 for funcname, text, comments in messages: | |
| 889 yield pos[1], funcname, text, comments | |
| 890 else: | |
| 891 messages = self.extract( | |
| 892 substream, gettext_functions, | |
| 893 search_text=search_text and not skip, msgbuf=msgbuf) | |
| 894 for lineno, funcname, text, comments in messages: | |
| 895 yield lineno, funcname, text, comments | |
| 896 | |
| 897 if in_comment: | |
| 898 comment_stack.pop() | |
| 899 | |
| 900 def get_directive_index(self, dir_cls): | |
| 901 total = len(self._dir_order) | |
| 902 if dir_cls in self._dir_order: | |
| 903 return self._dir_order.index(dir_cls) - total | |
| 904 return total | |
| 905 | |
| 906 def setup(self, template): | |
| 907 """Convenience function to register the `Translator` filter and the | |
| 908 related directives with the given template. | |
| 909 | |
| 910 :param template: a `Template` instance | |
| 911 """ | |
| 912 template.filters.insert(0, self) | |
| 913 if hasattr(template, 'add_directives'): | |
| 914 template.add_directives(Translator.NAMESPACE, self) | |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
915 |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
916 |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
917 class MessageBuffer(object): |
| 738 | 918 """Helper class for managing internationalized mixed content. |
| 576 | 919 |
| 920 :since: version 0.5 | |
| 921 """ | |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
922 |
| 849 | 923 def __init__(self, directive=None): |
| 738 | 924 """Initialize the message buffer. |
| 925 | |
|
882
6f49c23045b1
Fix regression in [1099]: templates must not have an implicit loader during message extraction.
cmlenz
parents:
873
diff
changeset
|
926 :param directive: the directive owning the buffer |
|
6f49c23045b1
Fix regression in [1099]: templates must not have an implicit loader during message extraction.
cmlenz
parents:
873
diff
changeset
|
927 :type directive: I18NDirective |
| 738 | 928 """ |
| 849 | 929 # params list needs to be copied so that directives can be evaluated |
| 930 # more than once | |
| 931 self.orig_params = self.params = directive.params[:] | |
| 932 self.directive = directive | |
| 738 | 933 self.string = [] |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
934 self.events = {} |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
935 self.values = {} |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
936 self.depth = 1 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
937 self.order = 1 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
938 self.stack = [0] |
| 849 | 939 self.subdirectives = {} |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
940 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
941 def append(self, kind, data, pos): |
| 738 | 942 """Append a stream event to the buffer. |
| 943 | |
| 944 :param kind: the stream event kind | |
| 945 :param data: the event data | |
| 946 :param pos: the position of the event in the source | |
| 947 """ | |
| 849 | 948 if kind is SUB: |
| 949 # The order needs to be +1 because a new START kind event will | |
| 950 # happen and we we need to wrap those events into our custom kind(s) | |
| 951 order = self.stack[-1] + 1 | |
| 952 subdirectives, substream = data | |
| 953 # Store the directives that should be applied after translation | |
| 954 self.subdirectives.setdefault(order, []).extend(subdirectives) | |
| 955 self.events.setdefault(order, []).append((SUB_START, None, pos)) | |
| 956 for skind, sdata, spos in substream: | |
| 957 self.append(skind, sdata, spos) | |
| 958 self.events.setdefault(order, []).append((SUB_END, None, pos)) | |
| 959 elif kind is TEXT: | |
| 960 if '[' in data or ']' in data: | |
| 961 # Quote [ and ] if it ain't us adding it, ie, if the user is | |
| 962 # using those chars in his templates, escape them | |
| 963 data = data.replace('[', '\[').replace(']', '\]') | |
| 738 | 964 self.string.append(data) |
| 849 | 965 self.events.setdefault(self.stack[-1], []).append((kind, data, pos)) |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
966 elif kind is EXPR: |
| 849 | 967 if self.params: |
| 968 param = self.params.pop(0) | |
| 969 else: | |
| 970 params = ', '.join(['"%s"' % p for p in self.orig_params if p]) | |
| 971 if params: | |
| 972 params = "(%s)" % params | |
| 973 raise IndexError("%d parameters%s given to 'i18n:%s' but " | |
| 974 "%d or more expressions used in '%s', line %s" | |
| 975 % (len(self.orig_params), params, | |
| 976 self.directive.tagname, | |
| 977 len(self.orig_params)+1, | |
| 978 os.path.basename(pos[0] or | |
| 872 | 979 'In-memory Template'), |
| 849 | 980 pos[1])) |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
981 self.string.append('%%(%s)s' % param) |
| 849 | 982 self.events.setdefault(self.stack[-1], []).append((kind, data, pos)) |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
983 self.values[param] = (kind, data, pos) |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
984 else: |
| 849 | 985 if kind is START: |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
986 self.string.append('[%d:' % self.order) |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
987 self.stack.append(self.order) |
| 849 | 988 self.events.setdefault(self.stack[-1], |
| 989 []).append((kind, data, pos)) | |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
990 self.depth += 1 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
991 self.order += 1 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
992 elif kind is END: |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
993 self.depth -= 1 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
994 if self.depth: |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
995 self.events[self.stack[-1]].append((kind, data, pos)) |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
996 self.string.append(']') |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
997 self.stack.pop() |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
998 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
999 def format(self): |
| 738 | 1000 """Return a message identifier representing the content in the |
| 1001 buffer. | |
| 1002 """ | |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
1003 return ''.join(self.string).strip() |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1004 |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
1005 def translate(self, string, regex=re.compile(r'%\((\w+)\)s')): |
| 738 | 1006 """Interpolate the given message translation with the events in the |
| 1007 buffer and return the translated stream. | |
| 1008 | |
| 1009 :param string: the translated message string | |
| 1010 """ | |
| 849 | 1011 substream = None |
| 1012 | |
| 1013 def yield_parts(string): | |
| 1014 for idx, part in enumerate(regex.split(string)): | |
| 1015 if idx % 2: | |
| 1016 yield self.values[part] | |
| 1017 elif part: | |
| 1018 yield (TEXT, | |
| 1019 part.replace('\[', '[').replace('\]', ']'), | |
| 1020 (None, -1, -1) | |
| 1021 ) | |
| 1022 | |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1023 parts = parse_msg(string) |
| 849 | 1024 parts_counter = {} |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1025 for order, string in parts: |
| 849 | 1026 parts_counter.setdefault(order, []).append(None) |
| 1027 | |
| 1028 while parts: | |
| 1029 order, string = parts.pop(0) | |
| 1030 if len(parts_counter[order]) == 1: | |
| 1031 events = self.events[order] | |
| 1032 else: | |
| 1033 events = [self.events[order].pop(0)] | |
| 1034 parts_counter[order].pop() | |
| 1035 | |
| 1036 for event in events: | |
| 1037 if event[0] is SUB_START: | |
| 1038 substream = [] | |
| 1039 elif event[0] is SUB_END: | |
| 1040 # Yield a substream which might have directives to be | |
| 1041 # applied to it (after translation events) | |
| 1042 yield SUB, (self.subdirectives[order], substream), event[2] | |
| 1043 substream = None | |
| 1044 elif event[0] is TEXT: | |
| 1045 if string: | |
| 1046 for part in yield_parts(string): | |
| 1047 if substream is not None: | |
| 1048 substream.append(part) | |
| 1049 else: | |
| 1050 yield part | |
| 1051 # String handled, reset it | |
| 1052 string = None | |
| 1053 elif event[0] is START: | |
| 1054 if substream is not None: | |
| 1055 substream.append(event) | |
| 1056 else: | |
| 1057 yield event | |
| 1058 if string: | |
| 1059 for part in yield_parts(string): | |
| 1060 if substream is not None: | |
| 1061 substream.append(part) | |
| 1062 else: | |
| 1063 yield part | |
| 1064 # String handled, reset it | |
| 1065 string = None | |
| 1066 elif event[0] is END: | |
| 1067 if string: | |
| 1068 for part in yield_parts(string): | |
| 1069 if substream is not None: | |
| 1070 substream.append(part) | |
| 1071 else: | |
| 1072 yield part | |
| 1073 # String handled, reset it | |
| 1074 string = None | |
| 1075 if substream is not None: | |
| 1076 substream.append(event) | |
| 1077 else: | |
| 1078 yield event | |
| 1079 elif event[0] is EXPR: | |
| 1080 # These are handled on the strings itself | |
| 1081 continue | |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
1082 else: |
| 849 | 1083 if string: |
| 1084 for part in yield_parts(string): | |
| 1085 if substream is not None: | |
| 1086 substream.append(part) | |
| 1087 else: | |
| 1088 yield part | |
| 1089 # String handled, reset it | |
| 1090 string = None | |
| 1091 if substream is not None: | |
| 1092 substream.append(event) | |
| 1093 else: | |
| 1094 yield event | |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1095 |
| 849 | 1096 def parse_msg(string, regex=re.compile(r'(?:\[(\d+)\:)|(?<!\\)\]')): |
| 738 | 1097 """Parse a translated message using Genshi mixed content message |
| 1098 formatting. | |
| 849 | 1099 |
| 738 | 1100 >>> parse_msg("See [1:Help].") |
| 1101 [(0, 'See '), (1, 'Help'), (0, '.')] | |
| 849 | 1102 |
| 738 | 1103 >>> parse_msg("See [1:our [2:Help] page] for details.") |
| 1104 [(0, 'See '), (1, 'our '), (2, 'Help'), (1, ' page'), (0, ' for details.')] | |
| 849 | 1105 |
| 738 | 1106 >>> parse_msg("[2:Details] finden Sie in [1:Hilfe].") |
| 1107 [(2, 'Details'), (0, ' finden Sie in '), (1, 'Hilfe'), (0, '.')] | |
| 849 | 1108 |
| 738 | 1109 >>> parse_msg("[1:] Bilder pro Seite anzeigen.") |
| 1110 [(1, ''), (0, ' Bilder pro Seite anzeigen.')] | |
| 849 | 1111 |
| 738 | 1112 :param string: the translated message string |
| 1113 :return: a list of ``(order, string)`` tuples | |
| 1114 :rtype: `list` | |
| 1115 """ | |
| 1116 parts = [] | |
| 1117 stack = [0] | |
| 1118 while True: | |
| 1119 mo = regex.search(string) | |
| 1120 if not mo: | |
| 1121 break | |
| 1122 | |
| 1123 if mo.start() or stack[-1]: | |
| 1124 parts.append((stack[-1], string[:mo.start()])) | |
| 1125 string = string[mo.end():] | |
| 1126 | |
| 1127 orderno = mo.group(1) | |
| 1128 if orderno is not None: | |
| 1129 stack.append(int(orderno)) | |
| 1130 else: | |
| 1131 stack.pop() | |
| 1132 if not stack: | |
| 1133 break | |
| 1134 | |
| 1135 if string: | |
| 1136 parts.append((stack[-1], string)) | |
| 1137 | |
| 1138 return parts | |
| 1139 | |
|
776
9e0ba5b9693c
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
1140 |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1141 def extract_from_code(code, gettext_functions): |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1142 """Extract strings from Python bytecode. |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1143 |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1144 >>> from genshi.template.eval import Expression |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1145 >>> expr = Expression('_("Hello")') |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1146 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS)) |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1147 [('_', u'Hello')] |
| 849 | 1148 |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1149 >>> expr = Expression('ngettext("You have %(num)s item", ' |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1150 ... '"You have %(num)s items", num)') |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1151 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS)) |
|
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1152 [('ngettext', (u'You have %(num)s item', u'You have %(num)s items', None))] |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1153 |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1154 :param code: the `Code` object |
|
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1155 :type code: `genshi.template.eval.Code` |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1156 :param gettext_functions: a sequence of function names |
| 576 | 1157 :since: version 0.5 |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1158 """ |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1159 def _walk(node): |
|
794
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1160 if isinstance(node, _ast.Call) and isinstance(node.func, _ast.Name) \ |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1161 and node.func.id in gettext_functions: |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1162 strings = [] |
|
600
0d802a7e3630
Handle starargs and dstarargs in the I18n extraction code.
cmlenz
parents:
596
diff
changeset
|
1163 def _add(arg): |
|
794
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1164 if isinstance(arg, _ast.Str) and isinstance(arg.s, basestring): |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1165 strings.append(unicode(arg.s, 'utf-8')) |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1166 elif arg: |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1167 strings.append(None) |
|
600
0d802a7e3630
Handle starargs and dstarargs in the I18n extraction code.
cmlenz
parents:
596
diff
changeset
|
1168 [_add(arg) for arg in node.args] |
|
794
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1169 _add(node.starargs) |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1170 _add(node.kwargs) |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1171 if len(strings) == 1: |
|
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1172 strings = strings[0] |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1173 else: |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1174 strings = tuple(strings) |
|
794
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1175 yield node.func.id, strings |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1176 elif node._fields: |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1177 children = [] |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1178 for field in node._fields: |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1179 child = getattr(node, field, None) |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1180 if isinstance(child, list): |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1181 for elem in child: |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1182 children.append(elem) |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1183 elif isinstance(child, _ast.AST): |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1184 children.append(child) |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1185 for child in children: |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1186 for funcname, strings in _walk(child): |
|
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1187 yield funcname, strings |
|
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1188 return _walk(code.ast) |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1189 |
|
776
9e0ba5b9693c
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
1190 |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1191 def extract(fileobj, keywords, comment_tags, options): |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1192 """Babel extraction method for Genshi templates. |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1193 |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1194 :param fileobj: the file-like object the messages should be extracted from |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1195 :param keywords: a list of keywords (i.e. function names) that should be |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1196 recognized as translation functions |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1197 :param comment_tags: a list of translator tags to search for and include |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1198 in the results |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1199 :param options: a dictionary of additional options (optional) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1200 :return: an iterator over ``(lineno, funcname, message, comments)`` tuples |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1201 :rtype: ``iterator`` |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1202 """ |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1203 template_class = options.get('template_class', MarkupTemplate) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1204 if isinstance(template_class, basestring): |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1205 module, clsname = template_class.split(':', 1) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1206 template_class = getattr(__import__(module, {}, {}, [clsname]), clsname) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1207 encoding = options.get('encoding', None) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1208 |
|
596
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1209 extract_text = options.get('extract_text', True) |
|
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1210 if isinstance(extract_text, basestring): |
|
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1211 extract_text = extract_text.lower() in ('1', 'on', 'yes', 'true') |
|
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1212 |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1213 ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1214 if isinstance(ignore_tags, basestring): |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1215 ignore_tags = ignore_tags.split() |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1216 ignore_tags = [QName(tag) for tag in ignore_tags] |
|
596
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1217 |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1218 include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1219 if isinstance(include_attrs, basestring): |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1220 include_attrs = include_attrs.split() |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1221 include_attrs = [QName(attr) for attr in include_attrs] |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1222 |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1223 tmpl = template_class(fileobj, filename=getattr(fileobj, 'name', None), |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1224 encoding=encoding) |
|
882
6f49c23045b1
Fix regression in [1099]: templates must not have an implicit loader during message extraction.
cmlenz
parents:
873
diff
changeset
|
1225 tmpl.loader = None |
| 849 | 1226 |
|
596
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1227 translator = Translator(None, ignore_tags, include_attrs, extract_text) |
| 849 | 1228 if hasattr(tmpl, 'add_directives'): |
| 1229 tmpl.add_directives(Translator.NAMESPACE, translator) | |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
1230 for message in translator.extract(tmpl.stream, gettext_functions=keywords): |
|
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
1231 yield message |