Mercurial > genshi > genshi-test
annotate genshi/filters/i18n.py @ 892:1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
| author | cmlenz |
|---|---|
| date | Wed, 21 Apr 2010 10:42:41 +0000 |
| parents | b40dbfee9ba6 |
| children | f30c9fb10272 |
| rev | line source |
|---|---|
| 531 | 1 # -*- coding: utf-8 -*- |
| 2 # | |
|
850
47297fd93363
Update copyright years on files changed due to advanced-i18n merge.
cmlenz
parents:
849
diff
changeset
|
3 # Copyright (C) 2007-2009 Edgewall Software |
| 531 | 4 # All rights reserved. |
| 5 # | |
| 6 # This software is licensed as described in the file COPYING, which | |
| 7 # you should have received as part of this distribution. The terms | |
| 8 # are also available at http://genshi.edgewall.org/wiki/License. | |
| 9 # | |
| 10 # This software consists of voluntary contributions made by many | |
| 11 # individuals. For the exact contribution history, see the revision | |
| 12 # history and logs, available at http://genshi.edgewall.org/log/. | |
| 13 | |
| 849 | 14 """Directives and utilities for internationalization and localization of |
| 15 templates. | |
| 576 | 16 |
| 17 :since: version 0.4 | |
| 849 | 18 :note: Directives support added since version 0.6 |
| 576 | 19 """ |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
20 |
| 856 | 21 try: |
| 22 any | |
| 23 except NameError: | |
| 24 from genshi.util import any | |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
25 from gettext import NullTranslations |
| 849 | 26 import os |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
27 import re |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
28 from types import FunctionType |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
29 |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
30 from genshi.core import Attrs, Namespace, QName, START, END, TEXT, START_NS, \ |
| 849 | 31 END_NS, XML_NAMESPACE, _ensure, StreamEventKind |
|
794
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
32 from genshi.template.eval import _ast |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
33 from genshi.template.base import DirectiveFactory, EXPR, SUB, _apply_directives |
| 849 | 34 from genshi.template.directives import Directive, StripDirective |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
35 from genshi.template.markup import MarkupTemplate, EXEC |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
36 |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
37 __all__ = ['Translator', 'extract'] |
|
501
3073ac688651
Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions.
cmlenz
parents:
493
diff
changeset
|
38 __docformat__ = 'restructuredtext en' |
|
3073ac688651
Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions.
cmlenz
parents:
493
diff
changeset
|
39 |
| 849 | 40 |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
41 I18N_NAMESPACE = Namespace('http://genshi.edgewall.org/i18n') |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
42 |
| 849 | 43 MSGBUF = StreamEventKind('MSGBUF') |
| 44 SUB_START = StreamEventKind('SUB_START') | |
| 45 SUB_END = StreamEventKind('SUB_END') | |
| 46 | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
47 GETTEXT_FUNCTIONS = ('_', 'gettext', 'ngettext', 'dgettext', 'dngettext', |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
48 'ugettext', 'ungettext') |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
49 |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
50 |
| 849 | 51 class I18NDirective(Directive): |
| 52 """Simple interface for i18n directives to support messages extraction.""" | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
53 |
| 849 | 54 def __call__(self, stream, directives, ctxt, **vars): |
| 55 return _apply_directives(stream, directives, ctxt, vars) | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
56 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
57 |
| 849 | 58 class ExtractableI18NDirective(I18NDirective): |
| 59 """Simple interface for directives to support messages extraction.""" | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
60 |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
61 def extract(self, translator, stream, gettext_functions=GETTEXT_FUNCTIONS, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
62 search_text=True, comment_stack=None): |
| 849 | 63 raise NotImplementedError |
| 64 | |
| 65 | |
| 66 class CommentDirective(I18NDirective): | |
| 67 """Implementation of the ``i18n:comment`` template directive which adds | |
| 68 translation comments. | |
| 69 | |
| 70 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 71 ... <p i18n:comment="As in Foo Bar">Foo</p> | |
| 72 ... </html>''') | |
| 73 >>> translator = Translator() | |
| 74 >>> translator.setup(tmpl) | |
| 75 >>> list(translator.extract(tmpl.stream)) | |
| 76 [(2, None, u'Foo', [u'As in Foo Bar'])] | |
| 77 """ | |
| 78 __slots__ = ['comment'] | |
| 79 | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
80 def __init__(self, value, template=None, namespaces=None, lineno=-1, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
81 offset=-1): |
| 849 | 82 Directive.__init__(self, None, template, namespaces, lineno, offset) |
| 83 self.comment = value | |
| 84 | |
| 85 | |
| 86 class MsgDirective(ExtractableI18NDirective): | |
| 87 r"""Implementation of the ``i18n:msg`` directive which marks inner content | |
| 88 as translatable. Consider the following examples: | |
| 89 | |
| 90 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 91 ... <div i18n:msg=""> | |
| 92 ... <p>Foo</p> | |
| 93 ... <p>Bar</p> | |
| 94 ... </div> | |
| 95 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
| 96 ... </html>''') | |
| 97 | |
| 98 >>> translator = Translator() | |
| 99 >>> translator.setup(tmpl) | |
| 100 >>> list(translator.extract(tmpl.stream)) | |
| 101 [(2, None, u'[1:Foo]\n [2:Bar]', []), (6, None, u'Foo [1:bar]!', [])] | |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
102 >>> print(tmpl.generate().render()) |
| 849 | 103 <html> |
| 104 <div><p>Foo</p> | |
| 105 <p>Bar</p></div> | |
| 106 <p>Foo <em>bar</em>!</p> | |
| 107 </html> | |
| 108 | |
| 109 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 110 ... <div i18n:msg="fname, lname"> | |
| 111 ... <p>First Name: ${fname}</p> | |
| 112 ... <p>Last Name: ${lname}</p> | |
| 113 ... </div> | |
| 114 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
| 115 ... </html>''') | |
| 116 >>> translator.setup(tmpl) | |
| 117 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
| 118 [(2, None, u'[1:First Name: %(fname)s]\n [2:Last Name: %(lname)s]', []), | |
| 119 (6, None, u'Foo [1:bar]!', [])] | |
| 120 | |
| 121 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 122 ... <div i18n:msg="fname, lname"> | |
| 123 ... <p>First Name: ${fname}</p> | |
| 124 ... <p>Last Name: ${lname}</p> | |
| 125 ... </div> | |
| 126 ... <p i18n:msg="">Foo <em>bar</em>!</p> | |
| 127 ... </html>''') | |
| 128 >>> translator.setup(tmpl) | |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
129 >>> print(tmpl.generate(fname='John', lname='Doe').render()) |
| 849 | 130 <html> |
| 131 <div><p>First Name: John</p> | |
| 132 <p>Last Name: Doe</p></div> | |
| 133 <p>Foo <em>bar</em>!</p> | |
| 134 </html> | |
| 135 | |
| 136 Starting and ending white-space is stripped of to make it simpler for | |
| 137 translators. Stripping it is not that important since it's on the html | |
| 138 source, the rendered output will remain the same. | |
| 139 """ | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
140 __slots__ = ['params', 'lineno'] |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
141 |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
142 def __init__(self, value, template=None, namespaces=None, lineno=-1, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
143 offset=-1): |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
144 Directive.__init__(self, None, template, namespaces, lineno, offset) |
| 849 | 145 self.params = [param.strip() for param in value.split(',') if param] |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
146 self.lineno = lineno |
| 849 | 147 |
| 148 @classmethod | |
| 149 def attach(cls, template, stream, value, namespaces, pos): | |
| 150 if type(value) is dict: | |
| 151 value = value.get('params', '').strip() | |
| 152 return super(MsgDirective, cls).attach(template, stream, value.strip(), | |
| 153 namespaces, pos) | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
154 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
155 def __call__(self, stream, directives, ctxt, **vars): |
| 849 | 156 gettext = ctxt.get('_i18n.gettext') |
| 157 if ctxt.get('_i18n.domain'): | |
|
891
b40dbfee9ba6
Removed some obsolete/unused code from the i18n filter.
cmlenz
parents:
888
diff
changeset
|
158 dgettext = ctxt.get('_i18n.dgettext') |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
159 assert hasattr(dgettext, '__call__'), \ |
|
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
160 'No domain gettext function passed' |
| 849 | 161 gettext = lambda msg: dgettext(ctxt.get('_i18n.domain'), msg) |
| 162 | |
| 163 def _generate(): | |
| 164 msgbuf = MessageBuffer(self) | |
| 165 previous = stream.next() | |
| 166 if previous[0] is START: | |
| 167 yield previous | |
| 168 else: | |
| 169 msgbuf.append(*previous) | |
| 170 previous = stream.next() | |
| 171 for kind, data, pos in stream: | |
| 172 msgbuf.append(*previous) | |
| 173 previous = kind, data, pos | |
| 174 if previous[0] is not END: | |
| 175 msgbuf.append(*previous) | |
| 176 previous = None | |
| 177 for event in msgbuf.translate(gettext(msgbuf.format())): | |
| 178 yield event | |
| 179 if previous: | |
| 180 yield previous | |
| 181 | |
| 182 return _apply_directives(_generate(), directives, ctxt, vars) | |
| 183 | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
184 def extract(self, translator, stream, gettext_functions=GETTEXT_FUNCTIONS, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
185 search_text=True, comment_stack=None): |
| 849 | 186 msgbuf = MessageBuffer(self) |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
187 strip = False |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
188 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
189 stream = iter(stream) |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
190 previous = stream.next() |
| 849 | 191 if previous[0] is START: |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
192 for message in translator._extract_attrs(previous, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
193 gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
194 search_text=search_text): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
195 yield message |
| 849 | 196 previous = stream.next() |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
197 strip = True |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
198 for event in stream: |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
199 if event[0] is START: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
200 for message in translator._extract_attrs(event, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
201 gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
202 search_text=search_text): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
203 yield message |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
204 msgbuf.append(*previous) |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
205 previous = event |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
206 if not strip: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
207 msgbuf.append(*previous) |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
208 |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
209 yield self.lineno, None, msgbuf.format(), comment_stack[-1:] |
| 849 | 210 |
| 211 | |
| 212 class ChooseBranchDirective(I18NDirective): | |
| 213 __slots__ = ['params'] | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
214 |
| 849 | 215 def __call__(self, stream, directives, ctxt, **vars): |
| 216 self.params = ctxt.get('_i18n.choose.params', [])[:] | |
| 217 msgbuf = MessageBuffer(self) | |
| 218 | |
| 219 stream = iter(_apply_directives(stream, directives, ctxt, vars)) | |
|
871
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
220 |
| 849 | 221 previous = stream.next() |
|
871
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
222 if previous[0] is START: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
223 yield previous |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
224 else: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
225 msgbuf.append(*previous) |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
226 |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
227 try: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
228 previous = stream.next() |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
229 except StopIteration: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
230 # For example <i18n:singular> or <i18n:plural> directives |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
231 yield MSGBUF, (), -1 # the place holder for msgbuf output |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
232 ctxt['_i18n.choose.%s' % type(self).__name__] = msgbuf |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
233 return |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
234 |
| 849 | 235 for kind, data, pos in stream: |
| 236 msgbuf.append(*previous) | |
| 237 previous = kind, data, pos | |
| 238 yield MSGBUF, (), -1 # the place holder for msgbuf output | |
|
871
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
239 |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
240 if previous[0] is END: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
241 yield previous # the outer end tag |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
242 else: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
243 msgbuf.append(*previous) |
| 849 | 244 ctxt['_i18n.choose.%s' % type(self).__name__] = msgbuf |
| 245 | |
| 246 | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
247 def extract(self, translator, stream, gettext_functions=GETTEXT_FUNCTIONS, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
248 search_text=True, comment_stack=None, msgbuf=None): |
| 849 | 249 stream = iter(stream) |
| 250 previous = stream.next() | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
251 |
| 849 | 252 if previous[0] is START: |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
253 # skip the enclosing element |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
254 for message in translator._extract_attrs(previous, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
255 gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
256 search_text=search_text): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
257 yield message |
| 849 | 258 previous = stream.next() |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
259 |
| 849 | 260 for event in stream: |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
261 if previous[0] is START: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
262 for message in translator._extract_attrs(previous, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
263 gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
264 search_text=search_text): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
265 yield message |
| 849 | 266 msgbuf.append(*previous) |
| 267 previous = event | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
268 |
| 849 | 269 if previous[0] is not END: |
| 270 msgbuf.append(*previous) | |
| 271 | |
| 272 | |
| 273 class SingularDirective(ChooseBranchDirective): | |
| 274 """Implementation of the ``i18n:singular`` directive to be used with the | |
| 275 ``i18n:choose`` directive.""" | |
| 276 | |
| 277 | |
| 278 class PluralDirective(ChooseBranchDirective): | |
| 279 """Implementation of the ``i18n:plural`` directive to be used with the | |
| 280 ``i18n:choose`` directive.""" | |
| 281 | |
| 282 | |
| 283 class ChooseDirective(ExtractableI18NDirective): | |
| 284 """Implementation of the ``i18n:choose`` directive which provides plural | |
| 285 internationalisation of strings. | |
| 286 | |
| 287 This directive requires at least one parameter, the one which evaluates to | |
| 288 an integer which will allow to choose the plural/singular form. If you also | |
| 289 have expressions inside the singular and plural version of the string you | |
| 290 also need to pass a name for those parameters. Consider the following | |
| 291 examples: | |
| 292 | |
| 293 >>> tmpl = MarkupTemplate('''\ | |
| 294 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 295 ... <div i18n:choose="num; num"> | |
| 296 ... <p i18n:singular="">There is $num coin</p> | |
| 297 ... <p i18n:plural="">There are $num coins</p> | |
| 298 ... </div> | |
| 299 ... </html>''') | |
| 300 >>> translator = Translator() | |
| 301 >>> translator.setup(tmpl) | |
| 302 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
| 303 [(2, 'ngettext', (u'There is %(num)s coin', | |
| 304 u'There are %(num)s coins'), [])] | |
| 305 | |
| 306 >>> tmpl = MarkupTemplate('''\ | |
| 307 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 308 ... <div i18n:choose="num; num"> | |
| 309 ... <p i18n:singular="">There is $num coin</p> | |
| 310 ... <p i18n:plural="">There are $num coins</p> | |
| 311 ... </div> | |
| 312 ... </html>''') | |
| 313 >>> translator.setup(tmpl) | |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
314 >>> print(tmpl.generate(num=1).render()) |
| 849 | 315 <html> |
| 316 <div> | |
| 317 <p>There is 1 coin</p> | |
| 318 </div> | |
| 319 </html> | |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
320 >>> print(tmpl.generate(num=2).render()) |
| 849 | 321 <html> |
| 322 <div> | |
| 323 <p>There are 2 coins</p> | |
| 324 </div> | |
| 325 </html> | |
| 326 | |
| 888 | 327 When used as a element and not as an attribute: |
| 849 | 328 |
| 329 >>> tmpl = MarkupTemplate('''\ | |
| 330 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 331 ... <i18n:choose numeral="num" params="num"> | |
| 332 ... <p i18n:singular="">There is $num coin</p> | |
| 333 ... <p i18n:plural="">There are $num coins</p> | |
| 334 ... </i18n:choose> | |
| 335 ... </html>''') | |
| 336 >>> translator.setup(tmpl) | |
| 337 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE | |
| 338 [(2, 'ngettext', (u'There is %(num)s coin', | |
| 339 u'There are %(num)s coins'), [])] | |
| 340 """ | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
341 __slots__ = ['numeral', 'params', 'lineno'] |
| 849 | 342 |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
343 def __init__(self, value, template=None, namespaces=None, lineno=-1, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
344 offset=-1): |
| 849 | 345 Directive.__init__(self, None, template, namespaces, lineno, offset) |
| 346 params = [v.strip() for v in value.split(';')] | |
| 347 self.numeral = self._parse_expr(params.pop(0), template, lineno, offset) | |
| 348 self.params = params and [name.strip() for name in | |
| 349 params[0].split(',') if name] or [] | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
350 self.lineno = lineno |
| 849 | 351 |
| 352 @classmethod | |
| 353 def attach(cls, template, stream, value, namespaces, pos): | |
| 354 if type(value) is dict: | |
| 355 numeral = value.get('numeral', '').strip() | |
| 356 assert numeral is not '', "at least pass the numeral param" | |
| 357 params = [v.strip() for v in value.get('params', '').split(',')] | |
| 358 value = '%s; ' % numeral + ', '.join(params) | |
| 359 return super(ChooseDirective, cls).attach(template, stream, value, | |
| 360 namespaces, pos) | |
| 361 | |
| 362 def __call__(self, stream, directives, ctxt, **vars): | |
| 363 ctxt.push({'_i18n.choose.params': self.params, | |
| 364 '_i18n.choose.SingularDirective': None, | |
| 365 '_i18n.choose.PluralDirective': None}) | |
| 366 | |
| 367 new_stream = [] | |
| 368 singular_stream = None | |
| 369 singular_msgbuf = None | |
| 370 plural_stream = None | |
| 371 plural_msgbuf = None | |
| 372 | |
| 373 ngettext = ctxt.get('_i18n.ungettext') | |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
374 assert hasattr(ngettext, '__call__'), 'No ngettext function available' |
| 849 | 375 dngettext = ctxt.get('_i18n.dngettext') |
| 376 if not dngettext: | |
| 377 dngettext = lambda d, s, p, n: ngettext(s, p, n) | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
378 |
| 849 | 379 for kind, event, pos in stream: |
| 380 if kind is SUB: | |
| 381 subdirectives, substream = event | |
| 382 if isinstance(subdirectives[0], | |
| 383 SingularDirective) and not singular_stream: | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
384 strip_directive_present = [] |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
385 for idx, subdirective in enumerate(subdirectives): |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
386 if isinstance(subdirective, StripDirective): |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
387 # Any strip directive should be applied AFTER |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
388 # the event's have been translated. |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
389 strip_directive_present.append( |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
390 subdirectives.pop(idx) |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
391 ) |
| 849 | 392 # Apply directives to update context |
| 393 singular_stream = list(_apply_directives(substream, | |
| 394 subdirectives, | |
| 395 ctxt, vars)) | |
|
871
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
396 if strip_directive_present: |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
397 singular_stream = list( |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
398 _apply_directives(singular_stream, |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
399 strip_directive_present, |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
400 ctxt, vars) |
|
7eb6f506bb54
Allow the use of `i18n:singular` and `i18n:plural` as directives and not just as attributes.
palgarvio
parents:
869
diff
changeset
|
401 ) |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
402 del strip_directive_present |
| 849 | 403 new_stream.append((MSGBUF, (), ('', -1))) # msgbuf place holder |
| 404 singular_msgbuf = ctxt.get('_i18n.choose.SingularDirective') | |
| 405 elif isinstance(subdirectives[0], | |
| 406 PluralDirective) and not plural_stream: | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
407 strip_directive_present = [] |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
408 for idx, subdirective in enumerate(subdirectives): |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
409 if isinstance(subdirective, StripDirective): |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
410 # Any strip directive should be applied AFTER |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
411 # the event's have been translated. |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
412 strip_directive_present.append( |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
413 subdirectives.pop(idx) |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
414 ) |
| 849 | 415 # Apply directives to update context |
| 416 plural_stream = list(_apply_directives(substream, | |
| 417 subdirectives, | |
| 418 ctxt, vars)) | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
419 if strip_directive_present: |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
420 plural_stream = list( |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
421 _apply_directives(plural_stream, |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
422 strip_directive_present, |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
423 ctxt, vars) |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
424 ) |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
425 del strip_directive_present |
| 849 | 426 plural_msgbuf = ctxt.get('_i18n.choose.PluralDirective') |
| 427 else: | |
| 428 new_stream.append((kind, event, pos)) | |
| 429 else: | |
| 430 new_stream.append((kind, event, pos)) | |
| 431 | |
| 432 if ctxt.get('_i18n.domain'): | |
| 433 ngettext = lambda s, p, n: dngettext(ctxt.get('_i18n.domain'), | |
| 434 s, p, n) | |
| 435 | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
436 # XXX: should we test which form was chosen like this!?!?!? |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
437 # There should be no match in any catalogue for these singular and |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
438 # plural test strings |
|
882
6f49c23045b1
Fix regression in [1099]: templates must not have an implicit loader during message extraction.
cmlenz
parents:
873
diff
changeset
|
439 singular_test = u'O\x85\xbe\xa9\xa8az\xc3?\xe6\xa1\x02n\x84\x93' |
|
6f49c23045b1
Fix regression in [1099]: templates must not have an implicit loader during message extraction.
cmlenz
parents:
873
diff
changeset
|
440 plural_test = u'\xcc\xfb+\xd3Pn\x9d\tT\xec\x1d\xda\x1a\x88\x00' |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
441 translation = ngettext(singular_test, plural_test, |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
442 self.numeral.evaluate(ctxt)) |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
443 if translation == singular_test: |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
444 chosen_msgbuf = singular_msgbuf |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
445 chosen_stream = singular_stream |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
446 else: |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
447 chosen_msgbuf = plural_msgbuf |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
448 chosen_stream = plural_stream |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
449 del singular_test, plural_test, translation |
|
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
450 |
| 849 | 451 for kind, data, pos in new_stream: |
| 452 if kind is MSGBUF: | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
453 for skind, sdata, spos in chosen_stream: |
| 849 | 454 if skind is MSGBUF: |
| 455 translation = ngettext(singular_msgbuf.format(), | |
| 456 plural_msgbuf.format(), | |
| 457 self.numeral.evaluate(ctxt)) | |
|
873
9786aa263217
Any `py:strip` involved on `i18n:singular` or `i18n:plural` is now handled separately, ie, if one has `py:strip` on a `i18n:singular` element or directive and the plural form is the one chosen, no stripping will be performed. Refs #371.
palgarvio
parents:
872
diff
changeset
|
458 for event in chosen_msgbuf.translate(translation): |
| 849 | 459 yield event |
| 460 else: | |
| 461 yield skind, sdata, spos | |
| 462 else: | |
| 463 yield kind, data, pos | |
| 464 | |
| 465 ctxt.pop() | |
| 466 | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
467 def extract(self, translator, stream, gettext_functions=GETTEXT_FUNCTIONS, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
468 search_text=True, comment_stack=None): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
469 strip = False |
| 849 | 470 stream = iter(stream) |
| 471 previous = stream.next() | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
472 |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
473 if previous[0] is START: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
474 # skip the enclosing element |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
475 for message in translator._extract_attrs(previous, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
476 gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
477 search_text=search_text): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
478 yield message |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
479 previous = stream.next() |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
480 strip = True |
| 849 | 481 |
| 482 singular_msgbuf = MessageBuffer(self) | |
| 483 plural_msgbuf = MessageBuffer(self) | |
| 484 | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
485 for event in stream: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
486 if previous[0] is SUB: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
487 directives, substream = previous[1] |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
488 for directive in directives: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
489 if isinstance(directive, SingularDirective): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
490 for message in directive.extract(translator, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
491 substream, gettext_functions, search_text, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
492 comment_stack, msgbuf=singular_msgbuf): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
493 yield message |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
494 elif isinstance(directive, PluralDirective): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
495 for message in directive.extract(translator, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
496 substream, gettext_functions, search_text, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
497 comment_stack, msgbuf=plural_msgbuf): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
498 yield message |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
499 elif not isinstance(directive, StripDirective): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
500 singular_msgbuf.append(*previous) |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
501 plural_msgbuf.append(*previous) |
| 849 | 502 else: |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
503 if previous[0] is START: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
504 for message in translator._extract_attrs(previous, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
505 gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
506 search_text): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
507 yield message |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
508 singular_msgbuf.append(*previous) |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
509 plural_msgbuf.append(*previous) |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
510 previous = event |
| 849 | 511 |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
512 if not strip: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
513 singular_msgbuf.append(*previous) |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
514 plural_msgbuf.append(*previous) |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
515 |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
516 yield self.lineno, 'ngettext', \ |
| 849 | 517 (singular_msgbuf.format(), plural_msgbuf.format()), \ |
| 518 comment_stack[-1:] | |
| 519 | |
| 520 | |
| 521 class DomainDirective(I18NDirective): | |
| 522 """Implementation of the ``i18n:domain`` directive which allows choosing | |
| 523 another i18n domain(catalog) to translate from. | |
| 524 | |
| 525 >>> from genshi.filters.tests.i18n import DummyTranslations | |
| 526 >>> tmpl = MarkupTemplate('''\ | |
| 527 <html xmlns:i18n="http://genshi.edgewall.org/i18n"> | |
| 528 ... <p i18n:msg="">Bar</p> | |
| 529 ... <div i18n:domain="foo"> | |
| 530 ... <p i18n:msg="">FooBar</p> | |
| 531 ... <p>Bar</p> | |
| 532 ... <p i18n:domain="bar" i18n:msg="">Bar</p> | |
| 533 ... <p i18n:domain="">Bar</p> | |
| 534 ... </div> | |
| 535 ... <p>Bar</p> | |
| 536 ... </html>''') | |
| 537 | |
| 538 >>> translations = DummyTranslations({'Bar': 'Voh'}) | |
| 539 >>> translations.add_domain('foo', {'FooBar': 'BarFoo', 'Bar': 'foo_Bar'}) | |
| 540 >>> translations.add_domain('bar', {'Bar': 'bar_Bar'}) | |
| 541 >>> translator = Translator(translations) | |
| 542 >>> translator.setup(tmpl) | |
| 543 | |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
544 >>> print(tmpl.generate().render()) |
| 849 | 545 <html> |
| 546 <p>Voh</p> | |
| 547 <div> | |
| 548 <p>BarFoo</p> | |
| 549 <p>foo_Bar</p> | |
| 550 <p>bar_Bar</p> | |
| 551 <p>Voh</p> | |
| 552 </div> | |
| 553 <p>Voh</p> | |
| 554 </html> | |
| 555 """ | |
| 556 __slots__ = ['domain'] | |
| 557 | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
558 def __init__(self, value, template=None, namespaces=None, lineno=-1, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
559 offset=-1): |
| 849 | 560 Directive.__init__(self, None, template, namespaces, lineno, offset) |
| 561 self.domain = value and value.strip() or '__DEFAULT__' | |
| 562 | |
| 563 @classmethod | |
| 564 def attach(cls, template, stream, value, namespaces, pos): | |
| 565 if type(value) is dict: | |
| 566 value = value.get('name') | |
| 567 return super(DomainDirective, cls).attach(template, stream, value, | |
| 568 namespaces, pos) | |
| 569 | |
| 570 def __call__(self, stream, directives, ctxt, **vars): | |
| 571 ctxt.push({'_i18n.domain': self.domain}) | |
| 572 for event in _apply_directives(stream, directives, ctxt, vars): | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
573 yield event |
| 849 | 574 ctxt.pop() |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
575 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
576 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
577 class Translator(DirectiveFactory): |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
578 """Can extract and translate localizable strings from markup streams and |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
579 templates. |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
580 |
| 849 | 581 For example, assume the following template: |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
582 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
583 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/"> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
584 ... <head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
585 ... <title>Example</title> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
586 ... </head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
587 ... <body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
588 ... <h1>Example</h1> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
589 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
590 ... </body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
591 ... </html>''', filename='example.html') |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
592 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
593 For demonstration, we define a dummy ``gettext``-style function with a |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
594 hard-coded translation table, and pass that to the `Translator` initializer: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
595 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
596 >>> def pseudo_gettext(string): |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
597 ... return { |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
598 ... 'Example': 'Beispiel', |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
599 ... 'Hello, %(name)s': 'Hallo, %(name)s' |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
600 ... }[string] |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
601 >>> translator = Translator(pseudo_gettext) |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
602 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
603 Next, the translator needs to be prepended to any already defined filters |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
604 on the template: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
605 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
606 >>> tmpl.filters.insert(0, translator) |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
607 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
608 When generating the template output, our hard-coded translations should be |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
609 applied as expected: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
610 |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
611 >>> print(tmpl.generate(username='Hans', _=pseudo_gettext)) |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
612 <html> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
613 <head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
614 <title>Beispiel</title> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
615 </head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
616 <body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
617 <h1>Beispiel</h1> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
618 <p>Hallo, Hans</p> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
619 </body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
620 </html> |
| 849 | 621 |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
622 Note that elements defining ``xml:lang`` attributes that do not contain |
|
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
623 variable expressions are ignored by this filter. That can be used to |
|
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
624 exclude specific parts of a template from being extracted and translated. |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
625 """ |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
626 |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
627 directives = [ |
| 849 | 628 ('domain', DomainDirective), |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
629 ('comment', CommentDirective), |
| 849 | 630 ('msg', MsgDirective), |
| 631 ('choose', ChooseDirective), | |
| 632 ('singular', SingularDirective), | |
| 633 ('plural', PluralDirective) | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
634 ] |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
635 |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
636 IGNORE_TAGS = frozenset([ |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
637 QName('script'), QName('http://www.w3.org/1999/xhtml}script'), |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
638 QName('style'), QName('http://www.w3.org/1999/xhtml}style') |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
639 ]) |
| 849 | 640 INCLUDE_ATTRS = frozenset([ |
| 641 'abbr', 'alt', 'label', 'prompt', 'standby', 'summary', 'title' | |
| 642 ]) | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
643 NAMESPACE = I18N_NAMESPACE |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
644 |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
645 def __init__(self, translate=NullTranslations(), ignore_tags=IGNORE_TAGS, |
|
594
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
646 include_attrs=INCLUDE_ATTRS, extract_text=True): |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
647 """Initialize the translator. |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
648 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
649 :param translate: the translation function, for example ``gettext`` or |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
650 ``ugettext``. |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
651 :param ignore_tags: a set of tag names that should not be localized |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
652 :param include_attrs: a set of attribute names should be localized |
|
594
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
653 :param extract_text: whether the content of text nodes should be |
|
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
654 extracted, or only text in explicit ``gettext`` |
|
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
655 function calls |
| 849 | 656 |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
657 :note: Changed in 0.6: the `translate` parameter can now be either |
|
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
658 a ``gettext``-style function, or an object compatible with the |
|
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
659 ``NullTransalations`` or ``GNUTranslations`` interface |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
660 """ |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
661 self.translate = translate |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
662 self.ignore_tags = ignore_tags |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
663 self.include_attrs = include_attrs |
|
594
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
664 self.extract_text = extract_text |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
665 |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
666 def __call__(self, stream, ctxt=None, search_text=True): |
| 448 | 667 """Translate any localizable strings in the given stream. |
| 668 | |
| 669 This function shouldn't be called directly. Instead, an instance of | |
| 670 the `Translator` class should be registered as a filter with the | |
| 671 `Template` or the `TemplateLoader`, or applied as a regular stream | |
| 672 filter. If used as a template filter, it should be inserted in front of | |
| 673 all the default filters. | |
| 674 | |
| 675 :param stream: the markup event stream | |
| 676 :param ctxt: the template context (not used) | |
| 677 :param search_text: whether text nodes should be translated (used | |
| 678 internally) | |
| 679 :return: the localized stream | |
| 680 """ | |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
681 ignore_tags = self.ignore_tags |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
682 include_attrs = self.include_attrs |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
683 skip = 0 |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
684 xml_lang = XML_NAMESPACE['lang'] |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
685 |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
686 if type(self.translate) is FunctionType: |
|
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
687 gettext = self.translate |
| 849 | 688 if ctxt: |
| 689 ctxt['_i18n.gettext'] = gettext | |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
690 else: |
|
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
691 gettext = self.translate.ugettext |
| 849 | 692 try: |
| 693 dgettext = self.translate.dugettext | |
| 694 except AttributeError: | |
| 695 dgettext = lambda x, y: gettext(y) | |
| 696 ngettext = self.translate.ungettext | |
| 697 try: | |
| 698 dngettext = self.translate.dungettext | |
| 699 except AttributeError: | |
| 700 dngettext = lambda d, s, p, n: ngettext(s, p, n) | |
| 701 | |
| 702 if ctxt: | |
| 703 ctxt['_i18n.gettext'] = gettext | |
| 704 ctxt['_i18n.ugettext'] = gettext | |
| 705 ctxt['_i18n.dgettext'] = dgettext | |
| 706 ctxt['_i18n.ngettext'] = ngettext | |
| 707 ctxt['_i18n.ungettext'] = ngettext | |
| 708 ctxt['_i18n.dngettext'] = dngettext | |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
709 |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
710 extract_text = self.extract_text |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
711 if not extract_text: |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
712 search_text = False |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
713 |
| 849 | 714 if ctxt and ctxt.get('_i18n.domain'): |
| 715 old_gettext = gettext | |
| 716 gettext = lambda msg: dgettext(ctxt.get('_i18n.domain'), msg) | |
| 717 | |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
718 for kind, data, pos in stream: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
719 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
720 # skip chunks that should not be localized |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
721 if skip: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
722 if kind is START: |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
723 skip += 1 |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
724 elif kind is END: |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
725 skip -= 1 |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
726 yield kind, data, pos |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
727 continue |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
728 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
729 # handle different events that can be localized |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
730 if kind is START: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
731 tag, attrs = data |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
732 if tag in self.ignore_tags or \ |
|
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
733 isinstance(attrs.get(xml_lang), basestring): |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
734 skip += 1 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
735 yield kind, data, pos |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
736 continue |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
737 |
|
493
abe6e4ab7ecf
Fix another bug in the translation filter: translated attributes were getting added instead of replaced.
cmlenz
parents:
485
diff
changeset
|
738 new_attrs = [] |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
739 changed = False |
| 849 | 740 |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
741 for name, value in attrs: |
|
483
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
742 newval = value |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
743 if extract_text and isinstance(value, basestring): |
|
483
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
744 if name in include_attrs: |
|
788
09531799bac2
Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents:
787
diff
changeset
|
745 newval = gettext(value) |
|
483
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
746 else: |
| 849 | 747 newval = list( |
| 748 self(_ensure(value), ctxt, search_text=False) | |
|
483
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
749 ) |
|
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
750 if newval != value: |
|
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
751 value = newval |
|
dabb9e1e2ba1
Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents:
481
diff
changeset
|
752 changed = True |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
753 new_attrs.append((name, value)) |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
754 if changed: |
|
667
077c9142dca0
Fix case where attributes weren't properly wrapped in an `Attrs` instance if one or more of them were translated by the I18n filter, potentially breaking things further down the chain. Closes #162.
cmlenz
parents:
600
diff
changeset
|
755 attrs = Attrs(new_attrs) |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
756 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
757 yield kind, (tag, attrs), pos |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
758 |
|
485
770ba8556940
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
759 elif search_text and kind is TEXT: |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
760 text = data.strip() |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
761 if text: |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
762 data = data.replace(text, unicode(gettext(text))) |
|
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
763 yield kind, data, pos |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
764 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
765 elif kind is SUB: |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
766 directives, substream = data |
| 849 | 767 current_domain = None |
| 768 for idx, directive in enumerate(directives): | |
| 769 # Organize directives to make everything work | |
| 770 if isinstance(directive, DomainDirective): | |
| 771 # Grab current domain and update context | |
| 772 current_domain = directive.domain | |
| 773 ctxt.push({'_i18n.domain': current_domain}) | |
| 774 # Put domain directive as the first one in order to | |
| 775 # update context before any other directives evaluation | |
| 776 directives.insert(0, directives.pop(idx)) | |
| 777 | |
| 778 # If this is an i18n directive, no need to translate text | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
779 # nodes here |
| 856 | 780 is_i18n_directive = any([ |
| 781 isinstance(d, ExtractableI18NDirective) | |
| 782 for d in directives | |
| 783 ]) | |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
784 substream = list(self(substream, ctxt, |
| 849 | 785 search_text=not is_i18n_directive)) |
|
790
1b6968d31089
Merged the custom-directives branch back into trunk.
cmlenz
parents:
788
diff
changeset
|
786 yield kind, (directives, substream), pos |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
787 |
| 849 | 788 if current_domain: |
| 789 ctxt.pop() | |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
790 else: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
791 yield kind, data, pos |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
792 |
|
485
770ba8556940
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
793 def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS, |
|
891
b40dbfee9ba6
Removed some obsolete/unused code from the i18n filter.
cmlenz
parents:
888
diff
changeset
|
794 search_text=True, comment_stack=None): |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
795 """Extract localizable strings from the given template stream. |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
796 |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
797 For every string found, this function yields a ``(lineno, function, |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
798 message, comments)`` tuple, where: |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
799 |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
800 * ``lineno`` is the number of the line on which the string was found, |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
801 * ``function`` is the name of the ``gettext`` function used (if the |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
802 string was extracted from embedded Python code), and |
|
469
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
803 * ``message`` is the string itself (a ``unicode`` object, or a tuple |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
804 of ``unicode`` objects for functions with multiple string |
|
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
805 arguments). |
|
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
806 * ``comments`` is a list of comments related to the message, extracted |
|
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
807 from ``i18n:comment`` attributes found in the markup |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
808 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
809 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/"> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
810 ... <head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
811 ... <title>Example</title> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
812 ... </head> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
813 ... <body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
814 ... <h1>Example</h1> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
815 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p> |
|
469
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
816 ... <p>${ngettext("You have %d item", "You have %d items", num)}</p> |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
817 ... </body> |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
818 ... </html>''', filename='example.html') |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
819 >>> for line, func, msg, comments in Translator().extract(tmpl.stream): |
|
853
4376010bb97e
Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents:
850
diff
changeset
|
820 ... print('%d, %r, %r' % (line, func, msg)) |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
821 3, None, u'Example' |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
822 6, None, u'Example' |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
823 7, '_', u'Hello, %(name)s' |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
824 8, 'ngettext', (u'You have %d item', u'You have %d items', None) |
|
469
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
825 |
|
450
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
826 :param stream: the event stream to extract strings from; can be a |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
827 regular stream or a template stream |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
828 :param gettext_functions: a sequence of function names that should be |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
829 treated as gettext-style localization |
|
d2fdcd320743
Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents:
448
diff
changeset
|
830 functions |
|
485
770ba8556940
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
831 :param search_text: whether the content of text nodes should be |
|
770ba8556940
Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents:
483
diff
changeset
|
832 extracted (used internally) |
|
469
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
833 |
|
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
834 :note: Changed in 0.4.1: For a function with multiple string arguments |
|
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
835 (such as ``ngettext``), a single item with a tuple of strings is |
|
5eae4a5c42ac
The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents:
467
diff
changeset
|
836 yielded, instead an item for each string argument. |
| 849 | 837 :note: Changed in 0.6: The returned tuples now include a fourth |
| 838 element, which is a list of comments for the translator. | |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
839 """ |
|
594
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
840 if not self.extract_text: |
|
0fb43dc2e165
Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents:
576
diff
changeset
|
841 search_text = False |
| 849 | 842 if comment_stack is None: |
| 843 comment_stack = [] | |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
844 skip = 0 |
| 849 | 845 |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
846 xml_lang = XML_NAMESPACE['lang'] |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
847 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
848 for kind, data, pos in stream: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
849 if skip: |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
850 if kind is START: |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
851 skip += 1 |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
852 if kind is END: |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
853 skip -= 1 |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
854 |
|
549
da5cbf6d134d
The I18n filter now extracts text from translation functions in ignored tags. Fixes #132.
cmlenz
parents:
535
diff
changeset
|
855 if kind is START and not skip: |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
856 tag, attrs = data |
|
522
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
857 if tag in self.ignore_tags or \ |
|
c1738dec04d9
The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents:
501
diff
changeset
|
858 isinstance(attrs.get(xml_lang), basestring): |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
859 skip += 1 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
860 continue |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
861 |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
862 for message in self._extract_attrs((kind, data, pos), |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
863 gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
864 search_text=search_text): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
865 yield message |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
866 |
|
549
da5cbf6d134d
The I18n filter now extracts text from translation functions in ignored tags. Fixes #132.
cmlenz
parents:
535
diff
changeset
|
867 elif not skip and search_text and kind is TEXT: |
|
891
b40dbfee9ba6
Removed some obsolete/unused code from the i18n filter.
cmlenz
parents:
888
diff
changeset
|
868 text = data.strip() |
|
b40dbfee9ba6
Removed some obsolete/unused code from the i18n filter.
cmlenz
parents:
888
diff
changeset
|
869 if text and [ch for ch in text if ch.isalpha()]: |
|
b40dbfee9ba6
Removed some obsolete/unused code from the i18n filter.
cmlenz
parents:
888
diff
changeset
|
870 yield pos[1], None, text, comment_stack[-1:] |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
871 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
872 elif kind is EXPR or kind is EXEC: |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
873 for funcname, strings in extract_from_code(data, |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
874 gettext_functions): |
| 849 | 875 # XXX: Do we need to grab i18n:comment from comment_stack ??? |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
876 yield pos[1], funcname, strings, [] |
|
446
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
877 |
|
90f5908cd10a
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff
changeset
|
878 elif kind is SUB: |
| 849 | 879 directives, substream = data |
| 880 in_comment = False | |
| 881 | |
| 882 for idx, directive in enumerate(directives): | |
| 883 # Do a first loop to see if there's a comment directive | |
| 884 # If there is update context and pop it from directives | |
| 885 if isinstance(directive, CommentDirective): | |
| 886 in_comment = True | |
| 887 comment_stack.append(directive.comment) | |
| 888 if len(directives) == 1: | |
| 889 # in case we're in the presence of something like: | |
| 890 # <p i18n:comment="foo">Foo</p> | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
891 for message in self.extract( |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
892 substream, gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
893 search_text=search_text and not skip, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
894 comment_stack=comment_stack): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
895 yield message |
| 849 | 896 directives.pop(idx) |
| 897 elif not isinstance(directive, I18NDirective): | |
| 898 # Remove all other non i18n directives from the process | |
| 899 directives.pop(idx) | |
| 900 | |
| 901 if not directives and not in_comment: | |
| 902 # Extract content if there's no directives because | |
| 903 # strip was pop'ed and not because comment was pop'ed. | |
| 904 # Extraction in this case has been taken care of. | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
905 for message in self.extract( |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
906 substream, gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
907 search_text=search_text and not skip): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
908 yield message |
| 849 | 909 |
| 910 for directive in directives: | |
| 911 if isinstance(directive, ExtractableI18NDirective): | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
912 for message in directive.extract(self, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
913 substream, gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
914 search_text=search_text and not skip, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
915 comment_stack=comment_stack): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
916 yield message |
| 849 | 917 else: |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
918 for message in self.extract( |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
919 substream, gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
920 search_text=search_text and not skip, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
921 comment_stack=comment_stack): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
922 yield message |
| 849 | 923 |
| 924 if in_comment: | |
| 925 comment_stack.pop() | |
| 926 | |
| 927 def get_directive_index(self, dir_cls): | |
| 928 total = len(self._dir_order) | |
| 929 if dir_cls in self._dir_order: | |
| 930 return self._dir_order.index(dir_cls) - total | |
| 931 return total | |
| 932 | |
| 933 def setup(self, template): | |
| 934 """Convenience function to register the `Translator` filter and the | |
| 935 related directives with the given template. | |
| 936 | |
| 937 :param template: a `Template` instance | |
| 938 """ | |
| 939 template.filters.insert(0, self) | |
| 940 if hasattr(template, 'add_directives'): | |
| 941 template.add_directives(Translator.NAMESPACE, self) | |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
942 |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
943 def _extract_attrs(self, event, gettext_functions, search_text): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
944 for name, value in event[1][1]: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
945 if search_text and isinstance(value, basestring): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
946 if name in self.include_attrs: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
947 text = value.strip() |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
948 if text: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
949 yield event[2][1], None, text, [] |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
950 else: |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
951 for message in self.extract(_ensure(value), gettext_functions, |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
952 search_text=False): |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
953 yield message |
|
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
954 |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
955 |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
956 class MessageBuffer(object): |
| 738 | 957 """Helper class for managing internationalized mixed content. |
| 576 | 958 |
| 959 :since: version 0.5 | |
| 960 """ | |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
961 |
| 849 | 962 def __init__(self, directive=None): |
| 738 | 963 """Initialize the message buffer. |
| 964 | |
|
882
6f49c23045b1
Fix regression in [1099]: templates must not have an implicit loader during message extraction.
cmlenz
parents:
873
diff
changeset
|
965 :param directive: the directive owning the buffer |
|
6f49c23045b1
Fix regression in [1099]: templates must not have an implicit loader during message extraction.
cmlenz
parents:
873
diff
changeset
|
966 :type directive: I18NDirective |
| 738 | 967 """ |
| 849 | 968 # params list needs to be copied so that directives can be evaluated |
| 969 # more than once | |
| 970 self.orig_params = self.params = directive.params[:] | |
| 971 self.directive = directive | |
| 738 | 972 self.string = [] |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
973 self.events = {} |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
974 self.values = {} |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
975 self.depth = 1 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
976 self.order = 1 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
977 self.stack = [0] |
| 849 | 978 self.subdirectives = {} |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
979 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
980 def append(self, kind, data, pos): |
| 738 | 981 """Append a stream event to the buffer. |
| 982 | |
| 983 :param kind: the stream event kind | |
| 984 :param data: the event data | |
| 985 :param pos: the position of the event in the source | |
| 986 """ | |
| 849 | 987 if kind is SUB: |
| 988 # The order needs to be +1 because a new START kind event will | |
| 989 # happen and we we need to wrap those events into our custom kind(s) | |
| 990 order = self.stack[-1] + 1 | |
| 991 subdirectives, substream = data | |
| 992 # Store the directives that should be applied after translation | |
| 993 self.subdirectives.setdefault(order, []).extend(subdirectives) | |
| 994 self.events.setdefault(order, []).append((SUB_START, None, pos)) | |
| 995 for skind, sdata, spos in substream: | |
| 996 self.append(skind, sdata, spos) | |
| 997 self.events.setdefault(order, []).append((SUB_END, None, pos)) | |
| 998 elif kind is TEXT: | |
| 999 if '[' in data or ']' in data: | |
| 1000 # Quote [ and ] if it ain't us adding it, ie, if the user is | |
| 1001 # using those chars in his templates, escape them | |
| 1002 data = data.replace('[', '\[').replace(']', '\]') | |
| 738 | 1003 self.string.append(data) |
| 849 | 1004 self.events.setdefault(self.stack[-1], []).append((kind, data, pos)) |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
1005 elif kind is EXPR: |
| 849 | 1006 if self.params: |
| 1007 param = self.params.pop(0) | |
| 1008 else: | |
| 1009 params = ', '.join(['"%s"' % p for p in self.orig_params if p]) | |
| 1010 if params: | |
| 1011 params = "(%s)" % params | |
| 1012 raise IndexError("%d parameters%s given to 'i18n:%s' but " | |
| 1013 "%d or more expressions used in '%s', line %s" | |
| 1014 % (len(self.orig_params), params, | |
| 1015 self.directive.tagname, | |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
1016 len(self.orig_params) + 1, |
| 849 | 1017 os.path.basename(pos[0] or |
| 872 | 1018 'In-memory Template'), |
| 849 | 1019 pos[1])) |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
1020 self.string.append('%%(%s)s' % param) |
| 849 | 1021 self.events.setdefault(self.stack[-1], []).append((kind, data, pos)) |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
1022 self.values[param] = (kind, data, pos) |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1023 else: |
| 849 | 1024 if kind is START: |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
1025 self.string.append('[%d:' % self.order) |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1026 self.stack.append(self.order) |
| 849 | 1027 self.events.setdefault(self.stack[-1], |
| 1028 []).append((kind, data, pos)) | |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1029 self.depth += 1 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1030 self.order += 1 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1031 elif kind is END: |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1032 self.depth -= 1 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1033 if self.depth: |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1034 self.events[self.stack[-1]].append((kind, data, pos)) |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
1035 self.string.append(']') |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1036 self.stack.pop() |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1037 |
|
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1038 def format(self): |
| 738 | 1039 """Return a message identifier representing the content in the |
| 1040 buffer. | |
| 1041 """ | |
|
854
0d9e87c6cf6e
More work on reducing the size of the diff produced by 2to3.
cmlenz
parents:
853
diff
changeset
|
1042 return ''.join(self.string).strip() |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1043 |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
1044 def translate(self, string, regex=re.compile(r'%\((\w+)\)s')): |
| 738 | 1045 """Interpolate the given message translation with the events in the |
| 1046 buffer and return the translated stream. | |
| 1047 | |
| 1048 :param string: the translated message string | |
| 1049 """ | |
| 849 | 1050 substream = None |
| 1051 | |
| 1052 def yield_parts(string): | |
| 1053 for idx, part in enumerate(regex.split(string)): | |
| 1054 if idx % 2: | |
| 1055 yield self.values[part] | |
| 1056 elif part: | |
| 1057 yield (TEXT, | |
| 1058 part.replace('\[', '[').replace('\]', ']'), | |
| 1059 (None, -1, -1) | |
| 1060 ) | |
| 1061 | |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1062 parts = parse_msg(string) |
| 849 | 1063 parts_counter = {} |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1064 for order, string in parts: |
| 849 | 1065 parts_counter.setdefault(order, []).append(None) |
| 1066 | |
| 1067 while parts: | |
| 1068 order, string = parts.pop(0) | |
| 1069 if len(parts_counter[order]) == 1: | |
| 1070 events = self.events[order] | |
| 1071 else: | |
| 1072 events = [self.events[order].pop(0)] | |
| 1073 parts_counter[order].pop() | |
| 1074 | |
| 1075 for event in events: | |
| 1076 if event[0] is SUB_START: | |
| 1077 substream = [] | |
| 1078 elif event[0] is SUB_END: | |
| 1079 # Yield a substream which might have directives to be | |
| 1080 # applied to it (after translation events) | |
| 1081 yield SUB, (self.subdirectives[order], substream), event[2] | |
| 1082 substream = None | |
| 1083 elif event[0] is TEXT: | |
| 1084 if string: | |
| 1085 for part in yield_parts(string): | |
| 1086 if substream is not None: | |
| 1087 substream.append(part) | |
| 1088 else: | |
| 1089 yield part | |
| 1090 # String handled, reset it | |
| 1091 string = None | |
| 1092 elif event[0] is START: | |
| 1093 if substream is not None: | |
| 1094 substream.append(event) | |
| 1095 else: | |
| 1096 yield event | |
| 1097 if string: | |
| 1098 for part in yield_parts(string): | |
| 1099 if substream is not None: | |
| 1100 substream.append(part) | |
| 1101 else: | |
| 1102 yield part | |
| 1103 # String handled, reset it | |
| 1104 string = None | |
| 1105 elif event[0] is END: | |
| 1106 if string: | |
| 1107 for part in yield_parts(string): | |
| 1108 if substream is not None: | |
| 1109 substream.append(part) | |
| 1110 else: | |
| 1111 yield part | |
| 1112 # String handled, reset it | |
| 1113 string = None | |
| 1114 if substream is not None: | |
| 1115 substream.append(event) | |
| 1116 else: | |
| 1117 yield event | |
| 1118 elif event[0] is EXPR: | |
| 1119 # These are handled on the strings itself | |
| 1120 continue | |
|
775
f0f1416a814f
Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents:
762
diff
changeset
|
1121 else: |
| 849 | 1122 if string: |
| 1123 for part in yield_parts(string): | |
| 1124 if substream is not None: | |
| 1125 substream.append(part) | |
| 1126 else: | |
| 1127 yield part | |
| 1128 # String handled, reset it | |
| 1129 string = None | |
| 1130 if substream is not None: | |
| 1131 substream.append(event) | |
| 1132 else: | |
| 1133 yield event | |
|
560
f227a2f12e5f
Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents:
549
diff
changeset
|
1134 |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
1135 |
| 849 | 1136 def parse_msg(string, regex=re.compile(r'(?:\[(\d+)\:)|(?<!\\)\]')): |
| 738 | 1137 """Parse a translated message using Genshi mixed content message |
| 1138 formatting. | |
| 849 | 1139 |
| 738 | 1140 >>> parse_msg("See [1:Help].") |
| 1141 [(0, 'See '), (1, 'Help'), (0, '.')] | |
| 849 | 1142 |
| 738 | 1143 >>> parse_msg("See [1:our [2:Help] page] for details.") |
| 1144 [(0, 'See '), (1, 'our '), (2, 'Help'), (1, ' page'), (0, ' for details.')] | |
| 849 | 1145 |
| 738 | 1146 >>> parse_msg("[2:Details] finden Sie in [1:Hilfe].") |
| 1147 [(2, 'Details'), (0, ' finden Sie in '), (1, 'Hilfe'), (0, '.')] | |
| 849 | 1148 |
| 738 | 1149 >>> parse_msg("[1:] Bilder pro Seite anzeigen.") |
| 1150 [(1, ''), (0, ' Bilder pro Seite anzeigen.')] | |
| 849 | 1151 |
| 738 | 1152 :param string: the translated message string |
| 1153 :return: a list of ``(order, string)`` tuples | |
| 1154 :rtype: `list` | |
| 1155 """ | |
| 1156 parts = [] | |
| 1157 stack = [0] | |
| 1158 while True: | |
| 1159 mo = regex.search(string) | |
| 1160 if not mo: | |
| 1161 break | |
| 1162 | |
| 1163 if mo.start() or stack[-1]: | |
| 1164 parts.append((stack[-1], string[:mo.start()])) | |
| 1165 string = string[mo.end():] | |
| 1166 | |
| 1167 orderno = mo.group(1) | |
| 1168 if orderno is not None: | |
| 1169 stack.append(int(orderno)) | |
| 1170 else: | |
| 1171 stack.pop() | |
| 1172 if not stack: | |
| 1173 break | |
| 1174 | |
| 1175 if string: | |
| 1176 parts.append((stack[-1], string)) | |
| 1177 | |
| 1178 return parts | |
| 1179 | |
|
776
9e0ba5b9693c
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
1180 |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1181 def extract_from_code(code, gettext_functions): |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1182 """Extract strings from Python bytecode. |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1183 |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1184 >>> from genshi.template.eval import Expression |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1185 >>> expr = Expression('_("Hello")') |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
1186 >>> list(extract_from_code(expr, GETTEXT_FUNCTIONS)) |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1187 [('_', u'Hello')] |
| 849 | 1188 |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1189 >>> expr = Expression('ngettext("You have %(num)s item", ' |
|
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1190 ... '"You have %(num)s items", num)') |
|
892
1de952fd479e
i18n: Support extraction of attributes in markup embedded in ``i18n:msg`` and ``i18n:choose`` directives. See also #380.
cmlenz
parents:
891
diff
changeset
|
1191 >>> list(extract_from_code(expr, GETTEXT_FUNCTIONS)) |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1192 [('ngettext', (u'You have %(num)s item', u'You have %(num)s items', None))] |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1193 |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1194 :param code: the `Code` object |
|
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1195 :type code: `genshi.template.eval.Code` |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1196 :param gettext_functions: a sequence of function names |
| 576 | 1197 :since: version 0.5 |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1198 """ |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1199 def _walk(node): |
|
794
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1200 if isinstance(node, _ast.Call) and isinstance(node.func, _ast.Name) \ |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1201 and node.func.id in gettext_functions: |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1202 strings = [] |
|
600
0d802a7e3630
Handle starargs and dstarargs in the I18n extraction code.
cmlenz
parents:
596
diff
changeset
|
1203 def _add(arg): |
|
794
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1204 if isinstance(arg, _ast.Str) and isinstance(arg.s, basestring): |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1205 strings.append(unicode(arg.s, 'utf-8')) |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1206 elif arg: |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1207 strings.append(None) |
|
600
0d802a7e3630
Handle starargs and dstarargs in the I18n extraction code.
cmlenz
parents:
596
diff
changeset
|
1208 [_add(arg) for arg in node.args] |
|
794
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1209 _add(node.starargs) |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1210 _add(node.kwargs) |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1211 if len(strings) == 1: |
|
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1212 strings = strings[0] |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1213 else: |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1214 strings = tuple(strings) |
|
794
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1215 yield node.func.id, strings |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1216 elif node._fields: |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1217 children = [] |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1218 for field in node._fields: |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1219 child = getattr(node, field, None) |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1220 if isinstance(child, list): |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1221 for elem in child: |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1222 children.append(elem) |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1223 elif isinstance(child, _ast.AST): |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1224 children.append(child) |
|
ada9d53ea751
Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents:
790
diff
changeset
|
1225 for child in children: |
|
565
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1226 for funcname, strings in _walk(child): |
|
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1227 yield funcname, strings |
|
aa8e85a4085e
* The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents:
561
diff
changeset
|
1228 return _walk(code.ast) |
|
561
3225bb74c672
Move code for extracting messages from bytecode into a separate function.
cmlenz
parents:
560
diff
changeset
|
1229 |
|
776
9e0ba5b9693c
Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents:
775
diff
changeset
|
1230 |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1231 def extract(fileobj, keywords, comment_tags, options): |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1232 """Babel extraction method for Genshi templates. |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1233 |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1234 :param fileobj: the file-like object the messages should be extracted from |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1235 :param keywords: a list of keywords (i.e. function names) that should be |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1236 recognized as translation functions |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1237 :param comment_tags: a list of translator tags to search for and include |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1238 in the results |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1239 :param options: a dictionary of additional options (optional) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1240 :return: an iterator over ``(lineno, funcname, message, comments)`` tuples |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1241 :rtype: ``iterator`` |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1242 """ |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1243 template_class = options.get('template_class', MarkupTemplate) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1244 if isinstance(template_class, basestring): |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1245 module, clsname = template_class.split(':', 1) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1246 template_class = getattr(__import__(module, {}, {}, [clsname]), clsname) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1247 encoding = options.get('encoding', None) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1248 |
|
596
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1249 extract_text = options.get('extract_text', True) |
|
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1250 if isinstance(extract_text, basestring): |
|
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1251 extract_text = extract_text.lower() in ('1', 'on', 'yes', 'true') |
|
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1252 |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1253 ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1254 if isinstance(ignore_tags, basestring): |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1255 ignore_tags = ignore_tags.split() |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1256 ignore_tags = [QName(tag) for tag in ignore_tags] |
|
596
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1257 |
|
528
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1258 include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS) |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1259 if isinstance(include_attrs, basestring): |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1260 include_attrs = include_attrs.split() |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1261 include_attrs = [QName(attr) for attr in include_attrs] |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1262 |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1263 tmpl = template_class(fileobj, filename=getattr(fileobj, 'name', None), |
|
f38ce008ab0a
Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents:
522
diff
changeset
|
1264 encoding=encoding) |
|
882
6f49c23045b1
Fix regression in [1099]: templates must not have an implicit loader during message extraction.
cmlenz
parents:
873
diff
changeset
|
1265 tmpl.loader = None |
| 849 | 1266 |
|
596
f436c7db99f5
Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents:
594
diff
changeset
|
1267 translator = Translator(None, ignore_tags, include_attrs, extract_text) |
| 849 | 1268 if hasattr(tmpl, 'add_directives'): |
| 1269 tmpl.add_directives(Translator.NAMESPACE, translator) | |
|
787
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
1270 for message in translator.extract(tmpl.stream, gettext_functions=keywords): |
|
422a9dd01e9f
Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents:
785
diff
changeset
|
1271 yield message |