annotate genshi/filters/i18n.py @ 856:21308bd343b8 trunk

Add a couple of fallback imports for Python 3.0.
author cmlenz
date Thu, 12 Nov 2009 11:36:14 +0000
parents 4d9bef447df9
children 013623773357
rev   line source
531
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
1 # -*- coding: utf-8 -*-
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
2 #
850
e8f596c726c4 Update copyright years on files changed due to advanced-i18n merge.
cmlenz
parents: 849
diff changeset
3 # Copyright (C) 2007-2009 Edgewall Software
531
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
4 # All rights reserved.
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
5 #
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
6 # This software is licensed as described in the file COPYING, which
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
7 # you should have received as part of this distribution. The terms
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
8 # are also available at http://genshi.edgewall.org/wiki/License.
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
9 #
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
10 # This software consists of voluntary contributions made by many
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
11 # individuals. For the exact contribution history, see the revision
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
12 # history and logs, available at http://genshi.edgewall.org/log/.
9ce91447fc6b Add missing copyright header to i18n.py.
cmlenz
parents: 528
diff changeset
13
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
14 """Directives and utilities for internationalization and localization of
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
15 templates.
576
b00765a115a5 Improve docs on `Stream.select()` for #135.
cmlenz
parents: 565
diff changeset
16
b00765a115a5 Improve docs on `Stream.select()` for #135.
cmlenz
parents: 565
diff changeset
17 :since: version 0.4
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
18 :note: Directives support added since version 0.6
576
b00765a115a5 Improve docs on `Stream.select()` for #135.
cmlenz
parents: 565
diff changeset
19 """
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
20
856
21308bd343b8 Add a couple of fallback imports for Python 3.0.
cmlenz
parents: 854
diff changeset
21 try:
21308bd343b8 Add a couple of fallback imports for Python 3.0.
cmlenz
parents: 854
diff changeset
22 any
21308bd343b8 Add a couple of fallback imports for Python 3.0.
cmlenz
parents: 854
diff changeset
23 except NameError:
21308bd343b8 Add a couple of fallback imports for Python 3.0.
cmlenz
parents: 854
diff changeset
24 from genshi.util import any
788
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents: 787
diff changeset
25 from gettext import NullTranslations
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
26 import os
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
27 import re
788
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents: 787
diff changeset
28 from types import FunctionType
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
29
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
30 from genshi.core import Attrs, Namespace, QName, START, END, TEXT, START_NS, \
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
31 END_NS, XML_NAMESPACE, _ensure, StreamEventKind
794
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
32 from genshi.template.eval import _ast
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
33 from genshi.template.base import DirectiveFactory, EXPR, SUB, _apply_directives
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
34 from genshi.template.directives import Directive, StripDirective
528
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
35 from genshi.template.markup import MarkupTemplate, EXEC
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
36
528
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
37 __all__ = ['Translator', 'extract']
501
5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions.
cmlenz
parents: 493
diff changeset
38 __docformat__ = 'restructuredtext en'
5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions.
cmlenz
parents: 493
diff changeset
39
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
40
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
41 I18N_NAMESPACE = Namespace('http://genshi.edgewall.org/i18n')
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
42
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
43 MSGBUF = StreamEventKind('MSGBUF')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
44 SUB_START = StreamEventKind('SUB_START')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
45 SUB_END = StreamEventKind('SUB_END')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
46
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
47
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
48 class I18NDirective(Directive):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
49 """Simple interface for i18n directives to support messages extraction."""
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
50
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
51 def __call__(self, stream, directives, ctxt, **vars):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
52 return _apply_directives(stream, directives, ctxt, vars)
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
53
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
54
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
55 class ExtractableI18NDirective(I18NDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
56 """Simple interface for directives to support messages extraction."""
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
57
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
58 def extract(self, stream, comment_stack):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
59 raise NotImplementedError
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
60
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
61
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
62 class CommentDirective(I18NDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
63 """Implementation of the ``i18n:comment`` template directive which adds
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
64 translation comments.
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
65
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
66 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
67 ... <p i18n:comment="As in Foo Bar">Foo</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
68 ... </html>''')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
69 >>> translator = Translator()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
70 >>> translator.setup(tmpl)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
71 >>> list(translator.extract(tmpl.stream))
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
72 [(2, None, u'Foo', [u'As in Foo Bar'])]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
73 """
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
74 __slots__ = ['comment']
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
75
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
76 def __init__(self, value, template, hints=None, namespaces=None,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
77 lineno=-1, offset=-1):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
78 Directive.__init__(self, None, template, namespaces, lineno, offset)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
79 self.comment = value
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
80
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
81
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
82 class MsgDirective(ExtractableI18NDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
83 r"""Implementation of the ``i18n:msg`` directive which marks inner content
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
84 as translatable. Consider the following examples:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
85
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
86 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
87 ... <div i18n:msg="">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
88 ... <p>Foo</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
89 ... <p>Bar</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
90 ... </div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
91 ... <p i18n:msg="">Foo <em>bar</em>!</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
92 ... </html>''')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
93
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
94 >>> translator = Translator()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
95 >>> translator.setup(tmpl)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
96 >>> list(translator.extract(tmpl.stream))
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
97 [(2, None, u'[1:Foo]\n [2:Bar]', []), (6, None, u'Foo [1:bar]!', [])]
853
f33ecf3c319e Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents: 850
diff changeset
98 >>> print(tmpl.generate().render())
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
99 <html>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
100 <div><p>Foo</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
101 <p>Bar</p></div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
102 <p>Foo <em>bar</em>!</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
103 </html>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
104
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
105 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
106 ... <div i18n:msg="fname, lname">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
107 ... <p>First Name: ${fname}</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
108 ... <p>Last Name: ${lname}</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
109 ... </div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
110 ... <p i18n:msg="">Foo <em>bar</em>!</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
111 ... </html>''')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
112 >>> translator.setup(tmpl)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
113 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
114 [(2, None, u'[1:First Name: %(fname)s]\n [2:Last Name: %(lname)s]', []),
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
115 (6, None, u'Foo [1:bar]!', [])]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
116
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
117 >>> tmpl = MarkupTemplate('''<html xmlns:i18n="http://genshi.edgewall.org/i18n">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
118 ... <div i18n:msg="fname, lname">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
119 ... <p>First Name: ${fname}</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
120 ... <p>Last Name: ${lname}</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
121 ... </div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
122 ... <p i18n:msg="">Foo <em>bar</em>!</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
123 ... </html>''')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
124 >>> translator.setup(tmpl)
853
f33ecf3c319e Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents: 850
diff changeset
125 >>> print(tmpl.generate(fname='John', lname='Doe').render())
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
126 <html>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
127 <div><p>First Name: John</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
128 <p>Last Name: Doe</p></div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
129 <p>Foo <em>bar</em>!</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
130 </html>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
131
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
132 Starting and ending white-space is stripped of to make it simpler for
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
133 translators. Stripping it is not that important since it's on the html
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
134 source, the rendered output will remain the same.
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
135 """
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
136 __slots__ = ['params']
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
137
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
138 def __init__(self, value, template, hints=None, namespaces=None,
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
139 lineno=-1, offset=-1):
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
140 Directive.__init__(self, None, template, namespaces, lineno, offset)
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
141 self.params = [param.strip() for param in value.split(',') if param]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
142
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
143 @classmethod
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
144 def attach(cls, template, stream, value, namespaces, pos):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
145 if type(value) is dict:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
146 value = value.get('params', '').strip()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
147 return super(MsgDirective, cls).attach(template, stream, value.strip(),
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
148 namespaces, pos)
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
149
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
150 def __call__(self, stream, directives, ctxt, **vars):
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
151 gettext = ctxt.get('_i18n.gettext')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
152 dgettext = ctxt.get('_i18n.dgettext')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
153 if ctxt.get('_i18n.domain'):
854
4d9bef447df9 More work on reducing the size of the diff produced by 2to3.
cmlenz
parents: 853
diff changeset
154 assert hasattr(dgettext, '__call__'), \
4d9bef447df9 More work on reducing the size of the diff produced by 2to3.
cmlenz
parents: 853
diff changeset
155 'No domain gettext function passed'
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
156 gettext = lambda msg: dgettext(ctxt.get('_i18n.domain'), msg)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
157
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
158 def _generate():
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
159 msgbuf = MessageBuffer(self)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
160 previous = stream.next()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
161 if previous[0] is START:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
162 yield previous
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
163 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
164 msgbuf.append(*previous)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
165 previous = stream.next()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
166 for kind, data, pos in stream:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
167 msgbuf.append(*previous)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
168 previous = kind, data, pos
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
169 if previous[0] is not END:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
170 msgbuf.append(*previous)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
171 previous = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
172 for event in msgbuf.translate(gettext(msgbuf.format())):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
173 yield event
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
174 if previous:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
175 yield previous
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
176
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
177 return _apply_directives(_generate(), directives, ctxt, vars)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
178
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
179 def extract(self, stream, comment_stack):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
180 msgbuf = MessageBuffer(self)
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
181
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
182 stream = iter(stream)
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
183 previous = stream.next()
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
184 if previous[0] is START:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
185 previous = stream.next()
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
186 for event in stream:
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
187 msgbuf.append(*previous)
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
188 previous = event
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
189 if previous[0] is not END:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
190 msgbuf.append(*previous)
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
191
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
192 yield None, msgbuf.format(), comment_stack[-1:]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
193
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
194
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
195 class ChooseBranchDirective(I18NDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
196 __slots__ = ['params']
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
197
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
198 def __call__(self, stream, directives, ctxt, **vars):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
199 self.params = ctxt.get('_i18n.choose.params', [])[:]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
200 msgbuf = MessageBuffer(self)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
201
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
202 stream = iter(_apply_directives(stream, directives, ctxt, vars))
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
203 yield stream.next() # the outer start tag
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
204 previous = stream.next()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
205 for kind, data, pos in stream:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
206 msgbuf.append(*previous)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
207 previous = kind, data, pos
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
208 yield MSGBUF, (), -1 # the place holder for msgbuf output
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
209 yield previous # the outer end tag
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
210 ctxt['_i18n.choose.%s' % type(self).__name__] = msgbuf
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
211
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
212
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
213 def extract(self, stream, comment_stack, msgbuf):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
214 stream = iter(stream)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
215 previous = stream.next()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
216 if previous[0] is START:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
217 previous = stream.next()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
218 for event in stream:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
219 msgbuf.append(*previous)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
220 previous = event
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
221 if previous[0] is not END:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
222 msgbuf.append(*previous)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
223 return msgbuf
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
224
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
225
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
226 class SingularDirective(ChooseBranchDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
227 """Implementation of the ``i18n:singular`` directive to be used with the
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
228 ``i18n:choose`` directive."""
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
229
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
230
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
231 class PluralDirective(ChooseBranchDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
232 """Implementation of the ``i18n:plural`` directive to be used with the
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
233 ``i18n:choose`` directive."""
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
234
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
235
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
236 class ChooseDirective(ExtractableI18NDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
237 """Implementation of the ``i18n:choose`` directive which provides plural
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
238 internationalisation of strings.
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
239
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
240 This directive requires at least one parameter, the one which evaluates to
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
241 an integer which will allow to choose the plural/singular form. If you also
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
242 have expressions inside the singular and plural version of the string you
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
243 also need to pass a name for those parameters. Consider the following
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
244 examples:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
245
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
246 >>> tmpl = MarkupTemplate('''\
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
247 <html xmlns:i18n="http://genshi.edgewall.org/i18n">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
248 ... <div i18n:choose="num; num">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
249 ... <p i18n:singular="">There is $num coin</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
250 ... <p i18n:plural="">There are $num coins</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
251 ... </div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
252 ... </html>''')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
253 >>> translator = Translator()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
254 >>> translator.setup(tmpl)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
255 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
256 [(2, 'ngettext', (u'There is %(num)s coin',
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
257 u'There are %(num)s coins'), [])]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
258
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
259 >>> tmpl = MarkupTemplate('''\
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
260 <html xmlns:i18n="http://genshi.edgewall.org/i18n">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
261 ... <div i18n:choose="num; num">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
262 ... <p i18n:singular="">There is $num coin</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
263 ... <p i18n:plural="">There are $num coins</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
264 ... </div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
265 ... </html>''')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
266 >>> translator.setup(tmpl)
853
f33ecf3c319e Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents: 850
diff changeset
267 >>> print(tmpl.generate(num=1).render())
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
268 <html>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
269 <div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
270 <p>There is 1 coin</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
271 </div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
272 </html>
853
f33ecf3c319e Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents: 850
diff changeset
273 >>> print(tmpl.generate(num=2).render())
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
274 <html>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
275 <div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
276 <p>There are 2 coins</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
277 </div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
278 </html>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
279
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
280 When used as a directive and not as an attribute:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
281
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
282 >>> tmpl = MarkupTemplate('''\
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
283 <html xmlns:i18n="http://genshi.edgewall.org/i18n">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
284 ... <i18n:choose numeral="num" params="num">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
285 ... <p i18n:singular="">There is $num coin</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
286 ... <p i18n:plural="">There are $num coins</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
287 ... </i18n:choose>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
288 ... </html>''')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
289 >>> translator.setup(tmpl)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
290 >>> list(translator.extract(tmpl.stream)) #doctest: +NORMALIZE_WHITESPACE
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
291 [(2, 'ngettext', (u'There is %(num)s coin',
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
292 u'There are %(num)s coins'), [])]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
293 """
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
294 __slots__ = ['numeral', 'params']
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
295
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
296 def __init__(self, value, template, hints=None, namespaces=None,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
297 lineno=-1, offset=-1):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
298 Directive.__init__(self, None, template, namespaces, lineno, offset)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
299 params = [v.strip() for v in value.split(';')]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
300 self.numeral = self._parse_expr(params.pop(0), template, lineno, offset)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
301 self.params = params and [name.strip() for name in
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
302 params[0].split(',') if name] or []
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
303
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
304 @classmethod
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
305 def attach(cls, template, stream, value, namespaces, pos):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
306 if type(value) is dict:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
307 numeral = value.get('numeral', '').strip()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
308 assert numeral is not '', "at least pass the numeral param"
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
309 params = [v.strip() for v in value.get('params', '').split(',')]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
310 value = '%s; ' % numeral + ', '.join(params)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
311 return super(ChooseDirective, cls).attach(template, stream, value,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
312 namespaces, pos)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
313
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
314 def __call__(self, stream, directives, ctxt, **vars):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
315 ctxt.push({'_i18n.choose.params': self.params,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
316 '_i18n.choose.SingularDirective': None,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
317 '_i18n.choose.PluralDirective': None})
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
318
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
319 new_stream = []
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
320 singular_stream = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
321 singular_msgbuf = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
322 plural_stream = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
323 plural_msgbuf = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
324
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
325 ngettext = ctxt.get('_i18n.ungettext')
854
4d9bef447df9 More work on reducing the size of the diff produced by 2to3.
cmlenz
parents: 853
diff changeset
326 assert hasattr(ngettext, '__call__'), 'No ngettext function available'
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
327 dngettext = ctxt.get('_i18n.dngettext')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
328 if not dngettext:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
329 dngettext = lambda d, s, p, n: ngettext(s, p, n)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
330
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
331 for kind, event, pos in stream:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
332 if kind is SUB:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
333 subdirectives, substream = event
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
334 if isinstance(subdirectives[0],
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
335 SingularDirective) and not singular_stream:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
336 # Apply directives to update context
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
337 singular_stream = list(_apply_directives(substream,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
338 subdirectives,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
339 ctxt, vars))
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
340 new_stream.append((MSGBUF, (), ('', -1))) # msgbuf place holder
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
341 singular_msgbuf = ctxt.get('_i18n.choose.SingularDirective')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
342 elif isinstance(subdirectives[0],
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
343 PluralDirective) and not plural_stream:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
344 # Apply directives to update context
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
345 plural_stream = list(_apply_directives(substream,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
346 subdirectives,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
347 ctxt, vars))
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
348 plural_msgbuf = ctxt.get('_i18n.choose.PluralDirective')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
349 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
350 new_stream.append((kind, event, pos))
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
351 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
352 new_stream.append((kind, event, pos))
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
353
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
354 if ctxt.get('_i18n.domain'):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
355 ngettext = lambda s, p, n: dngettext(ctxt.get('_i18n.domain'),
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
356 s, p, n)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
357
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
358 for kind, data, pos in new_stream:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
359 if kind is MSGBUF:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
360 for skind, sdata, spos in singular_stream:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
361 if skind is MSGBUF:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
362 translation = ngettext(singular_msgbuf.format(),
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
363 plural_msgbuf.format(),
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
364 self.numeral.evaluate(ctxt))
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
365 for event in singular_msgbuf.translate(translation):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
366 yield event
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
367 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
368 yield skind, sdata, spos
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
369 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
370 yield kind, data, pos
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
371
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
372 ctxt.pop()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
373
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
374 def extract(self, stream, comment_stack):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
375 stream = iter(stream)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
376 previous = stream.next()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
377 if previous is START:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
378 stream.next()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
379
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
380 singular_msgbuf = MessageBuffer(self)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
381 plural_msgbuf = MessageBuffer(self)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
382
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
383 for kind, event, pos in stream:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
384 if kind is SUB:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
385 subdirectives, substream = event
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
386 for subdirective in subdirectives:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
387 if isinstance(subdirective, SingularDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
388 singular_msgbuf = subdirective.extract(substream, comment_stack,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
389 singular_msgbuf)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
390 elif isinstance(subdirective, PluralDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
391 plural_msgbuf = subdirective.extract(substream, comment_stack,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
392 plural_msgbuf)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
393 elif not isinstance(subdirective, StripDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
394 singular_msgbuf.append(kind, event, pos)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
395 plural_msgbuf.append(kind, event, pos)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
396 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
397 singular_msgbuf.append(kind, event, pos)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
398 plural_msgbuf.append(kind, event, pos)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
399
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
400 yield 'ngettext', \
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
401 (singular_msgbuf.format(), plural_msgbuf.format()), \
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
402 comment_stack[-1:]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
403
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
404
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
405 class DomainDirective(I18NDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
406 """Implementation of the ``i18n:domain`` directive which allows choosing
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
407 another i18n domain(catalog) to translate from.
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
408
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
409 >>> from genshi.filters.tests.i18n import DummyTranslations
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
410 >>> tmpl = MarkupTemplate('''\
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
411 <html xmlns:i18n="http://genshi.edgewall.org/i18n">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
412 ... <p i18n:msg="">Bar</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
413 ... <div i18n:domain="foo">
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
414 ... <p i18n:msg="">FooBar</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
415 ... <p>Bar</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
416 ... <p i18n:domain="bar" i18n:msg="">Bar</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
417 ... <p i18n:domain="">Bar</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
418 ... </div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
419 ... <p>Bar</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
420 ... </html>''')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
421
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
422 >>> translations = DummyTranslations({'Bar': 'Voh'})
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
423 >>> translations.add_domain('foo', {'FooBar': 'BarFoo', 'Bar': 'foo_Bar'})
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
424 >>> translations.add_domain('bar', {'Bar': 'bar_Bar'})
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
425 >>> translator = Translator(translations)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
426 >>> translator.setup(tmpl)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
427
853
f33ecf3c319e Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents: 850
diff changeset
428 >>> print(tmpl.generate().render())
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
429 <html>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
430 <p>Voh</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
431 <div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
432 <p>BarFoo</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
433 <p>foo_Bar</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
434 <p>bar_Bar</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
435 <p>Voh</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
436 </div>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
437 <p>Voh</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
438 </html>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
439 """
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
440 __slots__ = ['domain']
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
441
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
442 def __init__(self, value, template, hints=None, namespaces=None,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
443 lineno=-1, offset=-1):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
444 Directive.__init__(self, None, template, namespaces, lineno, offset)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
445 self.domain = value and value.strip() or '__DEFAULT__'
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
446
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
447 @classmethod
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
448 def attach(cls, template, stream, value, namespaces, pos):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
449 if type(value) is dict:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
450 value = value.get('name')
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
451 return super(DomainDirective, cls).attach(template, stream, value,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
452 namespaces, pos)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
453
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
454 def __call__(self, stream, directives, ctxt, **vars):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
455 ctxt.push({'_i18n.domain': self.domain})
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
456 for event in _apply_directives(stream, directives, ctxt, vars):
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
457 yield event
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
458 ctxt.pop()
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
459
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
460
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
461 class Translator(DirectiveFactory):
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
462 """Can extract and translate localizable strings from markup streams and
450
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
463 templates.
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
464
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
465 For example, assume the following template:
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
466
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
467 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
468 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
469 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
470 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
471 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
472 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
473 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
474 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
475 ... </html>''', filename='example.html')
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
476
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
477 For demonstration, we define a dummy ``gettext``-style function with a
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
478 hard-coded translation table, and pass that to the `Translator` initializer:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
479
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
480 >>> def pseudo_gettext(string):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
481 ... return {
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
482 ... 'Example': 'Beispiel',
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
483 ... 'Hello, %(name)s': 'Hallo, %(name)s'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
484 ... }[string]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
485 >>> translator = Translator(pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
486
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
487 Next, the translator needs to be prepended to any already defined filters
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
488 on the template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
489
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
490 >>> tmpl.filters.insert(0, translator)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
491
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
492 When generating the template output, our hard-coded translations should be
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
493 applied as expected:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
494
853
f33ecf3c319e Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents: 850
diff changeset
495 >>> print(tmpl.generate(username='Hans', _=pseudo_gettext))
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
496 <html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
497 <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
498 <title>Beispiel</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
499 </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
500 <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
501 <h1>Beispiel</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
502 <p>Hallo, Hans</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
503 </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
504 </html>
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
505
522
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
506 Note that elements defining ``xml:lang`` attributes that do not contain
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
507 variable expressions are ignored by this filter. That can be used to
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
508 exclude specific parts of a template from being extracted and translated.
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
509 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
510
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
511 directives = [
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
512 ('domain', DomainDirective),
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
513 ('comment', CommentDirective),
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
514 ('msg', MsgDirective),
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
515 ('choose', ChooseDirective),
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
516 ('singular', SingularDirective),
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
517 ('plural', PluralDirective)
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
518 ]
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
519
450
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
520 IGNORE_TAGS = frozenset([
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
521 QName('script'), QName('http://www.w3.org/1999/xhtml}script'),
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
522 QName('style'), QName('http://www.w3.org/1999/xhtml}style')
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
523 ])
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
524 INCLUDE_ATTRS = frozenset([
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
525 'abbr', 'alt', 'label', 'prompt', 'standby', 'summary', 'title'
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
526 ])
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
527 NAMESPACE = I18N_NAMESPACE
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
528
788
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents: 787
diff changeset
529 def __init__(self, translate=NullTranslations(), ignore_tags=IGNORE_TAGS,
594
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents: 576
diff changeset
530 include_attrs=INCLUDE_ATTRS, extract_text=True):
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
531 """Initialize the translator.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
532
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
533 :param translate: the translation function, for example ``gettext`` or
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
534 ``ugettext``.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
535 :param ignore_tags: a set of tag names that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
536 :param include_attrs: a set of attribute names should be localized
594
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents: 576
diff changeset
537 :param extract_text: whether the content of text nodes should be
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents: 576
diff changeset
538 extracted, or only text in explicit ``gettext``
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents: 576
diff changeset
539 function calls
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
540
788
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents: 787
diff changeset
541 :note: Changed in 0.6: the `translate` parameter can now be either
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents: 787
diff changeset
542 a ``gettext``-style function, or an object compatible with the
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents: 787
diff changeset
543 ``NullTransalations`` or ``GNUTranslations`` interface
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
544 """
450
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
545 self.translate = translate
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
546 self.ignore_tags = ignore_tags
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
547 self.include_attrs = include_attrs
594
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents: 576
diff changeset
548 self.extract_text = extract_text
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
549
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
550 def __call__(self, stream, ctxt=None, search_text=True):
448
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
551 """Translate any localizable strings in the given stream.
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
552
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
553 This function shouldn't be called directly. Instead, an instance of
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
554 the `Translator` class should be registered as a filter with the
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
555 `Template` or the `TemplateLoader`, or applied as a regular stream
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
556 filter. If used as a template filter, it should be inserted in front of
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
557 all the default filters.
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
558
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
559 :param stream: the markup event stream
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
560 :param ctxt: the template context (not used)
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
561 :param search_text: whether text nodes should be translated (used
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
562 internally)
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
563 :return: the localized stream
1154f2aadb6c Add support for HTML5 doctype.
cmlenz
parents: 446
diff changeset
564 """
450
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
565 ignore_tags = self.ignore_tags
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
566 include_attrs = self.include_attrs
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
567 skip = 0
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
568 xml_lang = XML_NAMESPACE['lang']
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
569
788
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents: 787
diff changeset
570 if type(self.translate) is FunctionType:
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents: 787
diff changeset
571 gettext = self.translate
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
572 if ctxt:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
573 ctxt['_i18n.gettext'] = gettext
788
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents: 787
diff changeset
574 else:
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents: 787
diff changeset
575 gettext = self.translate.ugettext
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
576 try:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
577 dgettext = self.translate.dugettext
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
578 except AttributeError:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
579 dgettext = lambda x, y: gettext(y)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
580 ngettext = self.translate.ungettext
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
581 try:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
582 dngettext = self.translate.dungettext
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
583 except AttributeError:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
584 dngettext = lambda d, s, p, n: ngettext(s, p, n)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
585
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
586 if ctxt:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
587 ctxt['_i18n.gettext'] = gettext
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
588 ctxt['_i18n.ugettext'] = gettext
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
589 ctxt['_i18n.dgettext'] = dgettext
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
590 ctxt['_i18n.ngettext'] = ngettext
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
591 ctxt['_i18n.ungettext'] = ngettext
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
592 ctxt['_i18n.dngettext'] = dngettext
787
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
593
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
594 extract_text = self.extract_text
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
595 if not extract_text:
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
596 search_text = False
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
597
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
598 if ctxt and ctxt.get('_i18n.domain'):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
599 old_gettext = gettext
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
600 gettext = lambda msg: dgettext(ctxt.get('_i18n.domain'), msg)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
601
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
602 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
603
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
604 # skip chunks that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
605 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
606 if kind is START:
522
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
607 skip += 1
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
608 elif kind is END:
522
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
609 skip -= 1
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
610 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
611 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
612
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
613 # handle different events that can be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
614 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
615 tag, attrs = data
522
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
616 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
617 isinstance(attrs.get(xml_lang), basestring):
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
618 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
619 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
620 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
621
493
3f6582a5a4a5 Fix another bug in the translation filter: translated attributes were getting added instead of replaced.
cmlenz
parents: 485
diff changeset
622 new_attrs = []
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
623 changed = False
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
624
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
625 for name, value in attrs:
483
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents: 481
diff changeset
626 newval = value
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
627 if extract_text and isinstance(value, basestring):
483
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents: 481
diff changeset
628 if name in include_attrs:
788
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.
cmlenz
parents: 787
diff changeset
629 newval = gettext(value)
483
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents: 481
diff changeset
630 else:
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
631 newval = list(
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
632 self(_ensure(value), ctxt, search_text=False)
483
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents: 481
diff changeset
633 )
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents: 481
diff changeset
634 if newval != value:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents: 481
diff changeset
635 value = newval
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents: 481
diff changeset
636 changed = True
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
637 new_attrs.append((name, value))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
638 if changed:
667
c9a084ffaee6 Fix case where attributes weren't properly wrapped in an `Attrs` instance if one or more of them were translated by the I18n filter, potentially breaking things further down the chain. Closes #162.
cmlenz
parents: 600
diff changeset
639 attrs = Attrs(new_attrs)
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
640
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
641 yield kind, (tag, attrs), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
642
485
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents: 483
diff changeset
643 elif search_text and kind is TEXT:
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
644 text = data.strip()
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
645 if text:
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
646 data = data.replace(text, unicode(gettext(text)))
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
647 yield kind, data, pos
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
648
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
649 elif kind is SUB:
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
650 directives, substream = data
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
651 current_domain = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
652 for idx, directive in enumerate(directives):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
653 # Organize directives to make everything work
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
654 if isinstance(directive, DomainDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
655 # Grab current domain and update context
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
656 current_domain = directive.domain
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
657 ctxt.push({'_i18n.domain': current_domain})
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
658 # Put domain directive as the first one in order to
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
659 # update context before any other directives evaluation
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
660 directives.insert(0, directives.pop(idx))
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
661
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
662 # If this is an i18n directive, no need to translate text
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
663 # nodes here
856
21308bd343b8 Add a couple of fallback imports for Python 3.0.
cmlenz
parents: 854
diff changeset
664 is_i18n_directive = any([
21308bd343b8 Add a couple of fallback imports for Python 3.0.
cmlenz
parents: 854
diff changeset
665 isinstance(d, ExtractableI18NDirective)
21308bd343b8 Add a couple of fallback imports for Python 3.0.
cmlenz
parents: 854
diff changeset
666 for d in directives
21308bd343b8 Add a couple of fallback imports for Python 3.0.
cmlenz
parents: 854
diff changeset
667 ])
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
668 substream = list(self(substream, ctxt,
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
669 search_text=not is_i18n_directive))
790
da90cee22560 Merged the custom-directives branch back into trunk.
cmlenz
parents: 788
diff changeset
670 yield kind, (directives, substream), pos
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
671
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
672 if current_domain:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
673 ctxt.pop()
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
674 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
675 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
676
450
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
677 GETTEXT_FUNCTIONS = ('_', 'gettext', 'ngettext', 'dgettext', 'dngettext',
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
678 'ugettext', 'ungettext')
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
679
485
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents: 483
diff changeset
680 def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS,
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
681 search_text=True, msgbuf=None, comment_stack=None):
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
682 """Extract localizable strings from the given template stream.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
683
450
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
684 For every string found, this function yields a ``(lineno, function,
787
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
685 message, comments)`` tuple, where:
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
686
450
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
687 * ``lineno`` is the number of the line on which the string was found,
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
688 * ``function`` is the name of the ``gettext`` function used (if the
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
689 string was extracted from embedded Python code), and
469
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents: 467
diff changeset
690 * ``message`` is the string itself (a ``unicode`` object, or a tuple
787
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
691 of ``unicode`` objects for functions with multiple string
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
692 arguments).
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
693 * ``comments`` is a list of comments related to the message, extracted
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
694 from ``i18n:comment`` attributes found in the markup
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
695
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
696 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
697 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
698 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
699 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
700 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
701 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
702 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
469
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents: 467
diff changeset
703 ... <p>${ngettext("You have %d item", "You have %d items", num)}</p>
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
704 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
705 ... </html>''', filename='example.html')
787
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
706 >>> for line, func, msg, comments in Translator().extract(tmpl.stream):
853
f33ecf3c319e Convert a bunch of print statements to py3k compatible syntax.
cmlenz
parents: 850
diff changeset
707 ... print('%d, %r, %r' % (line, func, msg))
450
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
708 3, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
709 6, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
710 7, '_', u'Hello, %(name)s'
565
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
711 8, 'ngettext', (u'You have %d item', u'You have %d items', None)
469
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents: 467
diff changeset
712
450
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
713 :param stream: the event stream to extract strings from; can be a
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
714 regular stream or a template stream
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
715 :param gettext_functions: a sequence of function names that should be
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
716 treated as gettext-style localization
94601511cd68 Extend the I18n extraction to also yield function names if applicable.
cmlenz
parents: 448
diff changeset
717 functions
485
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents: 483
diff changeset
718 :param search_text: whether the content of text nodes should be
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents: 483
diff changeset
719 extracted (used internally)
469
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents: 467
diff changeset
720
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents: 467
diff changeset
721 :note: Changed in 0.4.1: For a function with multiple string arguments
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents: 467
diff changeset
722 (such as ``ngettext``), a single item with a tuple of strings is
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions.
cmlenz
parents: 467
diff changeset
723 yielded, instead an item for each string argument.
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
724 :note: Changed in 0.6: The returned tuples now include a fourth
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
725 element, which is a list of comments for the translator.
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
726 """
594
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents: 576
diff changeset
727 if not self.extract_text:
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents: 576
diff changeset
728 search_text = False
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
729 if comment_stack is None:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
730 comment_stack = []
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
731 skip = 0
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
732
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
733 # Un-comment bellow to extract messages without adding directives
522
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
734 xml_lang = XML_NAMESPACE['lang']
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
735
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
736 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
737 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
738 if kind is START:
522
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
739 skip += 1
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
740 if kind is END:
522
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
741 skip -= 1
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
742
549
7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132.
cmlenz
parents: 535
diff changeset
743 if kind is START and not skip:
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
744 tag, attrs = data
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
745
522
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
746 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.
cmlenz
parents: 501
diff changeset
747 isinstance(attrs.get(xml_lang), basestring):
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
748 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
749 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
750
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
751 for name, value in attrs:
594
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls.
cmlenz
parents: 576
diff changeset
752 if search_text and isinstance(value, basestring):
483
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents: 481
diff changeset
753 if name in self.include_attrs:
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
754 text = value.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
755 if text:
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
756 # XXX: Do we need to grab i18n:comment from comment_stack ???
787
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
757 yield pos[1], None, text, []
483
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter.
cmlenz
parents: 481
diff changeset
758 else:
787
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
759 for lineno, funcname, text, comments in self.extract(
485
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included.
cmlenz
parents: 483
diff changeset
760 _ensure(value), gettext_functions,
535
35a413f3f1dd The I18n filter no longer extracts or translates literal strings in attribute values that also contain expressions.
cmlenz
parents: 531
diff changeset
761 search_text=False):
787
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
762 yield lineno, funcname, text, comments
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
763
561
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
764 if msgbuf:
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
765 msgbuf.append(kind, data, pos)
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
766
549
7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132.
cmlenz
parents: 535
diff changeset
767 elif not skip and search_text and kind is TEXT:
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
768 if not msgbuf:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
769 text = data.strip()
854
4d9bef447df9 More work on reducing the size of the diff produced by 2to3.
cmlenz
parents: 853
diff changeset
770 if text and [ch for ch in text if ch.isalpha()]:
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
771 yield pos[1], None, text, comment_stack[-1:]
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
772 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
773 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
774
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
775 elif not skip and msgbuf and kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
776 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
777 if not msgbuf.depth:
854
4d9bef447df9 More work on reducing the size of the diff produced by 2to3.
cmlenz
parents: 853
diff changeset
778 yield msgbuf.lineno, None, msgbuf.format(), [
4d9bef447df9 More work on reducing the size of the diff produced by 2to3.
cmlenz
parents: 853
diff changeset
779 c for c in msgbuf.comment if c
4d9bef447df9 More work on reducing the size of the diff produced by 2to3.
cmlenz
parents: 853
diff changeset
780 ]
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
781 msgbuf = None
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
782
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
783 elif kind is EXPR or kind is EXEC:
775
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents: 762
diff changeset
784 if msgbuf:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents: 762
diff changeset
785 msgbuf.append(kind, data, pos)
565
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
786 for funcname, strings in extract_from_code(data,
561
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
787 gettext_functions):
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
788 # XXX: Do we need to grab i18n:comment from comment_stack ???
787
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
789 yield pos[1], funcname, strings, []
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
790
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
791 elif kind is SUB:
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
792 directives, substream = data
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
793 in_comment = False
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
794
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
795 for idx, directive in enumerate(directives):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
796 # Do a first loop to see if there's a comment directive
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
797 # If there is update context and pop it from directives
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
798 if isinstance(directive, CommentDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
799 in_comment = True
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
800 comment_stack.append(directive.comment)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
801 if len(directives) == 1:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
802 # in case we're in the presence of something like:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
803 # <p i18n:comment="foo">Foo</p>
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
804 messages = self.extract(
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
805 substream, gettext_functions,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
806 search_text=search_text and not skip,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
807 msgbuf=msgbuf, comment_stack=comment_stack)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
808 for lineno, funcname, text, comments in messages:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
809 yield lineno, funcname, text, comments
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
810 directives.pop(idx)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
811 elif not isinstance(directive, I18NDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
812 # Remove all other non i18n directives from the process
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
813 directives.pop(idx)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
814
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
815 if not directives and not in_comment:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
816 # Extract content if there's no directives because
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
817 # strip was pop'ed and not because comment was pop'ed.
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
818 # Extraction in this case has been taken care of.
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
819 messages = self.extract(
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
820 substream, gettext_functions,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
821 search_text=search_text and not skip, msgbuf=msgbuf)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
822 for lineno, funcname, text, comments in messages:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
823 yield lineno, funcname, text, comments
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
824
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
825 for directive in directives:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
826 if isinstance(directive, ExtractableI18NDirective):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
827 messages = directive.extract(substream, comment_stack)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
828 for funcname, text, comments in messages:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
829 yield pos[1], funcname, text, comments
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
830 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
831 messages = self.extract(
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
832 substream, gettext_functions,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
833 search_text=search_text and not skip, msgbuf=msgbuf)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
834 for lineno, funcname, text, comments in messages:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
835 yield lineno, funcname, text, comments
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
836
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
837 if in_comment:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
838 comment_stack.pop()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
839
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
840 def get_directive_index(self, dir_cls):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
841 total = len(self._dir_order)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
842 if dir_cls in self._dir_order:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
843 return self._dir_order.index(dir_cls) - total
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
844 return total
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
845
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
846 def setup(self, template):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
847 """Convenience function to register the `Translator` filter and the
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
848 related directives with the given template.
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
850 :param template: a `Template` instance
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
851 """
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
852 template.filters.insert(0, self)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
853 if hasattr(template, 'add_directives'):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
854 template.add_directives(Translator.NAMESPACE, self)
528
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
855
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
856
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
857 class MessageBuffer(object):
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
858 """Helper class for managing internationalized mixed content.
576
b00765a115a5 Improve docs on `Stream.select()` for #135.
cmlenz
parents: 565
diff changeset
859
b00765a115a5 Improve docs on `Stream.select()` for #135.
cmlenz
parents: 565
diff changeset
860 :since: version 0.5
b00765a115a5 Improve docs on `Stream.select()` for #135.
cmlenz
parents: 565
diff changeset
861 """
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
862
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
863 def __init__(self, directive=None):
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
864 """Initialize the message buffer.
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
865
776
ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents: 775
diff changeset
866 :param params: comma-separated list of parameter names
ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents: 775
diff changeset
867 :type params: `basestring`
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
868 :param lineno: the line number on which the first stream event
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
869 belonging to the message was found
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
870 """
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
871 # params list needs to be copied so that directives can be evaluated
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
872 # more than once
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
873 self.orig_params = self.params = directive.params[:]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
874 self.directive = directive
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
875 self.string = []
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
876 self.events = {}
775
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents: 762
diff changeset
877 self.values = {}
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
878 self.depth = 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
879 self.order = 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
880 self.stack = [0]
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
881 self.subdirectives = {}
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
882
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
883 def append(self, kind, data, pos):
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
884 """Append a stream event to the buffer.
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
885
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
886 :param kind: the stream event kind
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
887 :param data: the event data
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
888 :param pos: the position of the event in the source
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
889 """
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
890 if kind is SUB:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
891 # The order needs to be +1 because a new START kind event will
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
892 # happen and we we need to wrap those events into our custom kind(s)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
893 order = self.stack[-1] + 1
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
894 subdirectives, substream = data
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
895 # Store the directives that should be applied after translation
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
896 self.subdirectives.setdefault(order, []).extend(subdirectives)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
897 self.events.setdefault(order, []).append((SUB_START, None, pos))
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
898 for skind, sdata, spos in substream:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
899 self.append(skind, sdata, spos)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
900 self.events.setdefault(order, []).append((SUB_END, None, pos))
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
901 elif kind is TEXT:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
902 if '[' in data or ']' in data:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
903 # Quote [ and ] if it ain't us adding it, ie, if the user is
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
904 # using those chars in his templates, escape them
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
905 data = data.replace('[', '\[').replace(']', '\]')
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
906 self.string.append(data)
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
907 self.events.setdefault(self.stack[-1], []).append((kind, data, pos))
775
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents: 762
diff changeset
908 elif kind is EXPR:
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
909 if self.params:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
910 param = self.params.pop(0)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
911 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
912 params = ', '.join(['"%s"' % p for p in self.orig_params if p])
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
913 if params:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
914 params = "(%s)" % params
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
915 raise IndexError("%d parameters%s given to 'i18n:%s' but "
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
916 "%d or more expressions used in '%s', line %s"
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
917 % (len(self.orig_params), params,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
918 self.directive.tagname,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
919 len(self.orig_params)+1,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
920 os.path.basename(pos[0] or
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
921 'In Memmory Template'),
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
922 pos[1]))
775
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents: 762
diff changeset
923 self.string.append('%%(%s)s' % param)
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
924 self.events.setdefault(self.stack[-1], []).append((kind, data, pos))
775
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents: 762
diff changeset
925 self.values[param] = (kind, data, pos)
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
926 else:
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
927 if kind is START:
854
4d9bef447df9 More work on reducing the size of the diff produced by 2to3.
cmlenz
parents: 853
diff changeset
928 self.string.append('[%d:' % self.order)
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
929 self.stack.append(self.order)
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
930 self.events.setdefault(self.stack[-1],
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
931 []).append((kind, data, pos))
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
932 self.depth += 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
933 self.order += 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
934 elif kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
935 self.depth -= 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
936 if self.depth:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
937 self.events[self.stack[-1]].append((kind, data, pos))
854
4d9bef447df9 More work on reducing the size of the diff produced by 2to3.
cmlenz
parents: 853
diff changeset
938 self.string.append(']')
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
939 self.stack.pop()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
940
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
941 def format(self):
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
942 """Return a message identifier representing the content in the
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
943 buffer.
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
944 """
854
4d9bef447df9 More work on reducing the size of the diff produced by 2to3.
cmlenz
parents: 853
diff changeset
945 return ''.join(self.string).strip()
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
946
775
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents: 762
diff changeset
947 def translate(self, string, regex=re.compile(r'%\((\w+)\)s')):
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
948 """Interpolate the given message translation with the events in the
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
949 buffer and return the translated stream.
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
950
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
951 :param string: the translated message string
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
952 """
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
953 substream = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
954
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
955 def yield_parts(string):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
956 for idx, part in enumerate(regex.split(string)):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
957 if idx % 2:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
958 yield self.values[part]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
959 elif part:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
960 yield (TEXT,
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
961 part.replace('\[', '[').replace('\]', ']'),
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
962 (None, -1, -1)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
963 )
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
964
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
965 parts = parse_msg(string)
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
966 parts_counter = {}
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
967 for order, string in parts:
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
968 parts_counter.setdefault(order, []).append(None)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
969
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
970 while parts:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
971 order, string = parts.pop(0)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
972 if len(parts_counter[order]) == 1:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
973 events = self.events[order]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
974 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
975 events = [self.events[order].pop(0)]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
976 parts_counter[order].pop()
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
977
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
978 for event in events:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
979 if event[0] is SUB_START:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
980 substream = []
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
981 elif event[0] is SUB_END:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
982 # Yield a substream which might have directives to be
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
983 # applied to it (after translation events)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
984 yield SUB, (self.subdirectives[order], substream), event[2]
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
985 substream = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
986 elif event[0] is TEXT:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
987 if string:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
988 for part in yield_parts(string):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
989 if substream is not None:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
990 substream.append(part)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
991 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
992 yield part
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
993 # String handled, reset it
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
994 string = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
995 elif event[0] is START:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
996 if substream is not None:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
997 substream.append(event)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
998 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
999 yield event
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1000 if string:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1001 for part in yield_parts(string):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1002 if substream is not None:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1003 substream.append(part)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1004 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1005 yield part
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1006 # String handled, reset it
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1007 string = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1008 elif event[0] is END:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1009 if string:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1010 for part in yield_parts(string):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1011 if substream is not None:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1012 substream.append(part)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1013 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1014 yield part
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1015 # String handled, reset it
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1016 string = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1017 if substream is not None:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1018 substream.append(event)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1019 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1020 yield event
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1021 elif event[0] is EXPR:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1022 # These are handled on the strings itself
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1023 continue
775
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129.
cmlenz
parents: 762
diff changeset
1024 else:
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1025 if string:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1026 for part in yield_parts(string):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1027 if substream is not None:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1028 substream.append(part)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1029 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1030 yield part
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1031 # String handled, reset it
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1032 string = None
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1033 if substream is not None:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1034 substream.append(event)
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1035 else:
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1036 yield event
560
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay.
cmlenz
parents: 549
diff changeset
1037
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1038 def parse_msg(string, regex=re.compile(r'(?:\[(\d+)\:)|(?<!\\)\]')):
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1039 """Parse a translated message using Genshi mixed content message
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1040 formatting.
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1041
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1042 >>> parse_msg("See [1:Help].")
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1043 [(0, 'See '), (1, 'Help'), (0, '.')]
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1044
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1045 >>> parse_msg("See [1:our [2:Help] page] for details.")
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1046 [(0, 'See '), (1, 'our '), (2, 'Help'), (1, ' page'), (0, ' for details.')]
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1047
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1048 >>> parse_msg("[2:Details] finden Sie in [1:Hilfe].")
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1049 [(2, 'Details'), (0, ' finden Sie in '), (1, 'Hilfe'), (0, '.')]
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1050
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1051 >>> parse_msg("[1:] Bilder pro Seite anzeigen.")
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1052 [(1, ''), (0, ' Bilder pro Seite anzeigen.')]
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1053
738
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1054 :param string: the translated message string
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1055 :return: a list of ``(order, string)`` tuples
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1056 :rtype: `list`
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1057 """
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1058 parts = []
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1059 stack = [0]
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1060 while True:
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1061 mo = regex.search(string)
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1062 if not mo:
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1063 break
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1064
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1065 if mo.start() or stack[-1]:
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1066 parts.append((stack[-1], string[:mo.start()]))
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1067 string = string[mo.end():]
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1068
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1069 orderno = mo.group(1)
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1070 if orderno is not None:
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1071 stack.append(int(orderno))
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1072 else:
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1073 stack.pop()
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1074 if not stack:
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1075 break
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1076
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1077 if string:
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1078 parts.append((stack[-1], string))
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1079
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1080 return parts
3b8a38fcc1ab Minor cleanup in the i18n module.
cmlenz
parents: 667
diff changeset
1081
776
ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents: 775
diff changeset
1082
561
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1083 def extract_from_code(code, gettext_functions):
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1084 """Extract strings from Python bytecode.
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1085
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1086 >>> from genshi.template.eval import Expression
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1087 >>> expr = Expression('_("Hello")')
565
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1088 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS))
561
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1089 [('_', u'Hello')]
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1090
561
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1091 >>> expr = Expression('ngettext("You have %(num)s item", '
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1092 ... '"You have %(num)s items", num)')
565
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1093 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS))
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1094 [('ngettext', (u'You have %(num)s item', u'You have %(num)s items', None))]
561
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1095
565
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1096 :param code: the `Code` object
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1097 :type code: `genshi.template.eval.Code`
561
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1098 :param gettext_functions: a sequence of function names
576
b00765a115a5 Improve docs on `Stream.select()` for #135.
cmlenz
parents: 565
diff changeset
1099 :since: version 0.5
561
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1100 """
565
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1101 def _walk(node):
794
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1102 if isinstance(node, _ast.Call) and isinstance(node.func, _ast.Name) \
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1103 and node.func.id in gettext_functions:
565
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1104 strings = []
600
23a4784203ae Handle starargs and dstarargs in the I18n extraction code.
cmlenz
parents: 596
diff changeset
1105 def _add(arg):
794
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1106 if isinstance(arg, _ast.Str) and isinstance(arg.s, basestring):
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1107 strings.append(unicode(arg.s, 'utf-8'))
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1108 elif arg:
565
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1109 strings.append(None)
600
23a4784203ae Handle starargs and dstarargs in the I18n extraction code.
cmlenz
parents: 596
diff changeset
1110 [_add(arg) for arg in node.args]
794
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1111 _add(node.starargs)
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1112 _add(node.kwargs)
565
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1113 if len(strings) == 1:
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1114 strings = strings[0]
561
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1115 else:
565
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1116 strings = tuple(strings)
794
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1117 yield node.func.id, strings
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1118 elif node._fields:
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1119 children = []
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1120 for field in node._fields:
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1121 child = getattr(node, field, None)
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1122 if isinstance(child, list):
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1123 for elem in child:
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1124 children.append(elem)
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1125 elif isinstance(child, _ast.AST):
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1126 children.append(child)
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).
cmlenz
parents: 790
diff changeset
1127 for child in children:
565
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1128 for funcname, strings in _walk(child):
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1129 yield funcname, strings
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments.
cmlenz
parents: 561
diff changeset
1130 return _walk(code.ast)
561
21b473243e63 Move code for extracting messages from bytecode into a separate function.
cmlenz
parents: 560
diff changeset
1131
776
ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129.
cmlenz
parents: 775
diff changeset
1132
528
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1133 def extract(fileobj, keywords, comment_tags, options):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1134 """Babel extraction method for Genshi templates.
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1135
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1136 :param fileobj: the file-like object the messages should be extracted from
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1137 :param keywords: a list of keywords (i.e. function names) that should be
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1138 recognized as translation functions
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1139 :param comment_tags: a list of translator tags to search for and include
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1140 in the results
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1141 :param options: a dictionary of additional options (optional)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1142 :return: an iterator over ``(lineno, funcname, message, comments)`` tuples
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1143 :rtype: ``iterator``
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1144 """
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1145 template_class = options.get('template_class', MarkupTemplate)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1146 if isinstance(template_class, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1147 module, clsname = template_class.split(':', 1)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1148 template_class = getattr(__import__(module, {}, {}, [clsname]), clsname)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1149 encoding = options.get('encoding', None)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1150
596
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents: 594
diff changeset
1151 extract_text = options.get('extract_text', True)
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents: 594
diff changeset
1152 if isinstance(extract_text, basestring):
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents: 594
diff changeset
1153 extract_text = extract_text.lower() in ('1', 'on', 'yes', 'true')
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents: 594
diff changeset
1154
528
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1155 ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1156 if isinstance(ignore_tags, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1157 ignore_tags = ignore_tags.split()
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1158 ignore_tags = [QName(tag) for tag in ignore_tags]
596
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents: 594
diff changeset
1159
528
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1160 include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1161 if isinstance(include_attrs, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1162 include_attrs = include_attrs.split()
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1163 include_attrs = [QName(attr) for attr in include_attrs]
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1164
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1165 tmpl = template_class(fileobj, filename=getattr(fileobj, 'name', None),
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page.
cmlenz
parents: 522
diff changeset
1166 encoding=encoding)
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1167
596
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin.
cmlenz
parents: 594
diff changeset
1168 translator = Translator(None, ignore_tags, include_attrs, extract_text)
849
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1169 if hasattr(tmpl, 'add_directives'):
878c4313c7d5 Merged advanced-i18n branch back into trunk.
cmlenz
parents: 794
diff changeset
1170 tmpl.add_directives(Translator.NAMESPACE, translator)
787
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
1171 for message in translator.extract(tmpl.stream, gettext_functions=keywords):
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.
cmlenz
parents: 785
diff changeset
1172 yield message
Copyright (C) 2012-2017 Edgewall Software