Mercurial > genshi > mirror

annotate genshi/filters/i18n.py @ 446:fd9c4f7a249a trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
author cmlenz
date Fri, 13 Apr 2007 11:02:36 +0000
parents
children 1154f2aadb6c
rev   line source
446
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
1 """Utilities for internationalization and localization of templates."""
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
2
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
3 try:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
4 frozenset
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
5 except NameError:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
6 from sets import ImmutableSet as frozenset
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
7 from gettext import gettext
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
8 from opcode import opmap
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
9 import re
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
10
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
11 from genshi.core import Attrs, START, END, TEXT
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
12 from genshi.template.base import Template, EXPR, SUB
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
13 from genshi.template.markup import EXEC
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
14
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
15 LOAD_NAME = chr(opmap['LOAD_NAME'])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
16 LOAD_CONST = chr(opmap['LOAD_CONST'])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
17 CALL_FUNCTION = chr(opmap['CALL_FUNCTION'])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
18 BINARY_ADD = chr(opmap['BINARY_ADD'])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
19
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
20
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
21 class Translator(object):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
22 """Can extract and translate localizable strings from markup streams and
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
23 templates
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
24
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
25 For example, assume the followng template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
26
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
27 >>> from genshi.template import MarkupTemplate
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
28 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
29 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
30 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
31 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
32 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
33 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
34 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
35 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
36 ... </html>''', filename='example.html')
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
37
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
38 For demonstration, we define a dummy ``gettext``-style function with a
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
39 hard-coded translation table, and pass that to the `Translator` initializer:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
40
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
41 >>> def pseudo_gettext(string):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
42 ... return {
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
43 ... 'Example': 'Beispiel',
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
44 ... 'Hello, %(name)s': 'Hallo, %(name)s'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
45 ... }[string]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
46 >>> translator = Translator(pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
47
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
48 Next, the translator needs to be prepended to any already defined filters
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
49 on the template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
50
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
51 >>> tmpl.filters.insert(0, translator)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
52
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
53 When generating the template output, our hard-coded translations should be
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
54 applied as expected:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
55
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
56 >>> print tmpl.generate(username='Hans', _=pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
57 <html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
58 <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
59 <title>Beispiel</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
60 </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
61 <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
62 <h1>Beispiel</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
63 <p>Hallo, Hans</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
64 </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
65 </html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
66 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
67
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
68 IGNORE_TAGS = frozenset(['script', 'style'])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
69 INCLUDE_ATTRS = frozenset(['title', 'alt'])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
70
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
71 def __init__(self, translate=gettext, ignore_tags=IGNORE_TAGS,
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
72 include_attrs=INCLUDE_ATTRS):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
73 """Initialize the translator.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
74
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
75 :param translate: the translation function, for example ``gettext`` or
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
76 ``ugettext``.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
77 :param ignore_tags: a set of tag names that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
78 :param include_attrs: a set of attribute names should be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
79 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
80 self.gettext = translate
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
81 self.ignore_tags = ignore_tags
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
82 self.include_attrs = include_attrs
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
83
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
84 def __call__(self, stream, ctxt=None, search_text=True):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
85 skip = 0
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
86
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
87 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
88
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
89 # skip chunks that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
90 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
91 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
92 tag, attrs = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
93 tag = tag.localname
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
94 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
95 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
96 elif kind is END:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
97 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
98 skip -= 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
99 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
100 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
101
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
102 # handle different events that can be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
103 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
104 tag, attrs = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
105 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
106 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
107 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
108 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
109
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
110 new_attrs = list(attrs)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
111 changed = False
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
112 for name, value in attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
113 if name in include_attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
114 if isinstance(value, basestring):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
115 newval = ugettext(value)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
116 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
117 newval = list(self(value, ctxt, search_text=name in self.include_attrs))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
118 if newval != value:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
119 value = new_val
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
120 changed = True
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
121 new_attrs.append((name, value))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
122 if changed:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
123 attrs = new_attrs
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
124
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
125 yield kind, (tag, attrs), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
126
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
127 elif kind is TEXT:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
128 text = data.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
129 if text:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
130 data = data.replace(text, self.gettext(text))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
131 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
132
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
133 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
134 subkind, substream = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
135 new_substream = list(self(substream, ctxt))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
136 yield kind, (subkind, new_substream), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
137
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
138 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
139 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
140
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
141 def extract(self, stream, gettext_functions=('_', 'gettext', 'ngettext')):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
142 """Extract localizable strings from the given template stream.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
143
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
144 For every string found, this function yields a ``(lineno, message)``
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
145 tuple.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
146
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
147 :param stream: the event stream to extract strings from; can be a
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
148 regular stream or a template stream
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
149
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
150 >>> from genshi.template import MarkupTemplate
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
151 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
152 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
153 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
154 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
155 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
156 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
157 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
158 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
159 ... </html>''', filename='example.html')
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
160 >>> for lineno, message in Translator().extract(tmpl.stream):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
161 ... print "Line %d: %r" % (lineno, message)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
162 Line 3: u'Example'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
163 Line 6: u'Example'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
164 Line 7: u'Hello, %(name)s'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
165 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
166 tagname = None
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
167 skip = 0
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
168
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
169 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
170 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
171 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
172 tag, attrs = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
173 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
174 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
175 if kind is END:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
176 tag = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
177 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
178 skip -= 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
179 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
180
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
181 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
182 tag, attrs = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
183 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
184 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
185 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
186
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
187 for name, value in attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
188 if name in self.include_attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
189 if isinstance(value, basestring):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
190 text = value.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
191 if text:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
192 yield pos[1], text
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
193 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
194 for lineno, text in harvest(value):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
195 yield lineno, text
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
196
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
197 elif kind is TEXT:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
198 text = data.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
199 if text and filter(None, [ch.isalpha() for ch in text]):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
200 yield pos[1], text
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
201
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
202 elif kind is EXPR or kind is EXEC:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
203 consts = dict([(n, chr(i) + '\x00') for i, n in
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
204 enumerate(data.code.co_consts)])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
205 gettext_locs = [consts[n] for n in gettext_functions
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
206 if n in consts]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
207 ops = [
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
208 LOAD_CONST, '(', '|'.join(gettext_locs), ')',
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
209 CALL_FUNCTION, '.\x00',
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
210 '((?:', BINARY_ADD, '|', LOAD_CONST, '.\x00)+)'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
211 ]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
212 for _, opcodes in re.findall(''.join(ops), data.code.co_code):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
213 strings = []
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
214 opcodes = iter(opcodes)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
215 for opcode in opcodes:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
216 if opcode == BINARY_ADD:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
217 arg = strings.pop()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
218 strings[-1] += arg
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
219 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
220 arg = data.code.co_consts[ord(opcodes.next())]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
221 opcodes.next() # skip second byte
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
222 if not isinstance(arg, basestring):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
223 break
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
224 strings.append(unicode(arg))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
225 for string in strings:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
226 yield pos[1], string
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
227
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
228 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
229 subkind, substream = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
230 for lineno, text in self.harvest(substream):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization.
cmlenz
parents:
diff changeset
231 yield lineno, text
Copyright (C) 2012-2017 Edgewall Software