genshi/mirror: genshi/filters/i18n.py annotate

annotate genshi/filters/i18n.py @ 450:94601511cd68 trunk

Extend the I18n extraction to also yield function names if applicable.

author	cmlenz
date	Fri, 13 Apr 2007 20:58:48 +0000
parents	1154f2aadb6c
children	4b6dc4978691

rev	line source
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	1 """Utilities for internationalization and localization of templates."""
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	2
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	3 try:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	4 frozenset
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	5 except NameError:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	6 from sets import ImmutableSet as frozenset
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	7 from gettext import gettext
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	8 from opcode import opmap
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	9 import re
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	10
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	11 from genshi.core import Attrs, Namespace, QName, START, END, TEXT
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	12 from genshi.template.base import Template, EXPR, SUB
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	13 from genshi.template.markup import EXEC
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	14
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	15 _LOAD_NAME = chr(opmap['LOAD_NAME'])
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	16 _LOAD_CONST = chr(opmap['LOAD_CONST'])
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	17 _CALL_FUNCTION = chr(opmap['CALL_FUNCTION'])
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	18 _BINARY_ADD = chr(opmap['BINARY_ADD'])
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	19
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	20
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	21 class Translator(object):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	22 """Can extract and translate localizable strings from markup streams and
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	23 templates.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	24
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	25 For example, assume the followng template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	26
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	27 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	28 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	29 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	30 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	31 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	32 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	33 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	34 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	35 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	36 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	37 ... </html>''', filename='example.html')
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	38
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	39 For demonstration, we define a dummy ``gettext``-style function with a
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	40 hard-coded translation table, and pass that to the `Translator` initializer:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	41
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	42 >>> def pseudo_gettext(string):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	43 ... return {
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	44 ... 'Example': 'Beispiel',
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	45 ... 'Hello, %(name)s': 'Hallo, %(name)s'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	46 ... }[string]
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	47 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	48 >>> translator = Translator(pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	49
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	50 Next, the translator needs to be prepended to any already defined filters
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	51 on the template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	52
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	53 >>> tmpl.filters.insert(0, translator)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	54
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	55 When generating the template output, our hard-coded translations should be
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	56 applied as expected:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	57
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	58 >>> print tmpl.generate(username='Hans', _=pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	59 <html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	60 <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	61 <title>Beispiel</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	62 </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	63 <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	64 <h1>Beispiel</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	65 <p>Hallo, Hans</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	66 </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	67 </html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	68 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	69
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	70 IGNORE_TAGS = frozenset([
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	71 QName('script'), QName('http://www.w3.org/1999/xhtml}script'),
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	72 QName('style'), QName('http://www.w3.org/1999/xhtml}style')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	73 ])
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	74 INCLUDE_ATTRS = frozenset(['title', 'alt'])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	75
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	76 def __init__(self, translate=gettext, ignore_tags=IGNORE_TAGS,
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	77 include_attrs=INCLUDE_ATTRS):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	78 """Initialize the translator.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	79
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	80 :param translate: the translation function, for example ``gettext`` or
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	81 ``ugettext``.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	82 :param ignore_tags: a set of tag names that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	83 :param include_attrs: a set of attribute names should be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	84 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	85 self.translate = translate
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	86 self.ignore_tags = ignore_tags
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	87 self.include_attrs = include_attrs
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	88
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	89 def __call__(self, stream, ctxt=None, search_text=True):
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	90 """Translate any localizable strings in the given stream.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	91
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	92 This function shouldn't be called directly. Instead, an instance of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	93 the `Translator` class should be registered as a filter with the
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	94 `Template` or the `TemplateLoader`, or applied as a regular stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	95 filter. If used as a template filter, it should be inserted in front of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	96 all the default filters.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	97
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	98 :param stream: the markup event stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	99 :param ctxt: the template context (not used)
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	100 :param search_text: whether text nodes should be translated (used
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	101 internally)
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	102 :return: the localized stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	103 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	104 ignore_tags = self.ignore_tags
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	105 include_attrs = self.include_attrs
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	106 translate = self.translate
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	107 skip = 0
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	108
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	109 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	110
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	111 # skip chunks that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	112 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	113 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	114 tag, attrs = data
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	115 if tag in ignore_tags:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	116 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	117 elif kind is END:
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	118 if tag in ignore_tags:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	119 skip -= 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	120 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	121 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	122
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	123 # handle different events that can be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	124 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	125 tag, attrs = data
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	126 if tag in ignore_tags:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	127 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	128 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	129 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	130
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	131 new_attrs = list(attrs)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	132 changed = False
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	133 for name, value in attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	134 if name in include_attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	135 if isinstance(value, basestring):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	136 newval = ugettext(value)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	137 else:
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	138 newval = list(self(value, ctxt, search_text=name in include_attrs))
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	139 if newval != value:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	140 value = new_val
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	141 changed = True
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	142 new_attrs.append((name, value))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	143 if changed:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	144 attrs = new_attrs
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	145
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	146 yield kind, (tag, attrs), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	147
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	148 elif kind is TEXT:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	149 text = data.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	150 if text:
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	151 data = data.replace(text, translate(text))
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	152 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	153
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	154 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	155 subkind, substream = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	156 new_substream = list(self(substream, ctxt))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	157 yield kind, (subkind, new_substream), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	158
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	159 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	160 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	161
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	162 GETTEXT_FUNCTIONS = ('_', 'gettext', 'ngettext', 'dgettext', 'dngettext',
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	163 'ugettext', 'ungettext')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	164
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	165 def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	166 """Extract localizable strings from the given template stream.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	167
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	168 For every string found, this function yields a ``(lineno, function,
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	169 message)`` tuple, where:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	170
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	171 * ``lineno`` is the number of the line on which the string was found,
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	172 * ``function`` is the name of the ``gettext`` function used (if the
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	173 string was extracted from embedded Python code), and
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	174 * ``message`` is the string itself (a ``unicode`` object).
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	175
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	176 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	177 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	178 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	179 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	180 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	181 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	182 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	183 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	184 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	185 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	186 ... </html>''', filename='example.html')
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	187 >>>
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	188 >>> for lineno, funcname, message in Translator().extract(tmpl.stream):
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	189 ... print "%d, %r, %r" % (lineno, funcname, message)
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	190 3, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	191 6, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	192 7, '_', u'Hello, %(name)s'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	193
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	194 :param stream: the event stream to extract strings from; can be a
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	195 regular stream or a template stream
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	196 :param gettext_functions: a sequence of function names that should be
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	197 treated as gettext-style localization
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	198 functions
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	199 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	200 tagname = None
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	201 skip = 0
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	202
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	203 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	204 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	205 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	206 tag, attrs = data
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	207 if tag in self.ignore_tags:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	208 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	209 if kind is END:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	210 tag = data
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	211 if tag in self.ignore_tags:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	212 skip -= 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	213 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	214
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	215 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	216 tag, attrs = data
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	217 if tag in self.ignore_tags:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	218 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	219 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	220
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	221 for name, value in attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	222 if name in self.include_attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	223 if isinstance(value, basestring):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	224 text = value.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	225 if text:
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	226 yield pos[1], None, text
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	227 else:
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	228 for lineno, funcname, text in harvest(value):
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	229 yield lineno, funcname, text
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	230
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	231 elif kind is TEXT:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	232 text = data.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	233 if text and filter(None, [ch.isalpha() for ch in text]):
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	234 yield pos[1], None, text
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	235
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	236 elif kind is EXPR or kind is EXEC:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	237 consts = dict([(n, chr(i) + '\x00') for i, n in
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	238 enumerate(data.code.co_consts)])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	239 gettext_locs = [consts[n] for n in gettext_functions
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	240 if n in consts]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	241 ops = [
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	242 _LOAD_CONST, '(', '\|'.join(gettext_locs), ')',
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	243 _CALL_FUNCTION, '.\x00',
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	244 '((?:', _BINARY_ADD, '\|', _LOAD_CONST, '.\x00)+)'
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	245 ]
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	246 for loc, opcodes in re.findall(''.join(ops), data.code.co_code):
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	247 funcname = data.code.co_consts[ord(loc[0])]
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	248 strings = []
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	249 opcodes = iter(opcodes)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	250 for opcode in opcodes:
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	251 if opcode == _BINARY_ADD:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	252 arg = strings.pop()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	253 strings[-1] += arg
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	254 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	255 arg = data.code.co_consts[ord(opcodes.next())]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	256 opcodes.next() # skip second byte
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	257 if not isinstance(arg, basestring):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	258 break
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	259 strings.append(unicode(arg))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	260 for string in strings:
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	261 yield pos[1], funcname, string
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	262
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	263 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	264 subkind, substream = data
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	265 for lineno, funcname, text in self.harvest(substream):
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	266 yield lineno, funcname, text

Mercurial > genshi > mirror

annotate genshi/filters/i18n.py @ 450:94601511cd68 trunk