genshi/mirror: genshi/filters/i18n.py annotate

annotate genshi/filters/i18n.py @ 522:082535e5087c trunk

The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization.

author	cmlenz
date	Fri, 15 Jun 2007 20:17:56 +0000
parents	5e7604c2d60d
children	24df908da22d

rev	line source
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	1 """Utilities for internationalization and localization of templates."""
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	2
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	3 try:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	4 frozenset
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	5 except NameError:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	6 from sets import ImmutableSet as frozenset
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	7 from gettext import gettext
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	8 from opcode import opmap
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	9 import re
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	10
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	11 from genshi.core import Attrs, Namespace, QName, START, END, TEXT, \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	12 XML_NAMESPACE, _ensure
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	13 from genshi.template.base import Template, EXPR, SUB
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	14 from genshi.template.markup import EXEC
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	15
501 5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions. cmlenz parents: 493 diff changeset	16 __all__ = ['Translator']
5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions. cmlenz parents: 493 diff changeset	17 __docformat__ = 'restructuredtext en'
5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions. cmlenz parents: 493 diff changeset	18
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	19 _LOAD_NAME = chr(opmap['LOAD_NAME'])
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	20 _LOAD_CONST = chr(opmap['LOAD_CONST'])
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	21 _CALL_FUNCTION = chr(opmap['CALL_FUNCTION'])
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	22 _BINARY_ADD = chr(opmap['BINARY_ADD'])
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	23
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	24
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	25 class Translator(object):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	26 """Can extract and translate localizable strings from markup streams and
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	27 templates.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	28
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	29 For example, assume the followng template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	30
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	31 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	32 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	33 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	34 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	35 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	36 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	37 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	38 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	39 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	40 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	41 ... </html>''', filename='example.html')
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	42
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	43 For demonstration, we define a dummy ``gettext``-style function with a
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	44 hard-coded translation table, and pass that to the `Translator` initializer:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	45
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	46 >>> def pseudo_gettext(string):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	47 ... return {
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	48 ... 'Example': 'Beispiel',
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	49 ... 'Hello, %(name)s': 'Hallo, %(name)s'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	50 ... }[string]
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	51 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	52 >>> translator = Translator(pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	53
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	54 Next, the translator needs to be prepended to any already defined filters
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	55 on the template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	56
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	57 >>> tmpl.filters.insert(0, translator)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	58
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	59 When generating the template output, our hard-coded translations should be
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	60 applied as expected:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	61
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	62 >>> print tmpl.generate(username='Hans', _=pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	63 <html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	64 <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	65 <title>Beispiel</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	66 </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	67 <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	68 <h1>Beispiel</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	69 <p>Hallo, Hans</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	70 </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	71 </html>
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	72
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	73 Note that elements defining ``xml:lang`` attributes that do not contain
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	74 variable expressions are ignored by this filter. That can be used to
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	75 exclude specific parts of a template from being extracted and translated.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	76 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	77
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	78 IGNORE_TAGS = frozenset([
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	79 QName('script'), QName('http://www.w3.org/1999/xhtml}script'),
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	80 QName('style'), QName('http://www.w3.org/1999/xhtml}style')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	81 ])
467 23082baddbf9 Add some more localizable HTML attributes to the I18n filter. cmlenz parents: 466 diff changeset	82 INCLUDE_ATTRS = frozenset(['abbr', 'alt', 'label', 'prompt', 'standby',
23082baddbf9 Add some more localizable HTML attributes to the I18n filter. cmlenz parents: 466 diff changeset	83 'summary', 'title'])
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	84
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	85 def __init__(self, translate=gettext, ignore_tags=IGNORE_TAGS,
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	86 include_attrs=INCLUDE_ATTRS):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	87 """Initialize the translator.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	88
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	89 :param translate: the translation function, for example ``gettext`` or
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	90 ``ugettext``.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	91 :param ignore_tags: a set of tag names that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	92 :param include_attrs: a set of attribute names should be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	93 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	94 self.translate = translate
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	95 self.ignore_tags = ignore_tags
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	96 self.include_attrs = include_attrs
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	97
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	98 def __call__(self, stream, ctxt=None, search_text=True):
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	99 """Translate any localizable strings in the given stream.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	100
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	101 This function shouldn't be called directly. Instead, an instance of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	102 the `Translator` class should be registered as a filter with the
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	103 `Template` or the `TemplateLoader`, or applied as a regular stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	104 filter. If used as a template filter, it should be inserted in front of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	105 all the default filters.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	106
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	107 :param stream: the markup event stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	108 :param ctxt: the template context (not used)
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	109 :param search_text: whether text nodes should be translated (used
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	110 internally)
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	111 :return: the localized stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	112 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	113 ignore_tags = self.ignore_tags
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	114 include_attrs = self.include_attrs
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	115 translate = self.translate
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	116 skip = 0
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	117 xml_lang = XML_NAMESPACE['lang']
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	118
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	119 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	120
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	121 # skip chunks that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	122 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	123 if kind is START:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	124 skip += 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	125 elif kind is END:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	126 skip -= 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	127 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	128 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	129
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	130 # handle different events that can be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	131 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	132 tag, attrs = data
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	133 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	134 isinstance(attrs.get(xml_lang), basestring):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	135 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	136 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	137 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	138
493 3f6582a5a4a5 Fix another bug in the translation filter: translated attributes were getting added instead of replaced. cmlenz parents: 485 diff changeset	139 new_attrs = []
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	140 changed = False
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	141 for name, value in attrs:
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	142 newval = value
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	143 if isinstance(value, basestring):
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	144 if name in include_attrs:
456 4b6dc4978691 Fix incorrect reference to translation function in the I18N filter. cmlenz parents: 450 diff changeset	145 newval = self.translate(value)
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	146 else:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	147 newval = list(self(_ensure(value), ctxt,
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	148 search_text=name in include_attrs)
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	149 )
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	150 if newval != value:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	151 value = newval
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	152 changed = True
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	153 new_attrs.append((name, value))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	154 if changed:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	155 attrs = new_attrs
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	156
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	157 yield kind, (tag, attrs), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	158
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	159 elif search_text and kind is TEXT:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	160 text = data.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	161 if text:
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	162 data = data.replace(text, translate(text))
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	163 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	164
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	165 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	166 subkind, substream = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	167 new_substream = list(self(substream, ctxt))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	168 yield kind, (subkind, new_substream), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	169
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	170 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	171 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	172
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	173 GETTEXT_FUNCTIONS = ('_', 'gettext', 'ngettext', 'dgettext', 'dngettext',
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	174 'ugettext', 'ungettext')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	175
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	176 def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS,
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	177 search_text=True):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	178 """Extract localizable strings from the given template stream.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	179
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	180 For every string found, this function yields a ``(lineno, function,
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	181 message)`` tuple, where:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	182
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	183 * ``lineno`` is the number of the line on which the string was found,
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	184 * ``function`` is the name of the ``gettext`` function used (if the
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	185 string was extracted from embedded Python code), and
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	186 * ``message`` is the string itself (a ``unicode`` object, or a tuple
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	187 of ``unicode`` objects for functions with multiple string arguments).
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	188
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	189 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	190 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	191 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	192 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	193 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	194 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	195 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	196 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	197 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	198 ... <p>${ngettext("You have %d item", "You have %d items", num)}</p>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	199 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	200 ... </html>''', filename='example.html')
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	201 >>>
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	202 >>> for lineno, funcname, message in Translator().extract(tmpl.stream):
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	203 ... print "%d, %r, %r" % (lineno, funcname, message)
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	204 3, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	205 6, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	206 7, '_', u'Hello, %(name)s'
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	207 8, 'ngettext', (u'You have %d item', u'You have %d items')
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	208
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	209 :param stream: the event stream to extract strings from; can be a
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	210 regular stream or a template stream
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	211 :param gettext_functions: a sequence of function names that should be
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	212 treated as gettext-style localization
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	213 functions
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	214 :param search_text: whether the content of text nodes should be
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	215 extracted (used internally)
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	216
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	217 :note: Changed in 0.4.1: For a function with multiple string arguments
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	218 (such as ``ngettext``), a single item with a tuple of strings is
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	219 yielded, instead an item for each string argument.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	220 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	221 tagname = None
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	222 skip = 0
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	223 xml_lang = XML_NAMESPACE['lang']
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	224
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	225 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	226 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	227 if kind is START:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	228 skip += 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	229 if kind is END:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	230 skip -= 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	231 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	232
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	233 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	234 tag, attrs = data
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	235 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	236 isinstance(attrs.get(xml_lang), basestring):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	237 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	238 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	239
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	240 for name, value in attrs:
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	241 if isinstance(value, basestring):
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	242 if name in self.include_attrs:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	243 text = value.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	244 if text:
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	245 yield pos[1], None, text
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	246 else:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	247 for lineno, funcname, text in self.extract(
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	248 _ensure(value), gettext_functions,
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	249 search_text=name in self.include_attrs):
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	250 yield lineno, funcname, text
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	251
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	252 elif search_text and kind is TEXT:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	253 text = data.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	254 if text and filter(None, [ch.isalpha() for ch in text]):
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	255 yield pos[1], None, text
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	256
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	257 elif kind is EXPR or kind is EXEC:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	258 consts = dict([(n, chr(i) + '\x00') for i, n in
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	259 enumerate(data.code.co_consts)])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	260 gettext_locs = [consts[n] for n in gettext_functions
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	261 if n in consts]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	262 ops = [
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	263 _LOAD_CONST, '(', '\|'.join(gettext_locs), ')',
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	264 _CALL_FUNCTION, '.\x00',
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	265 '((?:', _BINARY_ADD, '\|', _LOAD_CONST, '.\x00)+)'
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	266 ]
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	267 for loc, opcodes in re.findall(''.join(ops), data.code.co_code):
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	268 funcname = data.code.co_consts[ord(loc[0])]
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	269 strings = []
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	270 opcodes = iter(opcodes)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	271 for opcode in opcodes:
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	272 if opcode == _BINARY_ADD:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	273 arg = strings.pop()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	274 strings[-1] += arg
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	275 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	276 arg = data.code.co_consts[ord(opcodes.next())]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	277 opcodes.next() # skip second byte
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	278 if not isinstance(arg, basestring):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	279 break
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	280 strings.append(unicode(arg))
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	281 if len(strings) == 1:
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	282 strings = strings[0]
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	283 else:
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	284 strings = tuple(strings)
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	285 yield pos[1], funcname, strings
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	286
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	287 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	288 subkind, substream = data
466 015419437645 Fix another bad ref in the I18n filter. cmlenz parents: 456 diff changeset	289 for lineno, funcname, text in self.extract(substream,
015419437645 Fix another bad ref in the I18n filter. cmlenz parents: 456 diff changeset	290 gettext_functions):
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	291 yield lineno, funcname, text

Mercurial > genshi > mirror

annotate genshi/filters/i18n.py @ 522:082535e5087c trunk