genshi/mirror: genshi/filters/i18n.py annotate

annotate genshi/filters/i18n.py @ 448:1154f2aadb6c trunk

Add support for HTML5 doctype.

author	cmlenz
date	Fri, 13 Apr 2007 11:54:02 +0000
parents	fd9c4f7a249a
children	94601511cd68

rev	line source
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	1 """Utilities for internationalization and localization of templates."""
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	2
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	3 try:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	4 frozenset
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	5 except NameError:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	6 from sets import ImmutableSet as frozenset
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	7 from gettext import gettext
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	8 from opcode import opmap
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	9 import re
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	10
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	11 from genshi.core import Attrs, START, END, TEXT
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	12 from genshi.template.base import Template, EXPR, SUB
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	13 from genshi.template.markup import EXEC
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	14
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	15 _LOAD_NAME = chr(opmap['LOAD_NAME'])
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	16 _LOAD_CONST = chr(opmap['LOAD_CONST'])
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	17 _CALL_FUNCTION = chr(opmap['CALL_FUNCTION'])
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	18 _BINARY_ADD = chr(opmap['BINARY_ADD'])
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	19
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	20
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	21 class Translator(object):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	22 """Can extract and translate localizable strings from markup streams and
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	23 templates
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	24
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	25 For example, assume the followng template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	26
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	27 >>> from genshi.template import MarkupTemplate
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	28 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	29 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	30 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	31 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	32 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	33 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	34 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	35 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	36 ... </html>''', filename='example.html')
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	37
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	38 For demonstration, we define a dummy ``gettext``-style function with a
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	39 hard-coded translation table, and pass that to the `Translator` initializer:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	40
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	41 >>> def pseudo_gettext(string):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	42 ... return {
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	43 ... 'Example': 'Beispiel',
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	44 ... 'Hello, %(name)s': 'Hallo, %(name)s'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	45 ... }[string]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	46 >>> translator = Translator(pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	47
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	48 Next, the translator needs to be prepended to any already defined filters
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	49 on the template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	50
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	51 >>> tmpl.filters.insert(0, translator)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	52
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	53 When generating the template output, our hard-coded translations should be
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	54 applied as expected:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	55
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	56 >>> print tmpl.generate(username='Hans', _=pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	57 <html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	58 <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	59 <title>Beispiel</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	60 </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	61 <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	62 <h1>Beispiel</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	63 <p>Hallo, Hans</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	64 </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	65 </html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	66 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	67
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	68 IGNORE_TAGS = frozenset(['script', 'style'])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	69 INCLUDE_ATTRS = frozenset(['title', 'alt'])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	70
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	71 def __init__(self, translate=gettext, ignore_tags=IGNORE_TAGS,
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	72 include_attrs=INCLUDE_ATTRS):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	73 """Initialize the translator.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	74
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	75 :param translate: the translation function, for example ``gettext`` or
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	76 ``ugettext``.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	77 :param ignore_tags: a set of tag names that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	78 :param include_attrs: a set of attribute names should be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	79 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	80 self.gettext = translate
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	81 self.ignore_tags = ignore_tags
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	82 self.include_attrs = include_attrs
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	83
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	84 def __call__(self, stream, ctxt=None, search_text=True):
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	85 """Translate any localizable strings in the given stream.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	86
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	87 This function shouldn't be called directly. Instead, an instance of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	88 the `Translator` class should be registered as a filter with the
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	89 `Template` or the `TemplateLoader`, or applied as a regular stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	90 filter. If used as a template filter, it should be inserted in front of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	91 all the default filters.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	92
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	93 :param stream: the markup event stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	94 :param ctxt: the template context (not used)
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	95 :param search_text: whether text nodes should be translated (used
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	96 internally)
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	97 :return: the localized stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	98 """
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	99 skip = 0
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	100
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	101 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	102
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	103 # skip chunks that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	104 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	105 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	106 tag, attrs = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	107 tag = tag.localname
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	108 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	109 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	110 elif kind is END:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	111 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	112 skip -= 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	113 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	114 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	115
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	116 # handle different events that can be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	117 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	118 tag, attrs = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	119 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	120 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	121 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	122 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	123
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	124 new_attrs = list(attrs)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	125 changed = False
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	126 for name, value in attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	127 if name in include_attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	128 if isinstance(value, basestring):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	129 newval = ugettext(value)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	130 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	131 newval = list(self(value, ctxt, search_text=name in self.include_attrs))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	132 if newval != value:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	133 value = new_val
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	134 changed = True
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	135 new_attrs.append((name, value))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	136 if changed:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	137 attrs = new_attrs
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	138
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	139 yield kind, (tag, attrs), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	140
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	141 elif kind is TEXT:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	142 text = data.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	143 if text:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	144 data = data.replace(text, self.gettext(text))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	145 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	146
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	147 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	148 subkind, substream = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	149 new_substream = list(self(substream, ctxt))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	150 yield kind, (subkind, new_substream), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	151
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	152 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	153 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	154
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	155 def extract(self, stream, gettext_functions=('_', 'gettext', 'ngettext')):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	156 """Extract localizable strings from the given template stream.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	157
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	158 For every string found, this function yields a ``(lineno, message)``
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	159 tuple.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	160
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	161 :param stream: the event stream to extract strings from; can be a
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	162 regular stream or a template stream
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	163
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	164 >>> from genshi.template import MarkupTemplate
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	165 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	166 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	167 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	168 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	169 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	170 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	171 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	172 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	173 ... </html>''', filename='example.html')
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	174 >>> for lineno, message in Translator().extract(tmpl.stream):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	175 ... print "Line %d: %r" % (lineno, message)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	176 Line 3: u'Example'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	177 Line 6: u'Example'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	178 Line 7: u'Hello, %(name)s'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	179 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	180 tagname = None
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	181 skip = 0
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	182
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	183 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	184 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	185 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	186 tag, attrs = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	187 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	188 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	189 if kind is END:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	190 tag = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	191 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	192 skip -= 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	193 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	194
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	195 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	196 tag, attrs = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	197 if tag.localname in self.ignore_tags:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	198 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	199 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	200
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	201 for name, value in attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	202 if name in self.include_attrs:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	203 if isinstance(value, basestring):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	204 text = value.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	205 if text:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	206 yield pos[1], text
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	207 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	208 for lineno, text in harvest(value):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	209 yield lineno, text
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	210
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	211 elif kind is TEXT:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	212 text = data.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	213 if text and filter(None, [ch.isalpha() for ch in text]):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	214 yield pos[1], text
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	215
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	216 elif kind is EXPR or kind is EXEC:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	217 consts = dict([(n, chr(i) + '\x00') for i, n in
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	218 enumerate(data.code.co_consts)])
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	219 gettext_locs = [consts[n] for n in gettext_functions
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	220 if n in consts]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	221 ops = [
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	222 _LOAD_CONST, '(', '\|'.join(gettext_locs), ')',
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	223 _CALL_FUNCTION, '.\x00',
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	224 '((?:', _BINARY_ADD, '\|', _LOAD_CONST, '.\x00)+)'
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	225 ]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	226 for _, opcodes in re.findall(''.join(ops), data.code.co_code):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	227 strings = []
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	228 opcodes = iter(opcodes)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	229 for opcode in opcodes:
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	230 if opcode == _BINARY_ADD:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	231 arg = strings.pop()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	232 strings[-1] += arg
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	233 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	234 arg = data.code.co_consts[ord(opcodes.next())]
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	235 opcodes.next() # skip second byte
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	236 if not isinstance(arg, basestring):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	237 break
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	238 strings.append(unicode(arg))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	239 for string in strings:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	240 yield pos[1], string
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	241
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	242 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	243 subkind, substream = data
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	244 for lineno, text in self.harvest(substream):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	245 yield lineno, text

Mercurial > genshi > mirror

annotate genshi/filters/i18n.py @ 448:1154f2aadb6c trunk