genshi/mirror: genshi/filters/i18n.py annotate

annotate genshi/filters/i18n.py @ 775:886934df7fea trunk

Support for parameters in internationalized `i18n:msg` content. See #129.

author	cmlenz
date	Tue, 08 Jul 2008 17:06:51 +0000
parents	df2eda814f2d
children	ddb58d74c8ee

rev	line source
531 9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	1 # -- coding: utf-8 --
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	2 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	3 # Copyright (C) 2007 Edgewall Software
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	4 # All rights reserved.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	5 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	6 # This software is licensed as described in the file COPYING, which
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	7 # you should have received as part of this distribution. The terms
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	8 # are also available at http://genshi.edgewall.org/wiki/License.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	9 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	10 # This software consists of voluntary contributions made by many
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	11 # individuals. For the exact contribution history, see the revision
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	12 # history and logs, available at http://genshi.edgewall.org/log/.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	13
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	14 """Utilities for internationalization and localization of templates.
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	15
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	16 :since: version 0.4
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	17 """
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	18
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	19 from compiler import ast
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	20 from gettext import gettext
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	21 import re
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	22
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	23 from genshi.core import Attrs, Namespace, QName, START, END, TEXT, START_NS, \
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	24 END_NS, XML_NAMESPACE, _ensure
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	25 from genshi.template.base import Template, EXPR, SUB
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	26 from genshi.template.markup import MarkupTemplate, EXEC
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	27
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	28 __all__ = ['Translator', 'extract']
501 5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions. cmlenz parents: 493 diff changeset	29 __docformat__ = 'restructuredtext en'
5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions. cmlenz parents: 493 diff changeset	30
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	31 I18N_NAMESPACE = Namespace('http://genshi.edgewall.org/i18n')
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	32
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	33
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	34 class Translator(object):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	35 """Can extract and translate localizable strings from markup streams and
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	36 templates.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	37
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	38 For example, assume the followng template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	39
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	40 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	41 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	42 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	43 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	44 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	45 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	46 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	47 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	48 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	49 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	50 ... </html>''', filename='example.html')
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	51
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	52 For demonstration, we define a dummy ``gettext``-style function with a
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	53 hard-coded translation table, and pass that to the `Translator` initializer:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	54
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	55 >>> def pseudo_gettext(string):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	56 ... return {
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	57 ... 'Example': 'Beispiel',
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	58 ... 'Hello, %(name)s': 'Hallo, %(name)s'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	59 ... }[string]
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	60 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	61 >>> translator = Translator(pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	62
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	63 Next, the translator needs to be prepended to any already defined filters
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	64 on the template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	65
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	66 >>> tmpl.filters.insert(0, translator)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	67
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	68 When generating the template output, our hard-coded translations should be
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	69 applied as expected:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	70
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	71 >>> print tmpl.generate(username='Hans', _=pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	72 <html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	73 <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	74 <title>Beispiel</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	75 </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	76 <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	77 <h1>Beispiel</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	78 <p>Hallo, Hans</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	79 </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	80 </html>
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	81
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	82 Note that elements defining ``xml:lang`` attributes that do not contain
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	83 variable expressions are ignored by this filter. That can be used to
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	84 exclude specific parts of a template from being extracted and translated.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	85 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	86
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	87 IGNORE_TAGS = frozenset([
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	88 QName('script'), QName('http://www.w3.org/1999/xhtml}script'),
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	89 QName('style'), QName('http://www.w3.org/1999/xhtml}style')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	90 ])
467 23082baddbf9 Add some more localizable HTML attributes to the I18n filter. cmlenz parents: 466 diff changeset	91 INCLUDE_ATTRS = frozenset(['abbr', 'alt', 'label', 'prompt', 'standby',
23082baddbf9 Add some more localizable HTML attributes to the I18n filter. cmlenz parents: 466 diff changeset	92 'summary', 'title'])
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	93
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	94 def __init__(self, translate=gettext, ignore_tags=IGNORE_TAGS,
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	95 include_attrs=INCLUDE_ATTRS, extract_text=True):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	96 """Initialize the translator.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	97
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	98 :param translate: the translation function, for example ``gettext`` or
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	99 ``ugettext``.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	100 :param ignore_tags: a set of tag names that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	101 :param include_attrs: a set of attribute names should be localized
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	102 :param extract_text: whether the content of text nodes should be
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	103 extracted, or only text in explicit ``gettext``
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	104 function calls
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	105 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	106 self.translate = translate
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	107 self.ignore_tags = ignore_tags
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	108 self.include_attrs = include_attrs
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	109 self.extract_text = extract_text
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	110
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	111 def __call__(self, stream, ctxt=None, search_text=True, msgbuf=None):
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	112 """Translate any localizable strings in the given stream.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	113
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	114 This function shouldn't be called directly. Instead, an instance of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	115 the `Translator` class should be registered as a filter with the
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	116 `Template` or the `TemplateLoader`, or applied as a regular stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	117 filter. If used as a template filter, it should be inserted in front of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	118 all the default filters.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	119
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	120 :param stream: the markup event stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	121 :param ctxt: the template context (not used)
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	122 :param search_text: whether text nodes should be translated (used
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	123 internally)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	124 :param msgbuf: a `MessageBuffer` object or `None` (used internally)
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	125 :return: the localized stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	126 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	127 ignore_tags = self.ignore_tags
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	128 include_attrs = self.include_attrs
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	129 translate = self.translate
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	130 if not self.extract_text:
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	131 search_text = False
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	132 skip = 0
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	133 i18n_msg = I18N_NAMESPACE['msg']
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	134 ns_prefixes = []
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	135 xml_lang = XML_NAMESPACE['lang']
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	136
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	137 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	138
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	139 # skip chunks that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	140 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	141 if kind is START:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	142 skip += 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	143 elif kind is END:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	144 skip -= 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	145 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	146 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	147
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	148 # handle different events that can be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	149 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	150 tag, attrs = data
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	151 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	152 isinstance(attrs.get(xml_lang), basestring):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	153 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	154 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	155 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	156
493 3f6582a5a4a5 Fix another bug in the translation filter: translated attributes were getting added instead of replaced. cmlenz parents: 485 diff changeset	157 new_attrs = []
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	158 changed = False
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	159 for name, value in attrs:
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	160 newval = value
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	161 if search_text and isinstance(value, basestring):
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	162 if name in include_attrs:
456 4b6dc4978691 Fix incorrect reference to translation function in the I18N filter. cmlenz parents: 450 diff changeset	163 newval = self.translate(value)
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	164 else:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	165 newval = list(self(_ensure(value), ctxt,
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	166 search_text=False, msgbuf=msgbuf)
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	167 )
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	168 if newval != value:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	169 value = newval
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	170 changed = True
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	171 new_attrs.append((name, value))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	172 if changed:
667 c9a084ffaee6 Fix case where attributes weren't properly wrapped in an `Attrs` instance if one or more of them were translated by the I18n filter, potentially breaking things further down the chain. Closes #162. cmlenz parents: 600 diff changeset	173 attrs = Attrs(new_attrs)
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	174
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	175 if msgbuf:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	176 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	177 continue
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	178 elif i18n_msg in attrs:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	179 params = attrs.get(i18n_msg)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	180 if params and type(params) is list: # event tuple
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	181 params = params[0][1]
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	182 msgbuf = MessageBuffer(params)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	183 attrs -= i18n_msg
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	184
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	185 yield kind, (tag, attrs), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	186
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	187 elif search_text and kind is TEXT:
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	188 if not msgbuf:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	189 text = data.strip()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	190 if text:
762 df2eda814f2d Fix for I18n filter problem with lazy translation functions. Closes #145. cmlenz parents: 750 diff changeset	191 data = data.replace(text, unicode(translate(text)))
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	192 yield kind, data, pos
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	193 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	194 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	195
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	196 elif msgbuf and kind is EXPR:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	197 msgbuf.append(kind, data, pos)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	198
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	199 elif not skip and msgbuf and kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	200 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	201 if not msgbuf.depth:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	202 for event in msgbuf.translate(translate(msgbuf.format())):
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	203 yield event
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	204 msgbuf = None
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	205 yield kind, data, pos
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	206
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	207 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	208 subkind, substream = data
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	209 new_substream = list(self(substream, ctxt, msgbuf=msgbuf))
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	210 yield kind, (subkind, new_substream), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	211
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	212 elif kind is START_NS and data[1] == I18N_NAMESPACE:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	213 ns_prefixes.append(data[0])
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	214
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	215 elif kind is END_NS and data in ns_prefixes:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	216 ns_prefixes.remove(data)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	217
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	218 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	219 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	220
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	221 GETTEXT_FUNCTIONS = ('_', 'gettext', 'ngettext', 'dgettext', 'dngettext',
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	222 'ugettext', 'ungettext')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	223
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	224 def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS,
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	225 search_text=True, msgbuf=None):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	226 """Extract localizable strings from the given template stream.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	227
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	228 For every string found, this function yields a ``(lineno, function,
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	229 message)`` tuple, where:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	230
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	231 * ``lineno`` is the number of the line on which the string was found,
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	232 * ``function`` is the name of the ``gettext`` function used (if the
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	233 string was extracted from embedded Python code), and
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	234 * ``message`` is the string itself (a ``unicode`` object, or a tuple
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	235 of ``unicode`` objects for functions with multiple string arguments).
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	236
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	237 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	238 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	239 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	240 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	241 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	242 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	243 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	244 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	245 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	246 ... <p>${ngettext("You have %d item", "You have %d items", num)}</p>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	247 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	248 ... </html>''', filename='example.html')
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	249 >>>
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	250 >>> for lineno, funcname, message in Translator().extract(tmpl.stream):
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	251 ... print "%d, %r, %r" % (lineno, funcname, message)
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	252 3, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	253 6, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	254 7, '_', u'Hello, %(name)s'
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	255 8, 'ngettext', (u'You have %d item', u'You have %d items', None)
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	256
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	257 :param stream: the event stream to extract strings from; can be a
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	258 regular stream or a template stream
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	259 :param gettext_functions: a sequence of function names that should be
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	260 treated as gettext-style localization
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	261 functions
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	262 :param search_text: whether the content of text nodes should be
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	263 extracted (used internally)
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	264
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	265 :note: Changed in 0.4.1: For a function with multiple string arguments
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	266 (such as ``ngettext``), a single item with a tuple of strings is
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	267 yielded, instead an item for each string argument.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	268 """
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	269 if not self.extract_text:
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	270 search_text = False
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	271 skip = 0
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	272 i18n_msg = I18N_NAMESPACE['msg']
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	273 xml_lang = XML_NAMESPACE['lang']
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	274
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	275 for kind, data, pos in stream:
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	276
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	277 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	278 if kind is START:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	279 skip += 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	280 if kind is END:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	281 skip -= 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	282
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	283 if kind is START and not skip:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	284 tag, attrs = data
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	285
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	286 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	287 isinstance(attrs.get(xml_lang), basestring):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	288 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	289 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	290
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	291 for name, value in attrs:
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	292 if search_text and isinstance(value, basestring):
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	293 if name in self.include_attrs:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	294 text = value.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	295 if text:
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	296 yield pos[1], None, text
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	297 else:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	298 for lineno, funcname, text in self.extract(
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	299 _ensure(value), gettext_functions,
535 35a413f3f1dd The I18n filter no longer extracts or translates literal strings in attribute values that also contain expressions. cmlenz parents: 531 diff changeset	300 search_text=False):
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	301 yield lineno, funcname, text
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	302
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	303 if msgbuf:
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	304 msgbuf.append(kind, data, pos)
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	305 elif i18n_msg in attrs:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	306 params = attrs.get(i18n_msg)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	307 if params and type(params) is list: # event tuple
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	308 params = params[0][1]
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	309 msgbuf = MessageBuffer(params, pos[1])
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	310
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	311 elif not skip and search_text and kind is TEXT:
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	312 if not msgbuf:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	313 text = data.strip()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	314 if text and filter(None, [ch.isalpha() for ch in text]):
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	315 yield pos[1], None, text
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	316 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	317 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	318
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	319 elif not skip and msgbuf and kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	320 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	321 if not msgbuf.depth:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	322 yield msgbuf.lineno, None, msgbuf.format()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	323 msgbuf = None
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	324
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	325 elif kind is EXPR or kind is EXEC:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	326 if msgbuf:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	327 msgbuf.append(kind, data, pos)
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	328 for funcname, strings in extract_from_code(data,
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	329 gettext_functions):
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	330 yield pos[1], funcname, strings
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	331
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	332 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	333 subkind, substream = data
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	334 messages = self.extract(substream, gettext_functions,
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	335 search_text=search_text and not skip,
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	336 msgbuf=msgbuf)
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	337 for lineno, funcname, text in messages:
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	338 yield lineno, funcname, text
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	339
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	340
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	341 class MessageBuffer(object):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	342 """Helper class for managing internationalized mixed content.
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	343
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	344 :since: version 0.5
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	345 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	346
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	347 def __init__(self, params=u'', lineno=-1):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	348 """Initialize the message buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	349
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	350 :param lineno: the line number on which the first stream event
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	351 belonging to the message was found
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	352 """
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	353 self.params = [name.strip() for name in params.split(',')]
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	354 self.lineno = lineno
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	355 self.string = []
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	356 self.events = {}
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	357 self.values = {}
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	358 self.depth = 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	359 self.order = 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	360 self.stack = [0]
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	361
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	362 def append(self, kind, data, pos):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	363 """Append a stream event to the buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	364
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	365 :param kind: the stream event kind
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	366 :param data: the event data
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	367 :param pos: the position of the event in the source
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	368 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	369 if kind is TEXT:
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	370 self.string.append(data)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	371 self.events.setdefault(self.stack[-1], []).append(None)
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	372 elif kind is EXPR:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	373 param = self.params.pop(0)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	374 self.string.append('%%(%s)s' % param)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	375 self.events.setdefault(self.stack[-1], []).append(None)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	376 self.values[param] = (kind, data, pos)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	377 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	378 if kind is START:
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	379 self.string.append(u'[%d:' % self.order)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	380 self.events.setdefault(self.order, []).append((kind, data, pos))
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	381 self.stack.append(self.order)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	382 self.depth += 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	383 self.order += 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	384 elif kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	385 self.depth -= 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	386 if self.depth:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	387 self.events[self.stack[-1]].append((kind, data, pos))
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	388 self.string.append(u']')
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	389 self.stack.pop()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	390
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	391 def format(self):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	392 """Return a message identifier representing the content in the
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	393 buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	394 """
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	395 return u''.join(self.string).strip()
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	396
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	397 def translate(self, string, regex=re.compile(r'%$(\w+)$s')):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	398 """Interpolate the given message translation with the events in the
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	399 buffer and return the translated stream.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	400
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	401 :param string: the translated message string
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	402 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	403 parts = parse_msg(string)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	404 for order, string in parts:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	405 events = self.events[order]
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	406 while events:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	407 event = events.pop(0)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	408 if event:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	409 yield event
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	410 else:
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	411 if not string:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	412 break
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	413 for idx, part in enumerate(regex.split(string)):
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	414 if idx % 2:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	415 yield self.values[part]
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	416 elif part:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	417 yield TEXT, part, (None, -1, -1)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	418 if not self.events[order] or not self.events[order][0]:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	419 break
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	420
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	421
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	422 def parse_msg(string, regex=re.compile(r'(?:\[(\d+)\:)\|\]')):
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	423 """Parse a translated message using Genshi mixed content message
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	424 formatting.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	425
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	426 >>> parse_msg("See [1:Help].")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	427 [(0, 'See '), (1, 'Help'), (0, '.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	428
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	429 >>> parse_msg("See [1:our [2:Help] page] for details.")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	430 [(0, 'See '), (1, 'our '), (2, 'Help'), (1, ' page'), (0, ' for details.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	431
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	432 >>> parse_msg("[2:Details] finden Sie in [1:Hilfe].")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	433 [(2, 'Details'), (0, ' finden Sie in '), (1, 'Hilfe'), (0, '.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	434
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	435 >>> parse_msg("[1:] Bilder pro Seite anzeigen.")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	436 [(1, ''), (0, ' Bilder pro Seite anzeigen.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	437
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	438 :param string: the translated message string
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	439 :return: a list of ``(order, string)`` tuples
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	440 :rtype: `list`
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	441 """
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	442 parts = []
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	443 stack = [0]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	444 while True:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	445 mo = regex.search(string)
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	446 if not mo:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	447 break
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	448
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	449 if mo.start() or stack[-1]:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	450 parts.append((stack[-1], string[:mo.start()]))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	451 string = string[mo.end():]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	452
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	453 orderno = mo.group(1)
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	454 if orderno is not None:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	455 stack.append(int(orderno))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	456 else:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	457 stack.pop()
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	458 if not stack:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	459 break
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	460
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	461 if string:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	462 parts.append((stack[-1], string))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	463
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	464 return parts
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	465
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	466 def extract_from_code(code, gettext_functions):
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	467 """Extract strings from Python bytecode.
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	468
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	469 >>> from genshi.template.eval import Expression
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	470
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	471 >>> expr = Expression('_("Hello")')
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	472 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS))
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	473 [('_', u'Hello')]
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	474
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	475 >>> expr = Expression('ngettext("You have %(num)s item", '
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	476 ... '"You have %(num)s items", num)')
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	477 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS))
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	478 [('ngettext', (u'You have %(num)s item', u'You have %(num)s items', None))]
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	479
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	480 :param code: the `Code` object
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	481 :type code: `genshi.template.eval.Code`
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	482 :param gettext_functions: a sequence of function names
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	483 :since: version 0.5
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	484 """
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	485 def _walk(node):
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	486 if isinstance(node, ast.CallFunc) and isinstance(node.node, ast.Name) \
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	487 and node.node.name in gettext_functions:
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	488 strings = []
600 23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	489 def _add(arg):
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	490 if isinstance(arg, ast.Const) \
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	491 and isinstance(arg.value, basestring):
600 23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	492 strings.append(unicode(arg.value, 'utf-8'))
23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	493 elif arg and not isinstance(arg, ast.Keyword):
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	494 strings.append(None)
600 23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	495 [_add(arg) for arg in node.args]
23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	496 _add(node.star_args)
23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	497 _add(node.dstar_args)
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	498 if len(strings) == 1:
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	499 strings = strings[0]
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	500 else:
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	501 strings = tuple(strings)
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	502 yield node.node.name, strings
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	503 else:
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	504 for child in node.getChildNodes():
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	505 for funcname, strings in _walk(child):
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	506 yield funcname, strings
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	507 return _walk(code.ast)
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	508
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	509 def extract(fileobj, keywords, comment_tags, options):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	510 """Babel extraction method for Genshi templates.
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	511
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	512 :param fileobj: the file-like object the messages should be extracted from
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	513 :param keywords: a list of keywords (i.e. function names) that should be
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	514 recognized as translation functions
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	515 :param comment_tags: a list of translator tags to search for and include
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	516 in the results
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	517 :param options: a dictionary of additional options (optional)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	518 :return: an iterator over ``(lineno, funcname, message, comments)`` tuples
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	519 :rtype: ``iterator``
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	520 """
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	521 template_class = options.get('template_class', MarkupTemplate)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	522 if isinstance(template_class, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	523 module, clsname = template_class.split(':', 1)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	524 template_class = getattr(__import__(module, {}, {}, [clsname]), clsname)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	525 encoding = options.get('encoding', None)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	526
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	527 extract_text = options.get('extract_text', True)
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	528 if isinstance(extract_text, basestring):
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	529 extract_text = extract_text.lower() in ('1', 'on', 'yes', 'true')
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	530
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	531 ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	532 if isinstance(ignore_tags, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	533 ignore_tags = ignore_tags.split()
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	534 ignore_tags = [QName(tag) for tag in ignore_tags]
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	535
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	536 include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	537 if isinstance(include_attrs, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	538 include_attrs = include_attrs.split()
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	539 include_attrs = [QName(attr) for attr in include_attrs]
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	540
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	541 tmpl = template_class(fileobj, filename=getattr(fileobj, 'name', None),
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	542 encoding=encoding)
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	543 translator = Translator(None, ignore_tags, include_attrs, extract_text)
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	544 for lineno, func, message in translator.extract(tmpl.stream,
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	545 gettext_functions=keywords):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	546 yield lineno, func, message, []

Mercurial > genshi > mirror

annotate genshi/filters/i18n.py @ 775:886934df7fea trunk