genshi/mirror: genshi/filters/i18n.py annotate

annotate genshi/filters/i18n.py @ 787:d7366797440f trunk

Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129.

author	cmlenz
date	Thu, 07 Aug 2008 23:15:11 +0000
parents	36fb0a57fe74
children	31432f30a6fb

rev	line source
531 9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	1 # -- coding: utf-8 --
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	2 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	3 # Copyright (C) 2007 Edgewall Software
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	4 # All rights reserved.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	5 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	6 # This software is licensed as described in the file COPYING, which
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	7 # you should have received as part of this distribution. The terms
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	8 # are also available at http://genshi.edgewall.org/wiki/License.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	9 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	10 # This software consists of voluntary contributions made by many
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	11 # individuals. For the exact contribution history, see the revision
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	12 # history and logs, available at http://genshi.edgewall.org/log/.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	13
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	14 """Utilities for internationalization and localization of templates.
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	15
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	16 :since: version 0.4
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	17 """
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	18
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	19 from compiler import ast
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	20 from gettext import gettext
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	21 import re
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	22
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	23 from genshi.core import Attrs, Namespace, QName, START, END, TEXT, START_NS, \
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	24 END_NS, XML_NAMESPACE, _ensure
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	25 from genshi.template.base import Template, EXPR, SUB
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	26 from genshi.template.markup import MarkupTemplate, EXEC
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	27
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	28 __all__ = ['Translator', 'extract']
501 5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions. cmlenz parents: 493 diff changeset	29 __docformat__ = 'restructuredtext en'
5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions. cmlenz parents: 493 diff changeset	30
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	31 I18N_NAMESPACE = Namespace('http://genshi.edgewall.org/i18n')
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	32
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	33
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	34 class Translator(object):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	35 """Can extract and translate localizable strings from markup streams and
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	36 templates.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	37
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	38 For example, assume the followng template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	39
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	40 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	41 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	42 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	43 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	44 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	45 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	46 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	47 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	48 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	49 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	50 ... </html>''', filename='example.html')
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	51
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	52 For demonstration, we define a dummy ``gettext``-style function with a
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	53 hard-coded translation table, and pass that to the `Translator` initializer:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	54
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	55 >>> def pseudo_gettext(string):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	56 ... return {
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	57 ... 'Example': 'Beispiel',
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	58 ... 'Hello, %(name)s': 'Hallo, %(name)s'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	59 ... }[string]
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	60 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	61 >>> translator = Translator(pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	62
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	63 Next, the translator needs to be prepended to any already defined filters
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	64 on the template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	65
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	66 >>> tmpl.filters.insert(0, translator)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	67
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	68 When generating the template output, our hard-coded translations should be
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	69 applied as expected:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	70
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	71 >>> print tmpl.generate(username='Hans', _=pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	72 <html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	73 <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	74 <title>Beispiel</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	75 </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	76 <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	77 <h1>Beispiel</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	78 <p>Hallo, Hans</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	79 </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	80 </html>
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	81
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	82 Note that elements defining ``xml:lang`` attributes that do not contain
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	83 variable expressions are ignored by this filter. That can be used to
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	84 exclude specific parts of a template from being extracted and translated.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	85 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	86
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	87 IGNORE_TAGS = frozenset([
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	88 QName('script'), QName('http://www.w3.org/1999/xhtml}script'),
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	89 QName('style'), QName('http://www.w3.org/1999/xhtml}style')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	90 ])
467 23082baddbf9 Add some more localizable HTML attributes to the I18n filter. cmlenz parents: 466 diff changeset	91 INCLUDE_ATTRS = frozenset(['abbr', 'alt', 'label', 'prompt', 'standby',
23082baddbf9 Add some more localizable HTML attributes to the I18n filter. cmlenz parents: 466 diff changeset	92 'summary', 'title'])
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	93
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	94 def __init__(self, translate=gettext, ignore_tags=IGNORE_TAGS,
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	95 include_attrs=INCLUDE_ATTRS, extract_text=True):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	96 """Initialize the translator.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	97
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	98 :param translate: the translation function, for example ``gettext`` or
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	99 ``ugettext``.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	100 :param ignore_tags: a set of tag names that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	101 :param include_attrs: a set of attribute names should be localized
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	102 :param extract_text: whether the content of text nodes should be
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	103 extracted, or only text in explicit ``gettext``
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	104 function calls
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	105 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	106 self.translate = translate
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	107 self.ignore_tags = ignore_tags
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	108 self.include_attrs = include_attrs
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	109 self.extract_text = extract_text
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	110
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	111 def __call__(self, stream, ctxt=None, search_text=True, msgbuf=None):
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	112 """Translate any localizable strings in the given stream.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	113
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	114 This function shouldn't be called directly. Instead, an instance of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	115 the `Translator` class should be registered as a filter with the
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	116 `Template` or the `TemplateLoader`, or applied as a regular stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	117 filter. If used as a template filter, it should be inserted in front of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	118 all the default filters.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	119
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	120 :param stream: the markup event stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	121 :param ctxt: the template context (not used)
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	122 :param search_text: whether text nodes should be translated (used
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	123 internally)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	124 :param msgbuf: a `MessageBuffer` object or `None` (used internally)
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	125 :return: the localized stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	126 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	127 ignore_tags = self.ignore_tags
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	128 include_attrs = self.include_attrs
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	129 translate = self.translate
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	130 if not self.extract_text:
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	131 search_text = False
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	132
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	133 ns_prefixes = []
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	134 skip = 0
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	135 i18n_comment = I18N_NAMESPACE['comment']
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	136 i18n_msg = I18N_NAMESPACE['msg']
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	137 xml_lang = XML_NAMESPACE['lang']
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	138
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	139 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	140
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	141 # skip chunks that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	142 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	143 if kind is START:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	144 skip += 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	145 elif kind is END:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	146 skip -= 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	147 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	148 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	149
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	150 # handle different events that can be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	151 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	152 tag, attrs = data
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	153 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	154 isinstance(attrs.get(xml_lang), basestring):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	155 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	156 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	157 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	158
493 3f6582a5a4a5 Fix another bug in the translation filter: translated attributes were getting added instead of replaced. cmlenz parents: 485 diff changeset	159 new_attrs = []
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	160 changed = False
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	161 for name, value in attrs:
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	162 newval = value
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	163 if search_text and isinstance(value, basestring):
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	164 if name in include_attrs:
456 4b6dc4978691 Fix incorrect reference to translation function in the I18N filter. cmlenz parents: 450 diff changeset	165 newval = self.translate(value)
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	166 else:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	167 newval = list(self(_ensure(value), ctxt,
785 36fb0a57fe74 Fix for #250: ignore expressions in attribute values when inside `i18n:msg` elements. cmlenz parents: 776 diff changeset	168 search_text=False)
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	169 )
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	170 if newval != value:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	171 value = newval
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	172 changed = True
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	173 new_attrs.append((name, value))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	174 if changed:
667 c9a084ffaee6 Fix case where attributes weren't properly wrapped in an `Attrs` instance if one or more of them were translated by the I18n filter, potentially breaking things further down the chain. Closes #162. cmlenz parents: 600 diff changeset	175 attrs = Attrs(new_attrs)
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	176
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	177 if msgbuf:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	178 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	179 continue
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	180 elif i18n_msg in attrs:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	181 params = attrs.get(i18n_msg)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	182 if params and type(params) is list: # event tuple
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	183 params = params[0][1]
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	184 msgbuf = MessageBuffer(params)
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	185 attrs -= (i18n_comment, i18n_msg)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	186
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	187 yield kind, (tag, attrs), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	188
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	189 elif search_text and kind is TEXT:
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	190 if not msgbuf:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	191 text = data.strip()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	192 if text:
762 df2eda814f2d Fix for I18n filter problem with lazy translation functions. Closes #145. cmlenz parents: 750 diff changeset	193 data = data.replace(text, unicode(translate(text)))
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	194 yield kind, data, pos
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	195 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	196 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	197
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	198 elif msgbuf and kind is EXPR:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	199 msgbuf.append(kind, data, pos)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	200
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	201 elif not skip and msgbuf and kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	202 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	203 if not msgbuf.depth:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	204 for event in msgbuf.translate(translate(msgbuf.format())):
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	205 yield event
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	206 msgbuf = None
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	207 yield kind, data, pos
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	208
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	209 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	210 subkind, substream = data
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	211 new_substream = list(self(substream, ctxt, msgbuf=msgbuf))
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	212 yield kind, (subkind, new_substream), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	213
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	214 elif kind is START_NS and data[1] == I18N_NAMESPACE:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	215 ns_prefixes.append(data[0])
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	216
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	217 elif kind is END_NS and data in ns_prefixes:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	218 ns_prefixes.remove(data)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	219
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	220 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	221 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	222
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	223 GETTEXT_FUNCTIONS = ('_', 'gettext', 'ngettext', 'dgettext', 'dngettext',
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	224 'ugettext', 'ungettext')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	225
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	226 def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS,
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	227 search_text=True, msgbuf=None):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	228 """Extract localizable strings from the given template stream.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	229
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	230 For every string found, this function yields a ``(lineno, function,
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	231 message, comments)`` tuple, where:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	232
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	233 * ``lineno`` is the number of the line on which the string was found,
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	234 * ``function`` is the name of the ``gettext`` function used (if the
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	235 string was extracted from embedded Python code), and
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	236 * ``message`` is the string itself (a ``unicode`` object, or a tuple
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	237 of ``unicode`` objects for functions with multiple string
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	238 arguments).
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	239 * ``comments`` is a list of comments related to the message, extracted
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	240 from ``i18n:comment`` attributes found in the markup
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	241
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	242 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	243 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	244 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	245 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	246 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	247 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	248 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	249 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	250 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	251 ... <p>${ngettext("You have %d item", "You have %d items", num)}</p>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	252 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	253 ... </html>''', filename='example.html')
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	254 >>>
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	255 >>> for line, func, msg, comments in Translator().extract(tmpl.stream):
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	256 ... print "%d, %r, %r" % (line, func, msg)
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	257 3, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	258 6, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	259 7, '_', u'Hello, %(name)s'
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	260 8, 'ngettext', (u'You have %d item', u'You have %d items', None)
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	261
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	262 :param stream: the event stream to extract strings from; can be a
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	263 regular stream or a template stream
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	264 :param gettext_functions: a sequence of function names that should be
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	265 treated as gettext-style localization
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	266 functions
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	267 :param search_text: whether the content of text nodes should be
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	268 extracted (used internally)
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	269
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	270 :note: Changed in 0.4.1: For a function with multiple string arguments
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	271 (such as ``ngettext``), a single item with a tuple of strings is
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	272 yielded, instead an item for each string argument.
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	273 :note: Changed in 0.6: The returned tuples now include a 4th element,
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	274 which is a list of comments for the translator
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	275 """
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	276 if not self.extract_text:
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	277 search_text = False
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	278 skip = 0
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	279 i18n_comment = I18N_NAMESPACE['comment']
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	280 i18n_msg = I18N_NAMESPACE['msg']
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	281 xml_lang = XML_NAMESPACE['lang']
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	282
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	283 for kind, data, pos in stream:
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	284
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	285 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	286 if kind is START:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	287 skip += 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	288 if kind is END:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	289 skip -= 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	290
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	291 if kind is START and not skip:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	292 tag, attrs = data
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	293
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	294 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	295 isinstance(attrs.get(xml_lang), basestring):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	296 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	297 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	298
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	299 for name, value in attrs:
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	300 if search_text and isinstance(value, basestring):
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	301 if name in self.include_attrs:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	302 text = value.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	303 if text:
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	304 yield pos[1], None, text, []
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	305 else:
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	306 for lineno, funcname, text, comments in self.extract(
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	307 _ensure(value), gettext_functions,
535 35a413f3f1dd The I18n filter no longer extracts or translates literal strings in attribute values that also contain expressions. cmlenz parents: 531 diff changeset	308 search_text=False):
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	309 yield lineno, funcname, text, comments
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	310
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	311 if msgbuf:
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	312 msgbuf.append(kind, data, pos)
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	313 else:
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	314 msg_params = attrs.get(i18n_msg)
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	315 if msg_params is not None:
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	316 if type(msg_params) is list: # event tuple
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	317 msg_params = msg_params[0][1]
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	318 msgbuf = MessageBuffer(
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	319 msg_params, attrs.get(i18n_comment), pos[1]
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	320 )
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	321
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	322 elif not skip and search_text and kind is TEXT:
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	323 if not msgbuf:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	324 text = data.strip()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	325 if text and filter(None, [ch.isalpha() for ch in text]):
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	326 yield pos[1], None, text, []
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	327 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	328 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	329
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	330 elif not skip and msgbuf and kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	331 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	332 if not msgbuf.depth:
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	333 yield msgbuf.lineno, None, msgbuf.format(), \
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	334 filter(None, [msgbuf.comment])
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	335 msgbuf = None
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	336
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	337 elif kind is EXPR or kind is EXEC:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	338 if msgbuf:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	339 msgbuf.append(kind, data, pos)
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	340 for funcname, strings in extract_from_code(data,
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	341 gettext_functions):
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	342 yield pos[1], funcname, strings, []
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	343
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	344 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	345 subkind, substream = data
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	346 messages = self.extract(substream, gettext_functions,
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	347 search_text=search_text and not skip,
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	348 msgbuf=msgbuf)
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	349 for lineno, funcname, text, comments in messages:
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	350 yield lineno, funcname, text, comments
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	351
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	352
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	353 class MessageBuffer(object):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	354 """Helper class for managing internationalized mixed content.
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	355
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	356 :since: version 0.5
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	357 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	358
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	359 def __init__(self, params=u'', comment=None, lineno=-1):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	360 """Initialize the message buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	361
776 ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	362 :param params: comma-separated list of parameter names
ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	363 :type params: `basestring`
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	364 :param lineno: the line number on which the first stream event
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	365 belonging to the message was found
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	366 """
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	367 self.params = [name.strip() for name in params.split(',')]
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	368 self.comment = comment
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	369 self.lineno = lineno
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	370 self.string = []
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	371 self.events = {}
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	372 self.values = {}
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	373 self.depth = 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	374 self.order = 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	375 self.stack = [0]
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	376
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	377 def append(self, kind, data, pos):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	378 """Append a stream event to the buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	379
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	380 :param kind: the stream event kind
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	381 :param data: the event data
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	382 :param pos: the position of the event in the source
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	383 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	384 if kind is TEXT:
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	385 self.string.append(data)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	386 self.events.setdefault(self.stack[-1], []).append(None)
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	387 elif kind is EXPR:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	388 param = self.params.pop(0)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	389 self.string.append('%%(%s)s' % param)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	390 self.events.setdefault(self.stack[-1], []).append(None)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	391 self.values[param] = (kind, data, pos)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	392 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	393 if kind is START:
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	394 self.string.append(u'[%d:' % self.order)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	395 self.events.setdefault(self.order, []).append((kind, data, pos))
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	396 self.stack.append(self.order)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	397 self.depth += 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	398 self.order += 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	399 elif kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	400 self.depth -= 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	401 if self.depth:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	402 self.events[self.stack[-1]].append((kind, data, pos))
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	403 self.string.append(u']')
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	404 self.stack.pop()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	405
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	406 def format(self):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	407 """Return a message identifier representing the content in the
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	408 buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	409 """
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	410 return u''.join(self.string).strip()
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	411
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	412 def translate(self, string, regex=re.compile(r'%$(\w+)$s')):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	413 """Interpolate the given message translation with the events in the
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	414 buffer and return the translated stream.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	415
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	416 :param string: the translated message string
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	417 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	418 parts = parse_msg(string)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	419 for order, string in parts:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	420 events = self.events[order]
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	421 while events:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	422 event = events.pop(0)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	423 if event:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	424 yield event
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	425 else:
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	426 if not string:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	427 break
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	428 for idx, part in enumerate(regex.split(string)):
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	429 if idx % 2:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	430 yield self.values[part]
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	431 elif part:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	432 yield TEXT, part, (None, -1, -1)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	433 if not self.events[order] or not self.events[order][0]:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	434 break
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	435
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	436
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	437 def parse_msg(string, regex=re.compile(r'(?:\[(\d+)\:)\|\]')):
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	438 """Parse a translated message using Genshi mixed content message
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	439 formatting.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	440
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	441 >>> parse_msg("See [1:Help].")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	442 [(0, 'See '), (1, 'Help'), (0, '.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	443
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	444 >>> parse_msg("See [1:our [2:Help] page] for details.")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	445 [(0, 'See '), (1, 'our '), (2, 'Help'), (1, ' page'), (0, ' for details.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	446
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	447 >>> parse_msg("[2:Details] finden Sie in [1:Hilfe].")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	448 [(2, 'Details'), (0, ' finden Sie in '), (1, 'Hilfe'), (0, '.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	449
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	450 >>> parse_msg("[1:] Bilder pro Seite anzeigen.")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	451 [(1, ''), (0, ' Bilder pro Seite anzeigen.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	452
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	453 :param string: the translated message string
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	454 :return: a list of ``(order, string)`` tuples
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	455 :rtype: `list`
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	456 """
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	457 parts = []
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	458 stack = [0]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	459 while True:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	460 mo = regex.search(string)
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	461 if not mo:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	462 break
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	463
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	464 if mo.start() or stack[-1]:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	465 parts.append((stack[-1], string[:mo.start()]))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	466 string = string[mo.end():]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	467
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	468 orderno = mo.group(1)
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	469 if orderno is not None:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	470 stack.append(int(orderno))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	471 else:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	472 stack.pop()
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	473 if not stack:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	474 break
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	475
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	476 if string:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	477 parts.append((stack[-1], string))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	478
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	479 return parts
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	480
776 ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	481
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	482 def extract_from_code(code, gettext_functions):
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	483 """Extract strings from Python bytecode.
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	484
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	485 >>> from genshi.template.eval import Expression
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	486
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	487 >>> expr = Expression('_("Hello")')
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	488 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS))
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	489 [('_', u'Hello')]
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	490
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	491 >>> expr = Expression('ngettext("You have %(num)s item", '
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	492 ... '"You have %(num)s items", num)')
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	493 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS))
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	494 [('ngettext', (u'You have %(num)s item', u'You have %(num)s items', None))]
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	495
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	496 :param code: the `Code` object
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	497 :type code: `genshi.template.eval.Code`
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	498 :param gettext_functions: a sequence of function names
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	499 :since: version 0.5
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	500 """
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	501 def _walk(node):
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	502 if isinstance(node, ast.CallFunc) and isinstance(node.node, ast.Name) \
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	503 and node.node.name in gettext_functions:
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	504 strings = []
600 23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	505 def _add(arg):
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	506 if isinstance(arg, ast.Const) \
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	507 and isinstance(arg.value, basestring):
600 23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	508 strings.append(unicode(arg.value, 'utf-8'))
23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	509 elif arg and not isinstance(arg, ast.Keyword):
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	510 strings.append(None)
600 23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	511 [_add(arg) for arg in node.args]
23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	512 _add(node.star_args)
23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	513 _add(node.dstar_args)
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	514 if len(strings) == 1:
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	515 strings = strings[0]
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	516 else:
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	517 strings = tuple(strings)
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	518 yield node.node.name, strings
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	519 else:
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	520 for child in node.getChildNodes():
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	521 for funcname, strings in _walk(child):
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	522 yield funcname, strings
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	523 return _walk(code.ast)
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	524
776 ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	525
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	526 def extract(fileobj, keywords, comment_tags, options):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	527 """Babel extraction method for Genshi templates.
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	528
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	529 :param fileobj: the file-like object the messages should be extracted from
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	530 :param keywords: a list of keywords (i.e. function names) that should be
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	531 recognized as translation functions
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	532 :param comment_tags: a list of translator tags to search for and include
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	533 in the results
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	534 :param options: a dictionary of additional options (optional)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	535 :return: an iterator over ``(lineno, funcname, message, comments)`` tuples
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	536 :rtype: ``iterator``
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	537 """
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	538 template_class = options.get('template_class', MarkupTemplate)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	539 if isinstance(template_class, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	540 module, clsname = template_class.split(':', 1)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	541 template_class = getattr(__import__(module, {}, {}, [clsname]), clsname)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	542 encoding = options.get('encoding', None)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	543
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	544 extract_text = options.get('extract_text', True)
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	545 if isinstance(extract_text, basestring):
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	546 extract_text = extract_text.lower() in ('1', 'on', 'yes', 'true')
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	547
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	548 ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	549 if isinstance(ignore_tags, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	550 ignore_tags = ignore_tags.split()
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	551 ignore_tags = [QName(tag) for tag in ignore_tags]
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	552
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	553 include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	554 if isinstance(include_attrs, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	555 include_attrs = include_attrs.split()
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	556 include_attrs = [QName(attr) for attr in include_attrs]
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	557
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	558 tmpl = template_class(fileobj, filename=getattr(fileobj, 'name', None),
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	559 encoding=encoding)
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	560 translator = Translator(None, ignore_tags, include_attrs, extract_text)
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	561 for message in translator.extract(tmpl.stream, gettext_functions=keywords):
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	562 yield message

Mercurial > genshi > mirror

annotate genshi/filters/i18n.py @ 787:d7366797440f trunk