genshi/mirror: genshi/filters/i18n.py annotate

annotate genshi/filters/i18n.py @ 788:31432f30a6fb trunk

Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface.

author	cmlenz
date	Sun, 10 Aug 2008 20:32:13 +0000
parents	d7366797440f
children	da90cee22560

rev	line source
531 9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	1 # -- coding: utf-8 --
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	2 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	3 # Copyright (C) 2007 Edgewall Software
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	4 # All rights reserved.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	5 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	6 # This software is licensed as described in the file COPYING, which
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	7 # you should have received as part of this distribution. The terms
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	8 # are also available at http://genshi.edgewall.org/wiki/License.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	9 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	10 # This software consists of voluntary contributions made by many
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	11 # individuals. For the exact contribution history, see the revision
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	12 # history and logs, available at http://genshi.edgewall.org/log/.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	13
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	14 """Utilities for internationalization and localization of templates.
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	15
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	16 :since: version 0.4
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	17 """
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	18
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	19 from compiler import ast
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	20 from gettext import NullTranslations
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	21 import re
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	22 from types import FunctionType
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	23
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	24 from genshi.core import Attrs, Namespace, QName, START, END, TEXT, START_NS, \
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	25 END_NS, XML_NAMESPACE, _ensure
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	26 from genshi.template.base import Template, EXPR, SUB
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	27 from genshi.template.markup import MarkupTemplate, EXEC
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	28
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	29 __all__ = ['Translator', 'extract']
501 5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions. cmlenz parents: 493 diff changeset	30 __docformat__ = 'restructuredtext en'
5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions. cmlenz parents: 493 diff changeset	31
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	32 I18N_NAMESPACE = Namespace('http://genshi.edgewall.org/i18n')
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	33
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	34
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	35 class Translator(object):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	36 """Can extract and translate localizable strings from markup streams and
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	37 templates.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	38
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	39 For example, assume the followng template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	40
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	41 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	42 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	43 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	44 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	45 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	46 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	47 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	48 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	49 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	50 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	51 ... </html>''', filename='example.html')
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	52
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	53 For demonstration, we define a dummy ``gettext``-style function with a
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	54 hard-coded translation table, and pass that to the `Translator` initializer:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	55
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	56 >>> def pseudo_gettext(string):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	57 ... return {
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	58 ... 'Example': 'Beispiel',
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	59 ... 'Hello, %(name)s': 'Hallo, %(name)s'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	60 ... }[string]
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	61 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	62 >>> translator = Translator(pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	63
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	64 Next, the translator needs to be prepended to any already defined filters
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	65 on the template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	66
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	67 >>> tmpl.filters.insert(0, translator)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	68
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	69 When generating the template output, our hard-coded translations should be
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	70 applied as expected:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	71
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	72 >>> print tmpl.generate(username='Hans', _=pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	73 <html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	74 <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	75 <title>Beispiel</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	76 </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	77 <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	78 <h1>Beispiel</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	79 <p>Hallo, Hans</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	80 </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	81 </html>
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	82
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	83 Note that elements defining ``xml:lang`` attributes that do not contain
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	84 variable expressions are ignored by this filter. That can be used to
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	85 exclude specific parts of a template from being extracted and translated.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	86 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	87
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	88 IGNORE_TAGS = frozenset([
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	89 QName('script'), QName('http://www.w3.org/1999/xhtml}script'),
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	90 QName('style'), QName('http://www.w3.org/1999/xhtml}style')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	91 ])
467 23082baddbf9 Add some more localizable HTML attributes to the I18n filter. cmlenz parents: 466 diff changeset	92 INCLUDE_ATTRS = frozenset(['abbr', 'alt', 'label', 'prompt', 'standby',
23082baddbf9 Add some more localizable HTML attributes to the I18n filter. cmlenz parents: 466 diff changeset	93 'summary', 'title'])
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	94
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	95 def __init__(self, translate=NullTranslations(), ignore_tags=IGNORE_TAGS,
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	96 include_attrs=INCLUDE_ATTRS, extract_text=True):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	97 """Initialize the translator.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	98
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	99 :param translate: the translation function, for example ``gettext`` or
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	100 ``ugettext``.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	101 :param ignore_tags: a set of tag names that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	102 :param include_attrs: a set of attribute names should be localized
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	103 :param extract_text: whether the content of text nodes should be
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	104 extracted, or only text in explicit ``gettext``
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	105 function calls
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	106
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	107 :note: Changed in 0.6: the `translate` parameter can now be either
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	108 a ``gettext``-style function, or an object compatible with the
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	109 ``NullTransalations`` or ``GNUTranslations`` interface
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	110 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	111 self.translate = translate
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	112 self.ignore_tags = ignore_tags
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	113 self.include_attrs = include_attrs
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	114 self.extract_text = extract_text
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	115
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	116 def __call__(self, stream, ctxt=None, search_text=True, msgbuf=None):
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	117 """Translate any localizable strings in the given stream.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	118
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	119 This function shouldn't be called directly. Instead, an instance of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	120 the `Translator` class should be registered as a filter with the
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	121 `Template` or the `TemplateLoader`, or applied as a regular stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	122 filter. If used as a template filter, it should be inserted in front of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	123 all the default filters.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	124
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	125 :param stream: the markup event stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	126 :param ctxt: the template context (not used)
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	127 :param search_text: whether text nodes should be translated (used
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	128 internally)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	129 :param msgbuf: a `MessageBuffer` object or `None` (used internally)
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	130 :return: the localized stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	131 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	132 ignore_tags = self.ignore_tags
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	133 include_attrs = self.include_attrs
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	134 if type(self.translate) is FunctionType:
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	135 gettext = self.translate
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	136 else:
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	137 gettext = self.translate.ugettext
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	138 if not self.extract_text:
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	139 search_text = False
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	140
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	141 ns_prefixes = []
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	142 skip = 0
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	143 i18n_comment = I18N_NAMESPACE['comment']
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	144 i18n_msg = I18N_NAMESPACE['msg']
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	145 xml_lang = XML_NAMESPACE['lang']
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	146
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	147 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	148
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	149 # skip chunks that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	150 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	151 if kind is START:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	152 skip += 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	153 elif kind is END:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	154 skip -= 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	155 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	156 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	157
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	158 # handle different events that can be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	159 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	160 tag, attrs = data
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	161 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	162 isinstance(attrs.get(xml_lang), basestring):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	163 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	164 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	165 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	166
493 3f6582a5a4a5 Fix another bug in the translation filter: translated attributes were getting added instead of replaced. cmlenz parents: 485 diff changeset	167 new_attrs = []
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	168 changed = False
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	169 for name, value in attrs:
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	170 newval = value
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	171 if search_text and isinstance(value, basestring):
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	172 if name in include_attrs:
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	173 newval = gettext(value)
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	174 else:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	175 newval = list(self(_ensure(value), ctxt,
785 36fb0a57fe74 Fix for #250: ignore expressions in attribute values when inside `i18n:msg` elements. cmlenz parents: 776 diff changeset	176 search_text=False)
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	177 )
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	178 if newval != value:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	179 value = newval
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	180 changed = True
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	181 new_attrs.append((name, value))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	182 if changed:
667 c9a084ffaee6 Fix case where attributes weren't properly wrapped in an `Attrs` instance if one or more of them were translated by the I18n filter, potentially breaking things further down the chain. Closes #162. cmlenz parents: 600 diff changeset	183 attrs = Attrs(new_attrs)
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	184
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	185 if msgbuf:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	186 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	187 continue
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	188 elif i18n_msg in attrs:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	189 params = attrs.get(i18n_msg)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	190 if params and type(params) is list: # event tuple
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	191 params = params[0][1]
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	192 msgbuf = MessageBuffer(params)
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	193 attrs -= (i18n_comment, i18n_msg)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	194
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	195 yield kind, (tag, attrs), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	196
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	197 elif search_text and kind is TEXT:
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	198 if not msgbuf:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	199 text = data.strip()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	200 if text:
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	201 data = data.replace(text, unicode(gettext(text)))
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	202 yield kind, data, pos
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	203 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	204 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	205
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	206 elif msgbuf and kind is EXPR:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	207 msgbuf.append(kind, data, pos)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	208
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	209 elif not skip and msgbuf and kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	210 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	211 if not msgbuf.depth:
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	212 for event in msgbuf.translate(gettext(msgbuf.format())):
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	213 yield event
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	214 msgbuf = None
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	215 yield kind, data, pos
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	216
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	217 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	218 subkind, substream = data
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	219 new_substream = list(self(substream, ctxt, msgbuf=msgbuf))
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	220 yield kind, (subkind, new_substream), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	221
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	222 elif kind is START_NS and data[1] == I18N_NAMESPACE:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	223 ns_prefixes.append(data[0])
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	224
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	225 elif kind is END_NS and data in ns_prefixes:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	226 ns_prefixes.remove(data)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	227
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	228 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	229 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	230
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	231 GETTEXT_FUNCTIONS = ('_', 'gettext', 'ngettext', 'dgettext', 'dngettext',
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	232 'ugettext', 'ungettext')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	233
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	234 def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS,
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	235 search_text=True, msgbuf=None):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	236 """Extract localizable strings from the given template stream.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	237
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	238 For every string found, this function yields a ``(lineno, function,
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	239 message, comments)`` tuple, where:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	240
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	241 * ``lineno`` is the number of the line on which the string was found,
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	242 * ``function`` is the name of the ``gettext`` function used (if the
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	243 string was extracted from embedded Python code), and
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	244 * ``message`` is the string itself (a ``unicode`` object, or a tuple
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	245 of ``unicode`` objects for functions with multiple string
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	246 arguments).
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	247 * ``comments`` is a list of comments related to the message, extracted
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	248 from ``i18n:comment`` attributes found in the markup
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	249
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	250 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	251 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	252 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	253 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	254 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	255 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	256 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	257 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	258 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	259 ... <p>${ngettext("You have %d item", "You have %d items", num)}</p>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	260 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	261 ... </html>''', filename='example.html')
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	262 >>>
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	263 >>> for line, func, msg, comments in Translator().extract(tmpl.stream):
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	264 ... print "%d, %r, %r" % (line, func, msg)
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	265 3, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	266 6, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	267 7, '_', u'Hello, %(name)s'
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	268 8, 'ngettext', (u'You have %d item', u'You have %d items', None)
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	269
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	270 :param stream: the event stream to extract strings from; can be a
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	271 regular stream or a template stream
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	272 :param gettext_functions: a sequence of function names that should be
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	273 treated as gettext-style localization
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	274 functions
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	275 :param search_text: whether the content of text nodes should be
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	276 extracted (used internally)
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	277
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	278 :note: Changed in 0.4.1: For a function with multiple string arguments
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	279 (such as ``ngettext``), a single item with a tuple of strings is
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	280 yielded, instead an item for each string argument.
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	281 :note: Changed in 0.6: The returned tuples now include a 4th element,
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	282 which is a list of comments for the translator
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	283 """
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	284 if not self.extract_text:
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	285 search_text = False
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	286 skip = 0
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	287 i18n_comment = I18N_NAMESPACE['comment']
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	288 i18n_msg = I18N_NAMESPACE['msg']
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	289 xml_lang = XML_NAMESPACE['lang']
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	290
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	291 for kind, data, pos in stream:
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	292
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	293 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	294 if kind is START:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	295 skip += 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	296 if kind is END:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	297 skip -= 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	298
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	299 if kind is START and not skip:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	300 tag, attrs = data
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	301
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	302 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	303 isinstance(attrs.get(xml_lang), basestring):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	304 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	305 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	306
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	307 for name, value in attrs:
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	308 if search_text and isinstance(value, basestring):
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	309 if name in self.include_attrs:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	310 text = value.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	311 if text:
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	312 yield pos[1], None, text, []
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	313 else:
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	314 for lineno, funcname, text, comments in self.extract(
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	315 _ensure(value), gettext_functions,
535 35a413f3f1dd The I18n filter no longer extracts or translates literal strings in attribute values that also contain expressions. cmlenz parents: 531 diff changeset	316 search_text=False):
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	317 yield lineno, funcname, text, comments
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	318
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	319 if msgbuf:
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	320 msgbuf.append(kind, data, pos)
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	321 else:
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	322 msg_params = attrs.get(i18n_msg)
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	323 if msg_params is not None:
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	324 if type(msg_params) is list: # event tuple
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	325 msg_params = msg_params[0][1]
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	326 msgbuf = MessageBuffer(
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	327 msg_params, attrs.get(i18n_comment), pos[1]
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	328 )
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	329
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	330 elif not skip and search_text and kind is TEXT:
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	331 if not msgbuf:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	332 text = data.strip()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	333 if text and filter(None, [ch.isalpha() for ch in text]):
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	334 yield pos[1], None, text, []
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	335 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	336 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	337
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	338 elif not skip and msgbuf and kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	339 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	340 if not msgbuf.depth:
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	341 yield msgbuf.lineno, None, msgbuf.format(), \
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	342 filter(None, [msgbuf.comment])
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	343 msgbuf = None
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	344
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	345 elif kind is EXPR or kind is EXEC:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	346 if msgbuf:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	347 msgbuf.append(kind, data, pos)
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	348 for funcname, strings in extract_from_code(data,
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	349 gettext_functions):
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	350 yield pos[1], funcname, strings, []
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	351
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	352 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	353 subkind, substream = data
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	354 messages = self.extract(substream, gettext_functions,
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	355 search_text=search_text and not skip,
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	356 msgbuf=msgbuf)
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	357 for lineno, funcname, text, comments in messages:
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	358 yield lineno, funcname, text, comments
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	359
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	360
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	361 class MessageBuffer(object):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	362 """Helper class for managing internationalized mixed content.
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	363
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	364 :since: version 0.5
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	365 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	366
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	367 def __init__(self, params=u'', comment=None, lineno=-1):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	368 """Initialize the message buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	369
776 ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	370 :param params: comma-separated list of parameter names
ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	371 :type params: `basestring`
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	372 :param lineno: the line number on which the first stream event
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	373 belonging to the message was found
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	374 """
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	375 self.params = [name.strip() for name in params.split(',')]
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	376 self.comment = comment
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	377 self.lineno = lineno
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	378 self.string = []
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	379 self.events = {}
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	380 self.values = {}
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	381 self.depth = 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	382 self.order = 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	383 self.stack = [0]
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	384
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	385 def append(self, kind, data, pos):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	386 """Append a stream event to the buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	387
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	388 :param kind: the stream event kind
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	389 :param data: the event data
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	390 :param pos: the position of the event in the source
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	391 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	392 if kind is TEXT:
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	393 self.string.append(data)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	394 self.events.setdefault(self.stack[-1], []).append(None)
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	395 elif kind is EXPR:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	396 param = self.params.pop(0)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	397 self.string.append('%%(%s)s' % param)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	398 self.events.setdefault(self.stack[-1], []).append(None)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	399 self.values[param] = (kind, data, pos)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	400 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	401 if kind is START:
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	402 self.string.append(u'[%d:' % self.order)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	403 self.events.setdefault(self.order, []).append((kind, data, pos))
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	404 self.stack.append(self.order)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	405 self.depth += 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	406 self.order += 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	407 elif kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	408 self.depth -= 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	409 if self.depth:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	410 self.events[self.stack[-1]].append((kind, data, pos))
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	411 self.string.append(u']')
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	412 self.stack.pop()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	413
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	414 def format(self):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	415 """Return a message identifier representing the content in the
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	416 buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	417 """
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	418 return u''.join(self.string).strip()
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	419
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	420 def translate(self, string, regex=re.compile(r'%$(\w+)$s')):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	421 """Interpolate the given message translation with the events in the
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	422 buffer and return the translated stream.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	423
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	424 :param string: the translated message string
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	425 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	426 parts = parse_msg(string)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	427 for order, string in parts:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	428 events = self.events[order]
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	429 while events:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	430 event = events.pop(0)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	431 if event:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	432 yield event
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	433 else:
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	434 if not string:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	435 break
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	436 for idx, part in enumerate(regex.split(string)):
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	437 if idx % 2:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	438 yield self.values[part]
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	439 elif part:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	440 yield TEXT, part, (None, -1, -1)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	441 if not self.events[order] or not self.events[order][0]:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	442 break
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	443
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	444
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	445 def parse_msg(string, regex=re.compile(r'(?:\[(\d+)\:)\|\]')):
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	446 """Parse a translated message using Genshi mixed content message
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	447 formatting.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	448
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	449 >>> parse_msg("See [1:Help].")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	450 [(0, 'See '), (1, 'Help'), (0, '.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	451
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	452 >>> parse_msg("See [1:our [2:Help] page] for details.")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	453 [(0, 'See '), (1, 'our '), (2, 'Help'), (1, ' page'), (0, ' for details.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	454
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	455 >>> parse_msg("[2:Details] finden Sie in [1:Hilfe].")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	456 [(2, 'Details'), (0, ' finden Sie in '), (1, 'Hilfe'), (0, '.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	457
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	458 >>> parse_msg("[1:] Bilder pro Seite anzeigen.")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	459 [(1, ''), (0, ' Bilder pro Seite anzeigen.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	460
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	461 :param string: the translated message string
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	462 :return: a list of ``(order, string)`` tuples
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	463 :rtype: `list`
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	464 """
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	465 parts = []
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	466 stack = [0]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	467 while True:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	468 mo = regex.search(string)
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	469 if not mo:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	470 break
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	471
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	472 if mo.start() or stack[-1]:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	473 parts.append((stack[-1], string[:mo.start()]))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	474 string = string[mo.end():]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	475
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	476 orderno = mo.group(1)
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	477 if orderno is not None:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	478 stack.append(int(orderno))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	479 else:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	480 stack.pop()
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	481 if not stack:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	482 break
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	483
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	484 if string:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	485 parts.append((stack[-1], string))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	486
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	487 return parts
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	488
776 ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	489
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	490 def extract_from_code(code, gettext_functions):
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	491 """Extract strings from Python bytecode.
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	492
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	493 >>> from genshi.template.eval import Expression
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	494
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	495 >>> expr = Expression('_("Hello")')
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	496 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS))
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	497 [('_', u'Hello')]
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	498
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	499 >>> expr = Expression('ngettext("You have %(num)s item", '
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	500 ... '"You have %(num)s items", num)')
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	501 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS))
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	502 [('ngettext', (u'You have %(num)s item', u'You have %(num)s items', None))]
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	503
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	504 :param code: the `Code` object
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	505 :type code: `genshi.template.eval.Code`
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	506 :param gettext_functions: a sequence of function names
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	507 :since: version 0.5
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	508 """
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	509 def _walk(node):
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	510 if isinstance(node, ast.CallFunc) and isinstance(node.node, ast.Name) \
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	511 and node.node.name in gettext_functions:
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	512 strings = []
600 23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	513 def _add(arg):
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	514 if isinstance(arg, ast.Const) \
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	515 and isinstance(arg.value, basestring):
600 23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	516 strings.append(unicode(arg.value, 'utf-8'))
23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	517 elif arg and not isinstance(arg, ast.Keyword):
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	518 strings.append(None)
600 23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	519 [_add(arg) for arg in node.args]
23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	520 _add(node.star_args)
23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	521 _add(node.dstar_args)
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	522 if len(strings) == 1:
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	523 strings = strings[0]
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	524 else:
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	525 strings = tuple(strings)
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	526 yield node.node.name, strings
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	527 else:
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	528 for child in node.getChildNodes():
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	529 for funcname, strings in _walk(child):
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	530 yield funcname, strings
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	531 return _walk(code.ast)
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	532
776 ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	533
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	534 def extract(fileobj, keywords, comment_tags, options):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	535 """Babel extraction method for Genshi templates.
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	536
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	537 :param fileobj: the file-like object the messages should be extracted from
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	538 :param keywords: a list of keywords (i.e. function names) that should be
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	539 recognized as translation functions
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	540 :param comment_tags: a list of translator tags to search for and include
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	541 in the results
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	542 :param options: a dictionary of additional options (optional)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	543 :return: an iterator over ``(lineno, funcname, message, comments)`` tuples
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	544 :rtype: ``iterator``
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	545 """
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	546 template_class = options.get('template_class', MarkupTemplate)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	547 if isinstance(template_class, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	548 module, clsname = template_class.split(':', 1)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	549 template_class = getattr(__import__(module, {}, {}, [clsname]), clsname)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	550 encoding = options.get('encoding', None)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	551
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	552 extract_text = options.get('extract_text', True)
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	553 if isinstance(extract_text, basestring):
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	554 extract_text = extract_text.lower() in ('1', 'on', 'yes', 'true')
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	555
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	556 ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	557 if isinstance(ignore_tags, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	558 ignore_tags = ignore_tags.split()
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	559 ignore_tags = [QName(tag) for tag in ignore_tags]
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	560
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	561 include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	562 if isinstance(include_attrs, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	563 include_attrs = include_attrs.split()
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	564 include_attrs = [QName(attr) for attr in include_attrs]
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	565
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	566 tmpl = template_class(fileobj, filename=getattr(fileobj, 'name', None),
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	567 encoding=encoding)
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	568 translator = Translator(None, ignore_tags, include_attrs, extract_text)
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	569 for message in translator.extract(tmpl.stream, gettext_functions=keywords):
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	570 yield message

Mercurial > genshi > mirror

annotate genshi/filters/i18n.py @ 788:31432f30a6fb trunk