genshi/mirror: genshi/filters/i18n.py annotate

annotate genshi/filters/i18n.py @ 794:f9e23d472a6e trunk

Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching).

author	cmlenz
date	Tue, 16 Dec 2008 23:02:36 +0000
parents	da90cee22560
children	878c4313c7d5

rev	line source
531 9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	1 # -- coding: utf-8 --
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	2 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	3 # Copyright (C) 2007 Edgewall Software
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	4 # All rights reserved.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	5 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	6 # This software is licensed as described in the file COPYING, which
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	7 # you should have received as part of this distribution. The terms
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	8 # are also available at http://genshi.edgewall.org/wiki/License.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	9 #
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	10 # This software consists of voluntary contributions made by many
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	11 # individuals. For the exact contribution history, see the revision
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	12 # history and logs, available at http://genshi.edgewall.org/log/.
9ce91447fc6b Add missing copyright header to i18n.py. cmlenz parents: 528 diff changeset	13
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	14 """Utilities for internationalization and localization of templates.
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	15
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	16 :since: version 0.4
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	17 """
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	18
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	19 from gettext import NullTranslations
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	20 import re
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	21 from types import FunctionType
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	22
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	23 from genshi.core import Attrs, Namespace, QName, START, END, TEXT, START_NS, \
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	24 END_NS, XML_NAMESPACE, _ensure
794 f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	25 from genshi.template.eval import _ast
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	26 from genshi.template.base import DirectiveFactory, EXPR, SUB, _apply_directives
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	27 from genshi.template.directives import Directive
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	28 from genshi.template.markup import MarkupTemplate, EXEC
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	29
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	30 __all__ = ['Translator', 'extract']
501 5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions. cmlenz parents: 493 diff changeset	31 __docformat__ = 'restructuredtext en'
5e7604c2d60d Added new markup transformation filter contributed by Alec Thomas (#122). This provides gorgeous jQuery-inspired stream transformation capabilities based on XPath expressions. cmlenz parents: 493 diff changeset	32
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	33 I18N_NAMESPACE = Namespace('http://genshi.edgewall.org/i18n')
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	34
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	35
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	36 class CommentDirective(Directive):
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	37
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	38 __slots__ = []
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	39
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	40 @classmethod
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	41 def attach(cls, template, stream, value, namespaces, pos):
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	42 return None, stream
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	43
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	44
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	45 class MsgDirective(Directive):
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	46
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	47 __slots__ = ['params']
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	48
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	49 def __init__(self, value, template, hints=None, namespaces=None,
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	50 lineno=-1, offset=-1):
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	51 Directive.__init__(self, None, template, namespaces, lineno, offset)
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	52 self.params = [name.strip() for name in value.split(',')]
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	53
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	54 def __call__(self, stream, directives, ctxt, **vars):
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	55 msgbuf = MessageBuffer(self.params)
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	56
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	57 stream = iter(stream)
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	58 yield stream.next() # the outer start tag
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	59 previous = stream.next()
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	60 for event in stream:
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	61 msgbuf.append(*previous)
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	62 previous = event
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	63
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	64 gettext = ctxt.get('_i18n.gettext')
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	65 for event in msgbuf.translate(gettext(msgbuf.format())):
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	66 yield event
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	67
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	68 yield previous # the outer end tag
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	69
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	70
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	71 class Translator(DirectiveFactory):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	72 """Can extract and translate localizable strings from markup streams and
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	73 templates.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	74
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	75 For example, assume the followng template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	76
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	77 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	78 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	79 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	80 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	81 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	82 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	83 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	84 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	85 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	86 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	87 ... </html>''', filename='example.html')
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	88
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	89 For demonstration, we define a dummy ``gettext``-style function with a
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	90 hard-coded translation table, and pass that to the `Translator` initializer:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	91
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	92 >>> def pseudo_gettext(string):
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	93 ... return {
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	94 ... 'Example': 'Beispiel',
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	95 ... 'Hello, %(name)s': 'Hallo, %(name)s'
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	96 ... }[string]
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	97 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	98 >>> translator = Translator(pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	99
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	100 Next, the translator needs to be prepended to any already defined filters
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	101 on the template:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	102
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	103 >>> tmpl.filters.insert(0, translator)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	104
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	105 When generating the template output, our hard-coded translations should be
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	106 applied as expected:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	107
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	108 >>> print tmpl.generate(username='Hans', _=pseudo_gettext)
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	109 <html>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	110 <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	111 <title>Beispiel</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	112 </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	113 <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	114 <h1>Beispiel</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	115 <p>Hallo, Hans</p>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	116 </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	117 </html>
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	118
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	119 Note that elements defining ``xml:lang`` attributes that do not contain
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	120 variable expressions are ignored by this filter. That can be used to
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	121 exclude specific parts of a template from being extracted and translated.
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	122 """
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	123
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	124 directives = [
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	125 ('comment', CommentDirective),
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	126 ('msg', MsgDirective)
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	127 ]
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	128
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	129 IGNORE_TAGS = frozenset([
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	130 QName('script'), QName('http://www.w3.org/1999/xhtml}script'),
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	131 QName('style'), QName('http://www.w3.org/1999/xhtml}style')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	132 ])
467 23082baddbf9 Add some more localizable HTML attributes to the I18n filter. cmlenz parents: 466 diff changeset	133 INCLUDE_ATTRS = frozenset(['abbr', 'alt', 'label', 'prompt', 'standby',
23082baddbf9 Add some more localizable HTML attributes to the I18n filter. cmlenz parents: 466 diff changeset	134 'summary', 'title'])
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	135 NAMESPACE = I18N_NAMESPACE
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	136
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	137 def __init__(self, translate=NullTranslations(), ignore_tags=IGNORE_TAGS,
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	138 include_attrs=INCLUDE_ATTRS, extract_text=True):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	139 """Initialize the translator.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	140
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	141 :param translate: the translation function, for example ``gettext`` or
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	142 ``ugettext``.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	143 :param ignore_tags: a set of tag names that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	144 :param include_attrs: a set of attribute names should be localized
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	145 :param extract_text: whether the content of text nodes should be
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	146 extracted, or only text in explicit ``gettext``
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	147 function calls
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	148
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	149 :note: Changed in 0.6: the `translate` parameter can now be either
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	150 a ``gettext``-style function, or an object compatible with the
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	151 ``NullTransalations`` or ``GNUTranslations`` interface
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	152 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	153 self.translate = translate
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	154 self.ignore_tags = ignore_tags
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	155 self.include_attrs = include_attrs
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	156 self.extract_text = extract_text
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	157
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	158 def __call__(self, stream, ctxt=None, search_text=True):
448 1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	159 """Translate any localizable strings in the given stream.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	160
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	161 This function shouldn't be called directly. Instead, an instance of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	162 the `Translator` class should be registered as a filter with the
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	163 `Template` or the `TemplateLoader`, or applied as a regular stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	164 filter. If used as a template filter, it should be inserted in front of
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	165 all the default filters.
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	166
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	167 :param stream: the markup event stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	168 :param ctxt: the template context (not used)
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	169 :param search_text: whether text nodes should be translated (used
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	170 internally)
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	171 :return: the localized stream
1154f2aadb6c Add support for HTML5 doctype. cmlenz parents: 446 diff changeset	172 """
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	173 ignore_tags = self.ignore_tags
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	174 include_attrs = self.include_attrs
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	175 skip = 0
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	176 xml_lang = XML_NAMESPACE['lang']
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	177
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	178 if type(self.translate) is FunctionType:
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	179 gettext = self.translate
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	180 else:
31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	181 gettext = self.translate.ugettext
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	182 if ctxt:
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	183 ctxt['_i18n.gettext'] = gettext
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	184
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	185 extract_text = self.extract_text
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	186 if not extract_text:
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	187 search_text = False
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	188
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	189 for kind, data, pos in stream:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	190
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	191 # skip chunks that should not be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	192 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	193 if kind is START:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	194 skip += 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	195 elif kind is END:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	196 skip -= 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	197 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	198 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	199
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	200 # handle different events that can be localized
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	201 if kind is START:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	202 tag, attrs = data
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	203 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	204 isinstance(attrs.get(xml_lang), basestring):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	205 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	206 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	207 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	208
493 3f6582a5a4a5 Fix another bug in the translation filter: translated attributes were getting added instead of replaced. cmlenz parents: 485 diff changeset	209 new_attrs = []
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	210 changed = False
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	211 for name, value in attrs:
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	212 newval = value
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	213 if extract_text and isinstance(value, basestring):
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	214 if name in include_attrs:
788 31432f30a6fb Change `Translator` class to accept either a `gettext`-style function, or an object compatible with the `NullTranslations` / `GNUTranslations` interface. cmlenz parents: 787 diff changeset	215 newval = gettext(value)
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	216 else:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	217 newval = list(self(_ensure(value), ctxt,
785 36fb0a57fe74 Fix for #250: ignore expressions in attribute values when inside `i18n:msg` elements. cmlenz parents: 776 diff changeset	218 search_text=False)
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	219 )
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	220 if newval != value:
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	221 value = newval
5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	222 changed = True
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	223 new_attrs.append((name, value))
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	224 if changed:
667 c9a084ffaee6 Fix case where attributes weren't properly wrapped in an `Attrs` instance if one or more of them were translated by the I18n filter, potentially breaking things further down the chain. Closes #162. cmlenz parents: 600 diff changeset	225 attrs = Attrs(new_attrs)
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	226
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	227 yield kind, (tag, attrs), pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	228
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	229 elif search_text and kind is TEXT:
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	230 text = data.strip()
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	231 if text:
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	232 data = data.replace(text, unicode(gettext(text)))
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	233 yield kind, data, pos
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	234
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	235 elif kind is SUB:
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	236 directives, substream = data
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	237 # If this is an i18n:msg directive, no need to translate text
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	238 # nodes here
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	239 is_msg = filter(None, [isinstance(d, MsgDirective)
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	240 for d in directives])
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	241 substream = list(self(substream, ctxt,
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	242 search_text=not is_msg))
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	243 yield kind, (directives, substream), pos
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	244
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	245 else:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	246 yield kind, data, pos
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	247
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	248 GETTEXT_FUNCTIONS = ('_', 'gettext', 'ngettext', 'dgettext', 'dngettext',
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	249 'ugettext', 'ungettext')
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	250
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	251 def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS,
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	252 search_text=True, msgbuf=None):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	253 """Extract localizable strings from the given template stream.
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	254
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	255 For every string found, this function yields a ``(lineno, function,
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	256 message, comments)`` tuple, where:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	257
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	258 * ``lineno`` is the number of the line on which the string was found,
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	259 * ``function`` is the name of the ``gettext`` function used (if the
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	260 string was extracted from embedded Python code), and
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	261 * ``message`` is the string itself (a ``unicode`` object, or a tuple
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	262 of ``unicode`` objects for functions with multiple string
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	263 arguments).
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	264 * ``comments`` is a list of comments related to the message, extracted
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	265 from ``i18n:comment`` attributes found in the markup
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	266
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	267 >>> from genshi.template import MarkupTemplate
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	268 >>>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	269 >>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	270 ... <head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	271 ... <title>Example</title>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	272 ... </head>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	273 ... <body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	274 ... <h1>Example</h1>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	275 ... <p>${_("Hello, %(name)s") % dict(name=username)}</p>
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	276 ... <p>${ngettext("You have %d item", "You have %d items", num)}</p>
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	277 ... </body>
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	278 ... </html>''', filename='example.html')
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	279 >>>
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	280 >>> for line, func, msg, comments in Translator().extract(tmpl.stream):
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	281 ... print "%d, %r, %r" % (line, func, msg)
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	282 3, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	283 6, None, u'Example'
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	284 7, '_', u'Hello, %(name)s'
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	285 8, 'ngettext', (u'You have %d item', u'You have %d items', None)
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	286
450 94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	287 :param stream: the event stream to extract strings from; can be a
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	288 regular stream or a template stream
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	289 :param gettext_functions: a sequence of function names that should be
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	290 treated as gettext-style localization
94601511cd68 Extend the I18n extraction to also yield function names if applicable. cmlenz parents: 448 diff changeset	291 functions
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	292 :param search_text: whether the content of text nodes should be
fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	293 extracted (used internally)
469 2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	294
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	295 :note: Changed in 0.4.1: For a function with multiple string arguments
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	296 (such as ``ngettext``), a single item with a tuple of strings is
2d3246f9ea54 The I18n extraction now returns a tuple of strings for `ngettext` and similar functions. cmlenz parents: 467 diff changeset	297 yielded, instead an item for each string argument.
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	298 :note: Changed in 0.6: The returned tuples now include a 4th element,
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	299 which is a list of comments for the translator
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	300 """
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	301 if not self.extract_text:
2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	302 search_text = False
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	303 skip = 0
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	304 i18n_comment = I18N_NAMESPACE['comment']
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	305 i18n_msg = I18N_NAMESPACE['msg']
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	306 xml_lang = XML_NAMESPACE['lang']
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	307
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	308 for kind, data, pos in stream:
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	309
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	310 if skip:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	311 if kind is START:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	312 skip += 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	313 if kind is END:
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	314 skip -= 1
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	315
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	316 if kind is START and not skip:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	317 tag, attrs = data
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	318
522 082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	319 if tag in self.ignore_tags or \
082535e5087c The I18n filter now skips the content of elements that have an `xml:lang` attribute with a fixed string value. Basically, `xml:lang` can now be used as a flag to mark specific sections as not needing localization. cmlenz parents: 501 diff changeset	320 isinstance(attrs.get(xml_lang), basestring):
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	321 skip += 1
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	322 continue
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	323
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	324 for name, value in attrs:
594 2bbaa61b328f Add option to I18n filter to only extract strings in gettext function calls. cmlenz parents: 576 diff changeset	325 if search_text and isinstance(value, basestring):
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	326 if name in self.include_attrs:
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	327 text = value.strip()
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	328 if text:
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	329 yield pos[1], None, text, []
483 5cc92db755c5 Fix for handling of interpolated attribute values in translation filter. cmlenz parents: 481 diff changeset	330 else:
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	331 for lineno, funcname, text, comments in self.extract(
485 fb66fb3e4b49 Follow-up to [583]: Don't extract strings from interpolated attribute values for attributes that shouldn't be included. cmlenz parents: 483 diff changeset	332 _ensure(value), gettext_functions,
535 35a413f3f1dd The I18n filter no longer extracts or translates literal strings in attribute values that also contain expressions. cmlenz parents: 531 diff changeset	333 search_text=False):
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	334 yield lineno, funcname, text, comments
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	335
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	336 if msgbuf:
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	337 msgbuf.append(kind, data, pos)
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	338 else:
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	339 msg_params = attrs.get(i18n_msg)
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	340 if msg_params is not None:
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	341 if type(msg_params) is list: # event tuple
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	342 msg_params = msg_params[0][1]
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	343 msgbuf = MessageBuffer(
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	344 msg_params, attrs.get(i18n_comment), pos[1]
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	345 )
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	346
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	347 elif not skip and search_text and kind is TEXT:
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	348 if not msgbuf:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	349 text = data.strip()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	350 if text and filter(None, [ch.isalpha() for ch in text]):
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	351 yield pos[1], None, text, []
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	352 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	353 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	354
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	355 elif not skip and msgbuf and kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	356 msgbuf.append(kind, data, pos)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	357 if not msgbuf.depth:
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	358 yield msgbuf.lineno, None, msgbuf.format(), \
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	359 filter(None, [msgbuf.comment])
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	360 msgbuf = None
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	361
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	362 elif kind is EXPR or kind is EXEC:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	363 if msgbuf:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	364 msgbuf.append(kind, data, pos)
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	365 for funcname, strings in extract_from_code(data,
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	366 gettext_functions):
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	367 yield pos[1], funcname, strings, []
446 fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	368
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	369 elif kind is SUB:
fd9c4f7a249a Add basic I18n/L10n functionality, based on GenshiRecipes/Localization. cmlenz parents: diff changeset	370 subkind, substream = data
549 7214c1bdb383 The I18n filter now extracts text from translation functions in ignored tags. Fixes #132. cmlenz parents: 535 diff changeset	371 messages = self.extract(substream, gettext_functions,
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	372 search_text=search_text and not skip,
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	373 msgbuf=msgbuf)
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	374 for lineno, funcname, text, comments in messages:
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	375 yield lineno, funcname, text, comments
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	376
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	377
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	378 class MessageBuffer(object):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	379 """Helper class for managing internationalized mixed content.
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	380
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	381 :since: version 0.5
b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	382 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	383
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	384 def __init__(self, params=u'', comment=None, lineno=-1):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	385 """Initialize the message buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	386
776 ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	387 :param params: comma-separated list of parameter names
ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	388 :type params: `basestring`
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	389 :param lineno: the line number on which the first stream event
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	390 belonging to the message was found
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	391 """
790 da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	392 if isinstance(params, basestring):
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	393 params = [name.strip() for name in params.split(',')]
da90cee22560 Merged the custom-directives branch back into trunk. cmlenz parents: 788 diff changeset	394 self.params = params
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	395 self.comment = comment
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	396 self.lineno = lineno
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	397 self.string = []
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	398 self.events = {}
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	399 self.values = {}
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	400 self.depth = 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	401 self.order = 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	402 self.stack = [0]
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	403
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	404 def append(self, kind, data, pos):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	405 """Append a stream event to the buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	406
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	407 :param kind: the stream event kind
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	408 :param data: the event data
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	409 :param pos: the position of the event in the source
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	410 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	411 if kind is TEXT:
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	412 self.string.append(data)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	413 self.events.setdefault(self.stack[-1], []).append(None)
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	414 elif kind is EXPR:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	415 param = self.params.pop(0)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	416 self.string.append('%%(%s)s' % param)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	417 self.events.setdefault(self.stack[-1], []).append(None)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	418 self.values[param] = (kind, data, pos)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	419 else:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	420 if kind is START:
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	421 self.string.append(u'[%d:' % self.order)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	422 self.events.setdefault(self.order, []).append((kind, data, pos))
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	423 self.stack.append(self.order)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	424 self.depth += 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	425 self.order += 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	426 elif kind is END:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	427 self.depth -= 1
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	428 if self.depth:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	429 self.events[self.stack[-1]].append((kind, data, pos))
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	430 self.string.append(u']')
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	431 self.stack.pop()
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	432
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	433 def format(self):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	434 """Return a message identifier representing the content in the
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	435 buffer.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	436 """
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	437 return u''.join(self.string).strip()
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	438
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	439 def translate(self, string, regex=re.compile(r'%$(\w+)$s')):
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	440 """Interpolate the given message translation with the events in the
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	441 buffer and return the translated stream.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	442
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	443 :param string: the translated message string
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	444 """
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	445 parts = parse_msg(string)
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	446 for order, string in parts:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	447 events = self.events[order]
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	448 while events:
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	449 event = events.pop(0)
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	450 if event:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	451 yield event
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	452 else:
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	453 if not string:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	454 break
775 886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	455 for idx, part in enumerate(regex.split(string)):
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	456 if idx % 2:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	457 yield self.values[part]
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	458 elif part:
886934df7fea Support for parameters in internationalized `i18n:msg` content. See #129. cmlenz parents: 762 diff changeset	459 yield TEXT, part, (None, -1, -1)
560 7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	460 if not self.events[order] or not self.events[order][0]:
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	461 break
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	462
7e83be231f96 Start implementation of advanced I18n as dicussed in #129 and the MailingList. This is not complete yet, but many simple cases work okay. cmlenz parents: 549 diff changeset	463
738 3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	464 def parse_msg(string, regex=re.compile(r'(?:\[(\d+)\:)\|\]')):
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	465 """Parse a translated message using Genshi mixed content message
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	466 formatting.
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	467
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	468 >>> parse_msg("See [1:Help].")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	469 [(0, 'See '), (1, 'Help'), (0, '.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	470
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	471 >>> parse_msg("See [1:our [2:Help] page] for details.")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	472 [(0, 'See '), (1, 'our '), (2, 'Help'), (1, ' page'), (0, ' for details.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	473
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	474 >>> parse_msg("[2:Details] finden Sie in [1:Hilfe].")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	475 [(2, 'Details'), (0, ' finden Sie in '), (1, 'Hilfe'), (0, '.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	476
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	477 >>> parse_msg("[1:] Bilder pro Seite anzeigen.")
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	478 [(1, ''), (0, ' Bilder pro Seite anzeigen.')]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	479
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	480 :param string: the translated message string
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	481 :return: a list of ``(order, string)`` tuples
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	482 :rtype: `list`
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	483 """
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	484 parts = []
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	485 stack = [0]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	486 while True:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	487 mo = regex.search(string)
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	488 if not mo:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	489 break
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	490
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	491 if mo.start() or stack[-1]:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	492 parts.append((stack[-1], string[:mo.start()]))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	493 string = string[mo.end():]
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	494
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	495 orderno = mo.group(1)
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	496 if orderno is not None:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	497 stack.append(int(orderno))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	498 else:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	499 stack.pop()
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	500 if not stack:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	501 break
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	502
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	503 if string:
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	504 parts.append((stack[-1], string))
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	505
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	506 return parts
3b8a38fcc1ab Minor cleanup in the i18n module. cmlenz parents: 667 diff changeset	507
776 ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	508
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	509 def extract_from_code(code, gettext_functions):
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	510 """Extract strings from Python bytecode.
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	511
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	512 >>> from genshi.template.eval import Expression
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	513
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	514 >>> expr = Expression('_("Hello")')
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	515 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS))
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	516 [('_', u'Hello')]
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	517
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	518 >>> expr = Expression('ngettext("You have %(num)s item", '
21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	519 ... '"You have %(num)s items", num)')
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	520 >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS))
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	521 [('ngettext', (u'You have %(num)s item', u'You have %(num)s items', None))]
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	522
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	523 :param code: the `Code` object
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	524 :type code: `genshi.template.eval.Code`
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	525 :param gettext_functions: a sequence of function names
576 b00765a115a5 Improve docs on `Stream.select()` for #135. cmlenz parents: 565 diff changeset	526 :since: version 0.5
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	527 """
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	528 def _walk(node):
794 f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	529 if isinstance(node, _ast.Call) and isinstance(node.func, _ast.Name) \
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	530 and node.func.id in gettext_functions:
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	531 strings = []
600 23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	532 def _add(arg):
794 f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	533 if isinstance(arg, _ast.Str) and isinstance(arg.s, basestring):
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	534 strings.append(unicode(arg.s, 'utf-8'))
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	535 elif arg:
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	536 strings.append(None)
600 23a4784203ae Handle starargs and dstarargs in the I18n extraction code. cmlenz parents: 596 diff changeset	537 [_add(arg) for arg in node.args]
794 f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	538 _add(node.starargs)
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	539 _add(node.kwargs)
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	540 if len(strings) == 1:
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	541 strings = strings[0]
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	542 else:
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	543 strings = tuple(strings)
794 f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	544 yield node.func.id, strings
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	545 elif node._fields:
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	546 children = []
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	547 for field in node._fields:
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	548 child = getattr(node, field, None)
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	549 if isinstance(child, list):
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	550 for elem in child:
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	551 children.append(elem)
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	552 elif isinstance(child, _ast.AST):
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	553 children.append(child)
f9e23d472a6e Merged AST branch back into trunk. Most of this code was written by Marcin Kurczych for his Google Summer of Code 2008 project. The merge of this branch means that Genshi now uses the native `_ast` module on Python >= 2.5, and an emulation thereof on Python 2.4. This replaces the usage of the `compiler` package, which was deprecated in Python 2.6 and removed in Python 3.0. Another effect is that Genshi now runs on Google AppEngine (although performance is bad due to the lack of template caching). cmlenz parents: 790 diff changeset	554 for child in children:
565 53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	555 for funcname, strings in _walk(child):
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	556 yield funcname, strings
53b37e4f2921 * The I18n extractor now handles gettext function calls that use non-string parameters as well as keyword arguments. cmlenz parents: 561 diff changeset	557 return _walk(code.ast)
561 21b473243e63 Move code for extracting messages from bytecode into a separate function. cmlenz parents: 560 diff changeset	558
776 ddb58d74c8ee Added tests for the parameter support added to advanced internationalization in [901]. See #129. cmlenz parents: 775 diff changeset	559
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	560 def extract(fileobj, keywords, comment_tags, options):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	561 """Babel extraction method for Genshi templates.
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	562
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	563 :param fileobj: the file-like object the messages should be extracted from
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	564 :param keywords: a list of keywords (i.e. function names) that should be
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	565 recognized as translation functions
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	566 :param comment_tags: a list of translator tags to search for and include
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	567 in the results
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	568 :param options: a dictionary of additional options (optional)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	569 :return: an iterator over ``(lineno, funcname, message, comments)`` tuples
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	570 :rtype: ``iterator``
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	571 """
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	572 template_class = options.get('template_class', MarkupTemplate)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	573 if isinstance(template_class, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	574 module, clsname = template_class.split(':', 1)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	575 template_class = getattr(__import__(module, {}, {}, [clsname]), clsname)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	576 encoding = options.get('encoding', None)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	577
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	578 extract_text = options.get('extract_text', True)
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	579 if isinstance(extract_text, basestring):
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	580 extract_text = extract_text.lower() in ('1', 'on', 'yes', 'true')
badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	581
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	582 ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	583 if isinstance(ignore_tags, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	584 ignore_tags = ignore_tags.split()
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	585 ignore_tags = [QName(tag) for tag in ignore_tags]
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	586
528 24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	587 include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS)
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	588 if isinstance(include_attrs, basestring):
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	589 include_attrs = include_attrs.split()
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	590 include_attrs = [QName(attr) for attr in include_attrs]
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	591
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	592 tmpl = template_class(fileobj, filename=getattr(fileobj, 'name', None),
24df908da22d Integrated [http://babel.edgewall.org/ Babel] message extraction plugin, and added I18n doc page. cmlenz parents: 522 diff changeset	593 encoding=encoding)
596 badb5e5b7bb9 Follow-up to [708]. The added `extract_text` option wasn't actually being handled by the Babel extraction plugin. cmlenz parents: 594 diff changeset	594 translator = Translator(None, ignore_tags, include_attrs, extract_text)
787 d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	595 for message in translator.extract(tmpl.stream, gettext_functions=keywords):
d7366797440f Add support for supplying comments on localizable messages in the i18n filter. Based on patch by Pedro Algarvio on #129. cmlenz parents: 785 diff changeset	596 yield message

Mercurial > genshi > mirror

annotate genshi/filters/i18n.py @ 794:f9e23d472a6e trunk