Mercurial > babel > mirror
annotate babel/messages/extract.py @ 75:0f74337264ce trunk
Fixed MIME type of new doc page.
author | cmlenz |
---|---|
date | Fri, 08 Jun 2007 21:54:56 +0000 |
parents | 2df27f49c320 |
children | 116e34b8cefa |
rev | line source |
---|---|
1 | 1 # -*- coding: utf-8 -*- |
2 # | |
3 # Copyright (C) 2007 Edgewall Software | |
4 # All rights reserved. | |
5 # | |
6 # This software is licensed as described in the file COPYING, which | |
7 # you should have received as part of this distribution. The terms | |
8 # are also available at http://babel.edgewall.org/wiki/License. | |
9 # | |
10 # This software consists of voluntary contributions made by many | |
11 # individuals. For the exact contribution history, see the revision | |
12 # history and logs, available at http://babel.edgewall.org/log/. | |
13 | |
14 """Basic infrastructure for extracting localizable messages from source files. | |
15 | |
16 This module defines an extensible system for collecting localizable message | |
17 strings from a variety of sources. A native extractor for Python source files | |
18 is builtin, extractors for other sources can be added using very simple plugins. | |
19 | |
20 The main entry points into the extraction functionality are the functions | |
21 `extract_from_dir` and `extract_from_file`. | |
22 """ | |
23 | |
24 import os | |
44 | 25 try: |
26 set | |
27 except NameError: | |
28 from sets import Set as set | |
1 | 29 import sys |
30 from tokenize import generate_tokens, NAME, OP, STRING | |
31 | |
44 | 32 from babel.util import pathmatch, relpath |
1 | 33 |
34 __all__ = ['extract', 'extract_from_dir', 'extract_from_file'] | |
35 __docformat__ = 'restructuredtext en' | |
36 | |
37 GROUP_NAME = 'babel.extractors' | |
38 | |
12
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
39 DEFAULT_KEYWORDS = { |
10
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
40 '_': None, |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
41 'gettext': None, |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
42 'ngettext': (1, 2), |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
43 'ugettext': None, |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
44 'ungettext': (1, 2), |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
45 'dgettext': (2,), |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
46 'dngettext': (2, 3), |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
47 } |
1 | 48 |
62
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
49 DEFAULT_MAPPING = [('**.py', 'python')] |
1 | 50 |
47
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
51 def extract_from_dir(dirname=os.getcwd(), method_map=DEFAULT_MAPPING, |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
52 options_map=None, keywords=DEFAULT_KEYWORDS, |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
53 callback=None): |
1 | 54 """Extract messages from any source files found in the given directory. |
55 | |
56 This function generates tuples of the form: | |
57 | |
56
f40fc143439c
Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents:
54
diff
changeset
|
58 ``(filename, lineno, message)`` |
1 | 59 |
44 | 60 Which extraction method is used per file is determined by the `method_map` |
61 parameter, which maps extended glob patterns to extraction method names. | |
62 For example, the following is the default mapping: | |
1 | 63 |
62
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
64 >>> method_map = [ |
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
65 ... ('**.py', 'python') |
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
66 ... ] |
1 | 67 |
68 This basically says that files with the filename extension ".py" at any | |
69 level inside the directory should be processed by the "python" extraction | |
44 | 70 method. Files that don't match any of the mapping patterns are ignored. See |
71 the documentation of the `pathmatch` function for details on the pattern | |
72 syntax. | |
1 | 73 |
62
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
74 The following extended mapping would also use the "genshi" extraction |
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
75 method on any file in "templates" subdirectory: |
1 | 76 |
62
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
77 >>> method_map = [ |
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
78 ... ('**/templates/**.*', 'genshi'), |
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
79 ... ('**.py', 'python') |
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
80 ... ] |
44 | 81 |
82 The dictionary provided by the optional `options_map` parameter augments | |
62
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
83 these mappings. It uses extended glob patterns as keys, and the values are |
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
84 dictionaries mapping options names to option values (both strings). |
44 | 85 |
86 The glob patterns of the `options_map` do not necessarily need to be the | |
62
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
87 same as those used in the method mapping. For example, while all files in |
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
88 the ``templates`` folders in an application may be Genshi applications, the |
44 | 89 options for those files may differ based on extension: |
90 | |
91 >>> options_map = { | |
92 ... '**/templates/**.txt': { | |
93 ... 'template_class': 'genshi.template.text.TextTemplate', | |
94 ... 'encoding': 'latin-1' | |
95 ... }, | |
96 ... '**/templates/**.html': { | |
97 ... 'include_attrs': '' | |
98 ... } | |
1 | 99 ... } |
100 | |
101 :param dirname: the path to the directory to extract messages from | |
62
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
102 :param method_map: a list of ``(pattern, method)`` tuples that maps of |
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
103 extraction method names to extended glob patterns |
44 | 104 :param options_map: a dictionary of additional options (optional) |
12
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
105 :param keywords: a dictionary mapping keywords (i.e. names of functions |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
106 that should be recognized as translation functions) to |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
107 tuples that specify which of their arguments contain |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
108 localizable strings |
47
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
109 :param callback: a function that is called for every file that message are |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
110 extracted from, just before the extraction itself is |
75 | 111 performed; the function is passed the filename, the name |
112 of the extraction method and and the options dictionary as | |
113 positional arguments, in that order | |
1 | 114 :return: an iterator over ``(filename, lineno, funcname, message)`` tuples |
115 :rtype: ``iterator`` | |
44 | 116 :see: `pathmatch` |
1 | 117 """ |
44 | 118 if options_map is None: |
119 options_map = {} | |
56
f40fc143439c
Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents:
54
diff
changeset
|
120 |
44 | 121 absname = os.path.abspath(dirname) |
122 for root, dirnames, filenames in os.walk(absname): | |
123 for subdir in dirnames: | |
124 if subdir.startswith('.') or subdir.startswith('_'): | |
125 dirnames.remove(subdir) | |
126 for filename in filenames: | |
127 filename = relpath( | |
128 os.path.join(root, filename).replace(os.sep, '/'), | |
129 dirname | |
130 ) | |
62
2df27f49c320
The order of extraction methods is now preserved (see #10).
cmlenz
parents:
57
diff
changeset
|
131 for pattern, method in method_map: |
44 | 132 if pathmatch(pattern, filename): |
133 filepath = os.path.join(absname, filename) | |
134 options = {} | |
135 for opattern, odict in options_map.items(): | |
136 if pathmatch(opattern, filename): | |
137 options = odict | |
47
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
138 if callback: |
57
d930a3dfbf3d
* The `extract_messages` distutils command now operators on configurable input directories again, instead of the complete current directory. The `input_dirs` default to the package directories.
cmlenz
parents:
56
diff
changeset
|
139 callback(filename, method, options) |
56
f40fc143439c
Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents:
54
diff
changeset
|
140 for lineno, message in extract_from_file(method, filepath, |
44 | 141 keywords=keywords, |
142 options=options): | |
56
f40fc143439c
Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents:
54
diff
changeset
|
143 yield filename, lineno, message |
57
d930a3dfbf3d
* The `extract_messages` distutils command now operators on configurable input directories again, instead of the complete current directory. The `input_dirs` default to the package directories.
cmlenz
parents:
56
diff
changeset
|
144 break |
1 | 145 |
12
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
146 def extract_from_file(method, filename, keywords=DEFAULT_KEYWORDS, |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
147 options=None): |
1 | 148 """Extract messages from a specific file. |
149 | |
150 This function returns a list of tuples of the form: | |
151 | |
152 ``(lineno, funcname, message)`` | |
153 | |
154 :param filename: the path to the file to extract messages from | |
155 :param method: a string specifying the extraction method (.e.g. "python") | |
12
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
156 :param keywords: a dictionary mapping keywords (i.e. names of functions |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
157 that should be recognized as translation functions) to |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
158 tuples that specify which of their arguments contain |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
159 localizable strings |
1 | 160 :param options: a dictionary of additional options (optional) |
161 :return: the list of extracted messages | |
162 :rtype: `list` | |
163 """ | |
164 fileobj = open(filename, 'U') | |
165 try: | |
166 return list(extract(method, fileobj, keywords, options=options)) | |
167 finally: | |
168 fileobj.close() | |
169 | |
12
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
170 def extract(method, fileobj, keywords=DEFAULT_KEYWORDS, options=None): |
1 | 171 """Extract messages from the given file-like object using the specified |
172 extraction method. | |
173 | |
174 This function returns a list of tuples of the form: | |
175 | |
56
f40fc143439c
Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents:
54
diff
changeset
|
176 ``(lineno, message)`` |
1 | 177 |
178 The implementation dispatches the actual extraction to plugins, based on the | |
179 value of the ``method`` parameter. | |
180 | |
181 >>> source = '''# foo module | |
182 ... def run(argv): | |
183 ... print _('Hello, world!') | |
184 ... ''' | |
10
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
185 |
1 | 186 >>> from StringIO import StringIO |
187 >>> for message in extract('python', StringIO(source)): | |
188 ... print message | |
56
f40fc143439c
Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents:
54
diff
changeset
|
189 (3, 'Hello, world!') |
1 | 190 |
191 :param method: a string specifying the extraction method (.e.g. "python") | |
192 :param fileobj: the file-like object the messages should be extracted from | |
12
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
193 :param keywords: a dictionary mapping keywords (i.e. names of functions |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
194 that should be recognized as translation functions) to |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
195 tuples that specify which of their arguments contain |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
196 localizable strings |
1 | 197 :param options: a dictionary of additional options (optional) |
198 :return: the list of extracted messages | |
199 :rtype: `list` | |
200 :raise ValueError: if the extraction method is not registered | |
201 """ | |
12
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
202 from pkg_resources import working_set |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
203 |
1 | 204 for entry_point in working_set.iter_entry_points(GROUP_NAME, method): |
205 func = entry_point.load(require=True) | |
10
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
206 m = [] |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
207 for lineno, funcname, messages in func(fileobj, keywords.keys(), |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
208 options=options or {}): |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
209 if isinstance(messages, (list, tuple)): |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
210 msgs = [] |
12
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
211 for index in keywords[funcname]: |
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
212 msgs.append(messages[index - 1]) |
10
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
213 messages = tuple(msgs) |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
214 if len(messages) == 1: |
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
215 messages = messages[0] |
56
f40fc143439c
Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents:
54
diff
changeset
|
216 yield lineno, messages |
10
4130d9c6cb34
Both Babel's [source:trunk/babel/catalog/frontend.py frontend] and [source:trunk/babel/catalog/extract.py extract] now handle keyword indices. Also added an extra boolean flag so that the default keywords defined by Babel are not included in the keywords to search for when extracting strings.
palgarvio
parents:
1
diff
changeset
|
217 return |
12
e6ba3e878b10
* Removed pkg_resources/setuptools requirement from various places.
cmlenz
parents:
10
diff
changeset
|
218 |
1 | 219 raise ValueError('Unknown extraction method %r' % method) |
220 | |
57
d930a3dfbf3d
* The `extract_messages` distutils command now operators on configurable input directories again, instead of the complete current directory. The `input_dirs` default to the package directories.
cmlenz
parents:
56
diff
changeset
|
221 def extract_nothing(fileobj, keywords, options): |
d930a3dfbf3d
* The `extract_messages` distutils command now operators on configurable input directories again, instead of the complete current directory. The `input_dirs` default to the package directories.
cmlenz
parents:
56
diff
changeset
|
222 """Pseudo extractor that does not actually extract anything, but simply |
d930a3dfbf3d
* The `extract_messages` distutils command now operators on configurable input directories again, instead of the complete current directory. The `input_dirs` default to the package directories.
cmlenz
parents:
56
diff
changeset
|
223 returns an empty list. |
d930a3dfbf3d
* The `extract_messages` distutils command now operators on configurable input directories again, instead of the complete current directory. The `input_dirs` default to the package directories.
cmlenz
parents:
56
diff
changeset
|
224 """ |
d930a3dfbf3d
* The `extract_messages` distutils command now operators on configurable input directories again, instead of the complete current directory. The `input_dirs` default to the package directories.
cmlenz
parents:
56
diff
changeset
|
225 return [] |
d930a3dfbf3d
* The `extract_messages` distutils command now operators on configurable input directories again, instead of the complete current directory. The `input_dirs` default to the package directories.
cmlenz
parents:
56
diff
changeset
|
226 |
1 | 227 def extract_genshi(fileobj, keywords, options): |
228 """Extract messages from Genshi templates. | |
229 | |
230 :param fileobj: the file-like object the messages should be extracted from | |
231 :param keywords: a list of keywords (i.e. function names) that should be | |
232 recognized as translation functions | |
233 :param options: a dictionary of additional options (optional) | |
234 :return: an iterator over ``(lineno, funcname, message)`` tuples | |
235 :rtype: ``iterator`` | |
236 """ | |
237 from genshi.filters.i18n import Translator | |
238 from genshi.template import MarkupTemplate | |
47
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
239 |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
240 template_class = options.get('template_class', MarkupTemplate) |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
241 if isinstance(template_class, basestring): |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
242 module, clsname = template_class.split(':', 1) |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
243 template_class = getattr(__import__(module, {}, {}, [clsname]), clsname) |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
244 encoding = options.get('encoding', None) |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
245 |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
246 ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS) |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
247 if isinstance(ignore_tags, basestring): |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
248 ignore_tags = ignore_tags.split() |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
249 include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS) |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
250 if isinstance(include_attrs, basestring): |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
251 include_attrs = include_attrs.split() |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
252 |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
253 tmpl = template_class(fileobj, filename=getattr(fileobj, 'name'), |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
254 encoding=encoding) |
f8469ab4b257
Support passing extraction method mapping and options from the frontends (see #4). No distutils/setuptools keyword supported yet, but the rest seems to be working okay.
cmlenz
parents:
44
diff
changeset
|
255 translator = Translator(None, ignore_tags, include_attrs) |
1 | 256 for message in translator.extract(tmpl.stream, gettext_functions=keywords): |
257 yield message | |
258 | |
259 def extract_python(fileobj, keywords, options): | |
260 """Extract messages from Python source code. | |
261 | |
262 :param fileobj: the file-like object the messages should be extracted from | |
263 :param keywords: a list of keywords (i.e. function names) that should be | |
264 recognized as translation functions | |
265 :param options: a dictionary of additional options (optional) | |
266 :return: an iterator over ``(lineno, funcname, message)`` tuples | |
267 :rtype: ``iterator`` | |
268 """ | |
269 funcname = None | |
270 lineno = None | |
271 buf = [] | |
272 messages = [] | |
273 in_args = False | |
274 | |
275 tokens = generate_tokens(fileobj.readline) | |
276 for tok, value, (lineno, _), _, _ in tokens: | |
277 if funcname and tok == OP and value == '(': | |
278 in_args = True | |
279 elif funcname and in_args: | |
280 if tok == OP and value == ')': | |
281 in_args = False | |
282 if buf: | |
283 messages.append(''.join(buf)) | |
284 del buf[:] | |
285 if filter(None, messages): | |
286 if len(messages) > 1: | |
287 messages = tuple(messages) | |
288 else: | |
289 messages = messages[0] | |
290 yield lineno, funcname, messages | |
291 funcname = lineno = None | |
292 messages = [] | |
293 elif tok == STRING: | |
294 if lineno is None: | |
295 lineno = stup[0] | |
36
cfd15b7921f9
Fix for #8: fix extraction of strings from Python source using prefixes ('u' or 'r') or triple quotes.
cmlenz
parents:
12
diff
changeset
|
296 # Unwrap quotes in a safe manner |
cfd15b7921f9
Fix for #8: fix extraction of strings from Python source using prefixes ('u' or 'r') or triple quotes.
cmlenz
parents:
12
diff
changeset
|
297 buf.append(eval(value, {'__builtins__':{}}, {})) |
1 | 298 elif tok == OP and value == ',': |
299 messages.append(''.join(buf)) | |
300 del buf[:] | |
301 elif funcname: | |
302 funcname = None | |
303 elif tok == NAME and value in keywords: | |
304 funcname = value |