Mercurial > babel > old > babel-test

annotate babel/messages/pofile.py @ 158:17dd31f104f5

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Minor cleanup in the `pofile` module.
author cmlenz
date Thu, 21 Jun 2007 12:10:59 +0000
parents ba5150e9544e
children 04c56e82c98b
rev   line source
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
1 # -*- coding: utf-8 -*-
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
2 #
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
3 # Copyright (C) 2007 Edgewall Software
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
4 # All rights reserved.
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
5 #
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
6 # This software is licensed as described in the file COPYING, which
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
7 # you should have received as part of this distribution. The terms
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
8 # are also available at http://babel.edgewall.org/wiki/License.
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
9 #
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
10 # This software consists of voluntary contributions made by many
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
11 # individuals. For the exact contribution history, see the revision
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
12 # history and logs, available at http://babel.edgewall.org/log/.
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
13
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
14 """Reading and writing of files in the ``gettext`` PO (portable object)
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
15 format.
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
16
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
17 :see: `The Format of PO Files
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
18 <http://www.gnu.org/software/gettext/manual/gettext.html#PO-Files>`_
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
19 """
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
20
5
50ad95bee876 * The creation-date header in generated PO files now includes the timezone offset.
cmlenz
parents: 1
diff changeset
21 from datetime import date, datetime
134
60565dc8495d More fixes for Windows compatibility:
cmlenz
parents: 120
diff changeset
22 import os
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
23 import re
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
24 try:
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
25 set
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
26 except NameError:
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
27 from sets import Set as set
103
7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14.
cmlenz
parents: 102
diff changeset
28 from textwrap import wrap
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
29
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
30 from babel import __version__ as VERSION
56
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
31 from babel.messages.catalog import Catalog
97
a02952b73cf1 Renamed `LOCAL` to `LOCALTZ`.
cmlenz
parents: 96
diff changeset
32 from babel.util import LOCALTZ
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
33
158
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
34 __all__ = ['unescape', 'denormalize', 'read_po', 'escape', 'normalize',
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
35 'write_po']
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
36
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
37 def unescape(string):
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
38 r"""Reverse `escape` the given string.
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
39
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
40 >>> print unescape('"Say:\\n \\"hello, world!\\"\\n"')
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
41 Say:
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
42 "hello, world!"
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
43 <BLANKLINE>
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
44
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
45 :param string: the string to unescape
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
46 :return: the unescaped string
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
47 :rtype: `str` or `unicode`
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
48 """
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
49 return string[1:-1].replace('\\\\', '\\') \
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
50 .replace('\\t', '\t') \
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
51 .replace('\\r', '\r') \
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
52 .replace('\\n', '\n') \
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
53 .replace('\\"', '\"')
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
54
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
55 def denormalize(string):
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
56 r"""Reverse the normalization done by the `normalize` function.
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
57
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
58 >>> print denormalize(r'''""
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
59 ... "Say:\n"
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
60 ... " \"hello, world!\"\n"''')
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
61 Say:
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
62 "hello, world!"
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
63 <BLANKLINE>
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
64
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
65 >>> print denormalize(r'''""
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
66 ... "Say:\n"
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
67 ... " \"Lorem ipsum dolor sit "
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
68 ... "amet, consectetur adipisicing"
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
69 ... " elit, \"\n"''')
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
70 Say:
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
71 "Lorem ipsum dolor sit amet, consectetur adipisicing elit, "
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
72 <BLANKLINE>
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
73
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
74 :param string: the string to denormalize
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
75 :return: the denormalized string
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
76 :rtype: `unicode` or `str`
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
77 """
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
78 if string.startswith('""'):
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
79 lines = []
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
80 for line in string.splitlines()[1:]:
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
81 lines.append(unescape(line))
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
82 return ''.join(lines)
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
83 else:
17dd31f104f5 Minor cleanup in the `pofile` module.
cmlenz
parents: 149
diff changeset
84 return unescape(string)
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
85
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
86 def read_po(fileobj):
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
87 """Read messages from a ``gettext`` PO (portable object) file from the given
64
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
88 file-like object and return a `Catalog`.
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
89
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
90 >>> from StringIO import StringIO
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
91 >>> buf = StringIO('''
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
92 ... #: main.py:1
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
93 ... #, fuzzy, python-format
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
94 ... msgid "foo %(name)s"
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
95 ... msgstr ""
21
ddfac856c34f Change pot header's first line, "Translations Template for %%(project)s." instead of "SOME DESCRIPTIVE TITLE.". '''`project`''' and '''`version`''' now default to '''PROJECT''' and '''VERSION''' respectively. Fixed a bug regarding '''Content-Transfer-Encoding''', it shouldn't be the charset, and we're defaulting to `8bit` untill someone complains.
palgarvio
parents: 17
diff changeset
96 ...
94
b176f325d127 Updated `read_po` to add user comments besides just auto comments.
palgarvio
parents: 84
diff changeset
97 ... # A user comment
b176f325d127 Updated `read_po` to add user comments besides just auto comments.
palgarvio
parents: 84
diff changeset
98 ... #. An auto comment
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
99 ... #: main.py:3
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
100 ... msgid "bar"
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
101 ... msgid_plural "baz"
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
102 ... msgstr[0] ""
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
103 ... msgstr[1] ""
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
104 ... ''')
64
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
105 >>> catalog = read_po(buf)
104
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction.
cmlenz
parents: 103
diff changeset
106 >>> catalog.revision_date = datetime(2007, 04, 01)
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction.
cmlenz
parents: 103
diff changeset
107
64
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
108 >>> for message in catalog:
67
5496b9127a07 Enhance catalog to also manage the MIME headers.
cmlenz
parents: 64
diff changeset
109 ... if message.id:
5496b9127a07 Enhance catalog to also manage the MIME headers.
cmlenz
parents: 64
diff changeset
110 ... print (message.id, message.string)
105
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
111 ... print ' ', (message.locations, message.flags)
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
112 ... print ' ', (message.user_comments, message.auto_comments)
149
ba5150e9544e Respect charset specified in PO headers in `read_po()`. Fixes #17.
cmlenz
parents: 134
diff changeset
113 (u'foo %(name)s', '')
ba5150e9544e Respect charset specified in PO headers in `read_po()`. Fixes #17.
cmlenz
parents: 134
diff changeset
114 ([(u'main.py', 1)], set([u'fuzzy', u'python-format']))
105
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
115 ([], [])
149
ba5150e9544e Respect charset specified in PO headers in `read_po()`. Fixes #17.
cmlenz
parents: 134
diff changeset
116 ((u'bar', u'baz'), ('', ''))
ba5150e9544e Respect charset specified in PO headers in `read_po()`. Fixes #17.
cmlenz
parents: 134
diff changeset
117 ([(u'main.py', 3)], set([]))
ba5150e9544e Respect charset specified in PO headers in `read_po()`. Fixes #17.
cmlenz
parents: 134
diff changeset
118 ([u'A user comment'], [u'An auto comment'])
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
119
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
120 :param fileobj: the file-like object to read the PO file from
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
121 :return: an iterator over ``(message, translation, location)`` tuples
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
122 :rtype: ``iterator``
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
123 """
64
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
124 catalog = Catalog()
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
125
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
126 messages = []
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
127 translations = []
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
128 locations = []
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
129 flags = []
105
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
130 user_comments = []
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
131 auto_comments = []
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
132 in_msgid = in_msgstr = False
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
133
64
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
134 def _add_message():
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
135 translations.sort()
64
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
136 if len(messages) > 1:
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
137 msgid = tuple([denormalize(m) for m in messages])
64
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
138 else:
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
139 msgid = denormalize(messages[0])
64
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
140 if len(translations) > 1:
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
141 string = tuple([denormalize(t[1]) for t in translations])
64
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
142 else:
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
143 string = denormalize(translations[0][1])
105
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
144 catalog.add(msgid, string, list(locations), set(flags),
108
9a2c3d76fce9 Fixed a bug introduced in [106].
palgarvio
parents: 106
diff changeset
145 list(auto_comments), list(user_comments))
84
4ff9cc26c11b Some cosmetic changes for the new translator comments support.
cmlenz
parents: 80
diff changeset
146 del messages[:]; del translations[:]; del locations[:];
105
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
147 del flags[:]; del auto_comments[:]; del user_comments[:]
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
148
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
149 for line in fileobj.readlines():
149
ba5150e9544e Respect charset specified in PO headers in `read_po()`. Fixes #17.
cmlenz
parents: 134
diff changeset
150 line = line.strip().decode(catalog.charset)
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
151 if line.startswith('#'):
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
152 in_msgid = in_msgstr = False
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
153 if messages:
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
154 _add_message()
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
155 if line[1:].startswith(':'):
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
156 for location in line[2:].lstrip().split():
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
157 filename, lineno = location.split(':', 1)
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
158 locations.append((filename, int(lineno)))
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
159 elif line[1:].startswith(','):
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
160 for flag in line[2:].lstrip().split(','):
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
161 flags.append(flag.strip())
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
162 elif line[1:].startswith('.'):
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
163 # These are called auto-comments
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
164 comment = line[2:].strip()
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
165 if comment:
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
166 # Just check that we're not adding empty comments
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
167 auto_comments.append(comment)
120
733cca7ff6a5 Added tests for `new_catalog` distutils command.
cmlenz
parents: 108
diff changeset
168 else:
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
169 # These are called user comments
120
733cca7ff6a5 Added tests for `new_catalog` distutils command.
cmlenz
parents: 108
diff changeset
170 user_comments.append(line[1:].strip())
104
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction.
cmlenz
parents: 103
diff changeset
171 else:
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
172 if line.startswith('msgid_plural'):
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
173 in_msgid = True
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
174 msg = line[12:].lstrip()
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
175 messages.append(msg)
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
176 elif line.startswith('msgid'):
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
177 in_msgid = True
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
178 if messages:
64
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
179 _add_message()
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
180 messages.append(line[5:].lstrip())
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
181 elif line.startswith('msgstr'):
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
182 in_msgid = False
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
183 in_msgstr = True
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
184 msg = line[6:].lstrip()
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
185 if msg.startswith('['):
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
186 idx, msg = msg[1:].split(']')
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
187 translations.append([int(idx), msg.lstrip()])
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
188 else:
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
189 translations.append([0, msg])
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
190 elif line.startswith('"'):
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
191 if in_msgid:
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
192 messages[-1] += u'\n' + line.rstrip()
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
193 elif in_msgstr:
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
194 translations[-1][1] += u'\n' + line.rstrip()
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
195
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
196 if messages:
64
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
197 _add_message()
0406c51c5463 `read_po` now returns a `Catalog`.
cmlenz
parents: 56
diff changeset
198 return catalog
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
199
24
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
200 WORD_SEP = re.compile('('
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
201 r'\s+|' # any whitespace
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
202 r'[^\s\w]*\w+[a-zA-Z]-(?=\w+[a-zA-Z])|' # hyphenated words
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
203 r'(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w)' # em-dash
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
204 ')')
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
205
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
206 def escape(string):
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
207 r"""Escape the given string so that it can be included in double-quoted
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
208 strings in ``PO`` files.
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
209
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
210 >>> escape('''Say:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
211 ... "hello, world!"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
212 ... ''')
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
213 '"Say:\\n \\"hello, world!\\"\\n"'
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
214
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
215 :param string: the string to escape
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
216 :return: the escaped string
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
217 :rtype: `str` or `unicode`
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
218 """
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
219 return '"%s"' % string.replace('\\', '\\\\') \
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
220 .replace('\t', '\\t') \
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
221 .replace('\r', '\\r') \
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
222 .replace('\n', '\\n') \
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
223 .replace('\"', '\\"')
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
224
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
225 def normalize(string, width=76):
106
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`.
cmlenz
parents: 105
diff changeset
226 r"""Convert a string into a format that is appropriate for .po files.
24
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
227
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
228 >>> print normalize('''Say:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
229 ... "hello, world!"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
230 ... ''', width=None)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
231 ""
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
232 "Say:\n"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
233 " \"hello, world!\"\n"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
234
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
235 >>> print normalize('''Say:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
236 ... "Lorem ipsum dolor sit amet, consectetur adipisicing elit, "
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
237 ... ''', width=32)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
238 ""
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
239 "Say:\n"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
240 " \"Lorem ipsum dolor sit "
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
241 "amet, consectetur adipisicing"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
242 " elit, \"\n"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
243
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
244 :param string: the string to normalize
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
245 :param width: the maximum line width; use `None`, 0, or a negative number
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
246 to completely disable line wrapping
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
247 :return: the normalized string
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
248 :rtype: `unicode`
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
249 """
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
250 if width and width > 0:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
251 lines = []
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
252 for idx, line in enumerate(string.splitlines(True)):
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
253 if len(escape(line)) > width:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
254 chunks = WORD_SEP.split(line)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
255 chunks.reverse()
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
256 while chunks:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
257 buf = []
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
258 size = 2
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
259 while chunks:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
260 l = len(escape(chunks[-1])) - 2
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
261 if size + l < width:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
262 buf.append(chunks.pop())
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
263 size += l
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
264 else:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
265 if not buf:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
266 # handle long chunks by putting them on a
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
267 # separate line
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
268 buf.append(chunks.pop())
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
269 break
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
270 lines.append(u''.join(buf))
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
271 else:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
272 lines.append(line)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
273 else:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
274 lines = string.splitlines(True)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
275
67
5496b9127a07 Enhance catalog to also manage the MIME headers.
cmlenz
parents: 64
diff changeset
276 if len(lines) <= 1:
24
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
277 return escape(string)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
278
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
279 # Remove empty trailing line
67
5496b9127a07 Enhance catalog to also manage the MIME headers.
cmlenz
parents: 64
diff changeset
280 if lines and not lines[-1]:
24
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
281 del lines[-1]
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
282 lines[-1] += '\n'
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
283 return u'""\n' + u'\n'.join([escape(l) for l in lines])
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
284
104
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction.
cmlenz
parents: 103
diff changeset
285 def write_po(fileobj, catalog, width=76, no_location=False, omit_header=False,
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction.
cmlenz
parents: 103
diff changeset
286 sort_output=False, sort_by_file=False):
56
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
287 r"""Write a ``gettext`` PO (portable object) template file for a given
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
288 message catalog to the provided file-like object.
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
289
56
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
290 >>> catalog = Catalog()
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
291 >>> catalog.add(u'foo %(name)s', locations=[('main.py', 1)],
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
292 ... flags=('fuzzy',))
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
293 >>> catalog.add((u'bar', u'baz'), locations=[('main.py', 3)])
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
294 >>> from StringIO import StringIO
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
295 >>> buf = StringIO()
104
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction.
cmlenz
parents: 103
diff changeset
296 >>> write_po(buf, catalog, omit_header=True)
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
297 >>> print buf.getvalue()
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
298 #: main.py:1
6
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
299 #, fuzzy, python-format
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy").
cmlenz
parents: 5
diff changeset
300 msgid "foo %(name)s"
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
301 msgstr ""
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
302 <BLANKLINE>
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
303 #: main.py:3
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
304 msgid "bar"
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
305 msgid_plural "baz"
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
306 msgstr[0] ""
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
307 msgstr[1] ""
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
308 <BLANKLINE>
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
309 <BLANKLINE>
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
310
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
311 :param fileobj: the file-like object to write to
67
5496b9127a07 Enhance catalog to also manage the MIME headers.
cmlenz
parents: 64
diff changeset
312 :param catalog: the `Catalog` instance
24
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
313 :param width: the maximum line width for the generated output; use `None`,
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
314 0, or a negative number to completely disable line wrapping
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
315 :param no_location: do not emit a location comment for every message
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
316 :param omit_header: do not include the ``msgid ""`` entry at the top of the
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
317 output
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
318 """
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
319 def _normalize(key):
102
eb0d9591d555 Project name and version, and the charset are available via the `Catalog` object, and do not need to be passed to `write_pot()`.
cmlenz
parents: 97
diff changeset
320 return normalize(key, width=width).encode(catalog.charset,
eb0d9591d555 Project name and version, and the charset are available via the `Catalog` object, and do not need to be passed to `write_pot()`.
cmlenz
parents: 97
diff changeset
321 'backslashreplace')
24
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
322
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
323 def _write(text):
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
324 if isinstance(text, unicode):
102
eb0d9591d555 Project name and version, and the charset are available via the `Catalog` object, and do not need to be passed to `write_pot()`.
cmlenz
parents: 97
diff changeset
325 text = text.encode(catalog.charset)
24
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
326 fileobj.write(text)
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
327
104
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction.
cmlenz
parents: 103
diff changeset
328 messages = list(catalog)
71
ea4cb904df8f Implemented message sorting, see #7.
palgarvio
parents: 68
diff changeset
329 if sort_output:
ea4cb904df8f Implemented message sorting, see #7.
palgarvio
parents: 68
diff changeset
330 messages.sort(lambda x,y: cmp(x.id, y.id))
ea4cb904df8f Implemented message sorting, see #7.
palgarvio
parents: 68
diff changeset
331 elif sort_by_file:
ea4cb904df8f Implemented message sorting, see #7.
palgarvio
parents: 68
diff changeset
332 messages.sort(lambda x,y: cmp(x.locations, y.locations))
68
7e64668126d9 Add back POT header broken in previous check-in.
cmlenz
parents: 67
diff changeset
333
71
ea4cb904df8f Implemented message sorting, see #7.
palgarvio
parents: 68
diff changeset
334 for message in messages:
67
5496b9127a07 Enhance catalog to also manage the MIME headers.
cmlenz
parents: 64
diff changeset
335 if not message.id: # This is the header "message"
5496b9127a07 Enhance catalog to also manage the MIME headers.
cmlenz
parents: 64
diff changeset
336 if omit_header:
5496b9127a07 Enhance catalog to also manage the MIME headers.
cmlenz
parents: 64
diff changeset
337 continue
104
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction.
cmlenz
parents: 103
diff changeset
338 comment_header = catalog.header_comment
103
7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14.
cmlenz
parents: 102
diff changeset
339 if width and width > 0:
7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14.
cmlenz
parents: 102
diff changeset
340 lines = []
104
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction.
cmlenz
parents: 103
diff changeset
341 for line in comment_header.splitlines():
103
7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14.
cmlenz
parents: 102
diff changeset
342 lines += wrap(line, width=width, subsequent_indent='# ',
7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14.
cmlenz
parents: 102
diff changeset
343 break_long_words=False)
104
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction.
cmlenz
parents: 103
diff changeset
344 comment_header = u'\n'.join(lines) + u'\n'
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction.
cmlenz
parents: 103
diff changeset
345 _write(comment_header)
102
eb0d9591d555 Project name and version, and the charset are available via the `Catalog` object, and do not need to be passed to `write_pot()`.
cmlenz
parents: 97
diff changeset
346
105
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
347 if message.user_comments:
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
348 for comment in message.user_comments:
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
349 for line in wrap(comment, width, break_long_words=False):
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
350 _write('# %s\n' % line.strip())
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
351
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
352 if message.auto_comments:
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly.
palgarvio
parents: 104
diff changeset
353 for comment in message.auto_comments:
103
7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14.
cmlenz
parents: 102
diff changeset
354 for line in wrap(comment, width, break_long_words=False):
80
9c84b9fa5d30 Added support for translator comments at the API and frontends levels.(See #12, item 1). Updated docs and tests accordingly.
palgarvio
parents: 79
diff changeset
355 _write('#. %s\n' % line.strip())
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
356
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
357 if not no_location:
134
60565dc8495d More fixes for Windows compatibility:
cmlenz
parents: 120
diff changeset
358 locs = u' '.join([u'%s:%d' % (filename.replace(os.sep, '/'), lineno)
60565dc8495d More fixes for Windows compatibility:
cmlenz
parents: 120
diff changeset
359 for filename, lineno in message.locations])
24
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
360 if width and width > 0:
103
7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14.
cmlenz
parents: 102
diff changeset
361 locs = wrap(locs, width, break_long_words=False)
24
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
362 for line in locs:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
363 _write('#: %s\n' % line.strip())
56
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
364 if message.flags:
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
365 _write('#%s\n' % ', '.join([''] + list(message.flags)))
24
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
366
56
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
367 if isinstance(message.id, (list, tuple)):
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
368 _write('msgid %s\n' % _normalize(message.id[0]))
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
369 _write('msgid_plural %s\n' % _normalize(message.id[1]))
68
7e64668126d9 Add back POT header broken in previous check-in.
cmlenz
parents: 67
diff changeset
370 for i, string in enumerate(message.string):
7e64668126d9 Add back POT header broken in previous check-in.
cmlenz
parents: 67
diff changeset
371 _write('msgstr[%d] %s\n' % (i, _normalize(message.string[i])))
1
f71ca60f2a4a Import of initial code base.
cmlenz
parents:
diff changeset
372 else:
56
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends.
cmlenz
parents: 55
diff changeset
373 _write('msgid %s\n' % _normalize(message.id))
68
7e64668126d9 Add back POT header broken in previous check-in.
cmlenz
parents: 67
diff changeset
374 _write('msgstr %s\n' % _normalize(message.string or ''))
24
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks).
cmlenz
parents: 23
diff changeset
375 _write('\n')
Copyright (C) 2012-2017 Edgewall Software