babel/old/babel-test: babel/messages/pofile.py annotate

annotate babel/messages/pofile.py @ 120:733cca7ff6a5

Added tests for `new_catalog` distutils command.

author	cmlenz
date	Fri, 15 Jun 2007 22:18:59 +0000
parents	9a2c3d76fce9
children	60565dc8495d

rev	line source
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	1 # -- coding: utf-8 --
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	2 #
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	3 # Copyright (C) 2007 Edgewall Software
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	4 # All rights reserved.
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	5 #
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	6 # This software is licensed as described in the file COPYING, which
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	7 # you should have received as part of this distribution. The terms
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	8 # are also available at http://babel.edgewall.org/wiki/License.
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	9 #
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	10 # This software consists of voluntary contributions made by many
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	11 # individuals. For the exact contribution history, see the revision
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	12 # history and logs, available at http://babel.edgewall.org/log/.
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	13
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	14 """Reading and writing of files in the ``gettext`` PO (portable object)
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	15 format.
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	16
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	17 :see: `The Format of PO Files
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	18 <http://www.gnu.org/software/gettext/manual/gettext.html#PO-Files>`_
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	19 """
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	20
5 50ad95bee876 * The creation-date header in generated PO files now includes the timezone offset. cmlenz parents: 1 diff changeset	21 from datetime import date, datetime
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	22 import re
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	23 try:
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	24 set
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	25 except NameError:
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	26 from sets import Set as set
103 7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14. cmlenz parents: 102 diff changeset	27 from textwrap import wrap
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	28
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	29 from babel import __version__ as VERSION
56 27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	30 from babel.messages.catalog import Catalog
97 a02952b73cf1 Renamed `LOCAL` to `LOCALTZ`. cmlenz parents: 96 diff changeset	31 from babel.util import LOCALTZ
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	32
104 22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	33 __all__ = ['escape', 'normalize', 'read_po', 'write_po']
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	34
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	35 def read_po(fileobj):
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	36 """Read messages from a ``gettext`` PO (portable object) file from the given
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	37 file-like object and return a `Catalog`.
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	38
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	39 >>> from StringIO import StringIO
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	40 >>> buf = StringIO('''
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	41 ... #: main.py:1
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	42 ... #, fuzzy, python-format
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	43 ... msgid "foo %(name)s"
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	44 ... msgstr ""
21 ddfac856c34f Change pot header's first line, "Translations Template for %%(project)s." instead of "SOME DESCRIPTIVE TITLE.". '''`project`''' and '''`version`''' now default to '''PROJECT''' and '''VERSION''' respectively. Fixed a bug regarding '''Content-Transfer-Encoding''', it shouldn't be the charset, and we're defaulting to `8bit` untill someone complains. palgarvio parents: 17 diff changeset	45 ...
94 b176f325d127 Updated `read_po` to add user comments besides just auto comments. palgarvio parents: 84 diff changeset	46 ... # A user comment
b176f325d127 Updated `read_po` to add user comments besides just auto comments. palgarvio parents: 84 diff changeset	47 ... #. An auto comment
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	48 ... #: main.py:3
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	49 ... msgid "bar"
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	50 ... msgid_plural "baz"
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	51 ... msgstr[0] ""
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	52 ... msgstr[1] ""
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	53 ... ''')
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	54 >>> catalog = read_po(buf)
104 22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	55 >>> catalog.revision_date = datetime(2007, 04, 01)
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	56
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	57 >>> for message in catalog:
67 5496b9127a07 Enhance catalog to also manage the MIME headers. cmlenz parents: 64 diff changeset	58 ... if message.id:
5496b9127a07 Enhance catalog to also manage the MIME headers. cmlenz parents: 64 diff changeset	59 ... print (message.id, message.string)
105 f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	60 ... print ' ', (message.locations, message.flags)
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	61 ... print ' ', (message.user_comments, message.auto_comments)
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	62 ('foo %(name)s', '')
105 f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	63 ([('main.py', 1)], set(['fuzzy', 'python-format']))
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	64 ([], [])
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	65 (('bar', 'baz'), ('', ''))
105 f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	66 ([('main.py', 3)], set([]))
108 9a2c3d76fce9 Fixed a bug introduced in [106]. palgarvio parents: 106 diff changeset	67 (['A user comment'], ['An auto comment'])
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	68
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	69 :param fileobj: the file-like object to read the PO file from
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	70 :return: an iterator over ``(message, translation, location)`` tuples
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	71 :rtype: ``iterator``
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	72 """
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	73 catalog = Catalog()
0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	74
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	75 messages = []
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	76 translations = []
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	77 locations = []
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	78 flags = []
105 f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	79 user_comments = []
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	80 auto_comments = []
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	81 in_msgid = in_msgstr = False
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	82
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	83 def _add_message():
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	84 translations.sort()
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	85 if len(messages) > 1:
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	86 msgid = tuple([denormalize(m) for m in messages])
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	87 else:
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	88 msgid = denormalize(messages[0])
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	89 if len(translations) > 1:
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	90 string = tuple([denormalize(t[1]) for t in translations])
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	91 else:
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	92 string = denormalize(translations[0][1])
105 f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	93 catalog.add(msgid, string, list(locations), set(flags),
108 9a2c3d76fce9 Fixed a bug introduced in [106]. palgarvio parents: 106 diff changeset	94 list(auto_comments), list(user_comments))
84 4ff9cc26c11b Some cosmetic changes for the new translator comments support. cmlenz parents: 80 diff changeset	95 del messages[:]; del translations[:]; del locations[:];
105 f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	96 del flags[:]; del auto_comments[:]; del user_comments[:]
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	97
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	98 for line in fileobj.readlines():
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	99 line = line.strip()
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	100 if line.startswith('#'):
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	101 in_msgid = in_msgstr = False
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	102 if messages:
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	103 _add_message()
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	104 if line[1:].startswith(':'):
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	105 for location in line[2:].lstrip().split():
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	106 filename, lineno = location.split(':', 1)
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	107 locations.append((filename, int(lineno)))
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	108 elif line[1:].startswith(','):
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	109 for flag in line[2:].lstrip().split(','):
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	110 flags.append(flag.strip())
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	111 elif line[1:].startswith('.'):
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	112 # These are called auto-comments
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	113 comment = line[2:].strip()
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	114 if comment:
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	115 # Just check that we're not adding empty comments
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	116 auto_comments.append(comment)
120 733cca7ff6a5 Added tests for `new_catalog` distutils command. cmlenz parents: 108 diff changeset	117 else:
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	118 # These are called user comments
120 733cca7ff6a5 Added tests for `new_catalog` distutils command. cmlenz parents: 108 diff changeset	119 user_comments.append(line[1:].strip())
104 22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	120 else:
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	121 if line.startswith('msgid_plural'):
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	122 in_msgid = True
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	123 msg = line[12:].lstrip()
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	124 messages.append(msg)
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	125 elif line.startswith('msgid'):
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	126 in_msgid = True
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	127 if messages:
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	128 _add_message()
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	129 messages.append(line[5:].lstrip())
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	130 elif line.startswith('msgstr'):
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	131 in_msgid = False
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	132 in_msgstr = True
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	133 msg = line[6:].lstrip()
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	134 if msg.startswith('['):
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	135 idx, msg = msg[1:].split(']')
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	136 translations.append([int(idx), msg.lstrip()])
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	137 else:
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	138 translations.append([0, msg])
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	139 elif line.startswith('"'):
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	140 if in_msgid:
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	141 messages[-1] += u'\n' + line.rstrip()
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	142 elif in_msgstr:
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	143 translations[-1][1] += u'\n' + line.rstrip()
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	144
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	145 if messages:
64 0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	146 _add_message()
0406c51c5463 `read_po` now returns a `Catalog`. cmlenz parents: 56 diff changeset	147 return catalog
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	148
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	149 WORD_SEP = re.compile('('
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	150 r'\s+\|' # any whitespace
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	151 r'[^\s\w]*\w+[a-zA-Z]-(?=\w+[a-zA-Z])\|' # hyphenated words
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	152 r'(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w)' # em-dash
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	153 ')')
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	154
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	155 def escape(string):
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	156 r"""Escape the given string so that it can be included in double-quoted
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	157 strings in ``PO`` files.
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	158
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	159 >>> escape('''Say:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	160 ... "hello, world!"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	161 ... ''')
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	162 '"Say:\\n \\"hello, world!\\"\\n"'
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	163
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	164 :param string: the string to escape
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	165 :return: the escaped string
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	166 :rtype: `str` or `unicode`
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	167 """
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	168 return '"%s"' % string.replace('\\', '\\\\') \
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	169 .replace('\t', '\\t') \
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	170 .replace('\r', '\\r') \
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	171 .replace('\n', '\\n') \
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	172 .replace('\"', '\\"')
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	173
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	174 def unescape(string):
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	175 r"""Reverse escape the given string.
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	176
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	177 >>> print unescape('"Say:\\n \\"hello, world!\\"\\n"')
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	178 Say:
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	179 "hello, world!"
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	180 <BLANKLINE>
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	181
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	182 :param string: the string to unescape
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	183 :return: the unescaped string
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	184 :rtype: `str` or `unicode`
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	185 """
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	186 return string[1:-1].replace('\\\\', '\\') \
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	187 .replace('\\t', '\t') \
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	188 .replace('\\r', '\r') \
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	189 .replace('\\n', '\n') \
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	190 .replace('\\"', '\"')
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	191
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	192 def normalize(string, width=76):
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	193 r"""Convert a string into a format that is appropriate for .po files.
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	194
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	195 >>> print normalize('''Say:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	196 ... "hello, world!"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	197 ... ''', width=None)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	198 ""
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	199 "Say:\n"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	200 " \"hello, world!\"\n"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	201
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	202 >>> print normalize('''Say:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	203 ... "Lorem ipsum dolor sit amet, consectetur adipisicing elit, "
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	204 ... ''', width=32)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	205 ""
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	206 "Say:\n"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	207 " \"Lorem ipsum dolor sit "
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	208 "amet, consectetur adipisicing"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	209 " elit, \"\n"
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	210
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	211 :param string: the string to normalize
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	212 :param width: the maximum line width; use `None`, 0, or a negative number
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	213 to completely disable line wrapping
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	214 :return: the normalized string
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	215 :rtype: `unicode`
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	216 """
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	217 if width and width > 0:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	218 lines = []
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	219 for idx, line in enumerate(string.splitlines(True)):
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	220 if len(escape(line)) > width:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	221 chunks = WORD_SEP.split(line)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	222 chunks.reverse()
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	223 while chunks:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	224 buf = []
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	225 size = 2
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	226 while chunks:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	227 l = len(escape(chunks[-1])) - 2
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	228 if size + l < width:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	229 buf.append(chunks.pop())
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	230 size += l
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	231 else:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	232 if not buf:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	233 # handle long chunks by putting them on a
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	234 # separate line
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	235 buf.append(chunks.pop())
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	236 break
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	237 lines.append(u''.join(buf))
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	238 else:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	239 lines.append(line)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	240 else:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	241 lines = string.splitlines(True)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	242
67 5496b9127a07 Enhance catalog to also manage the MIME headers. cmlenz parents: 64 diff changeset	243 if len(lines) <= 1:
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	244 return escape(string)
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	245
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	246 # Remove empty trailing line
67 5496b9127a07 Enhance catalog to also manage the MIME headers. cmlenz parents: 64 diff changeset	247 if lines and not lines[-1]:
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	248 del lines[-1]
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	249 lines[-1] += '\n'
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	250 return u'""\n' + u'\n'.join([escape(l) for l in lines])
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	251
106 9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	252 def denormalize(string):
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	253 r"""Reverse the normalization done by the `normalize` function.
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	254
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	255 >>> print denormalize(r'''""
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	256 ... "Say:\n"
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	257 ... " \"hello, world!\"\n"''')
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	258 Say:
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	259 "hello, world!"
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	260 <BLANKLINE>
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	261
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	262 >>> print denormalize(r'''""
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	263 ... "Say:\n"
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	264 ... " \"Lorem ipsum dolor sit "
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	265 ... "amet, consectetur adipisicing"
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	266 ... " elit, \"\n"''')
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	267 Say:
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	268 "Lorem ipsum dolor sit amet, consectetur adipisicing elit, "
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	269 <BLANKLINE>
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	270
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	271 :param string: the string to denormalize
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	272 :return: the denormalized string
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	273 :rtype: `unicode` or `str`
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	274 """
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	275 if string.startswith('""'):
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	276 lines = []
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	277 for line in string.splitlines()[1:]:
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	278 lines.append(unescape(line))
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	279 return ''.join(lines)
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	280 else:
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	281 return unescape(string)
9b22b36066f6 Fix for #16: the header message (`msgid = ""`) is now treated specially by `read_po` and `Catalog`. cmlenz parents: 105 diff changeset	282
104 22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	283 def write_po(fileobj, catalog, width=76, no_location=False, omit_header=False,
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	284 sort_output=False, sort_by_file=False):
56 27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	285 r"""Write a ``gettext`` PO (portable object) template file for a given
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	286 message catalog to the provided file-like object.
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	287
56 27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	288 >>> catalog = Catalog()
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	289 >>> catalog.add(u'foo %(name)s', locations=[('main.py', 1)],
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	290 ... flags=('fuzzy',))
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	291 >>> catalog.add((u'bar', u'baz'), locations=[('main.py', 3)])
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	292 >>> from StringIO import StringIO
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	293 >>> buf = StringIO()
104 22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	294 >>> write_po(buf, catalog, omit_header=True)
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	295 >>> print buf.getvalue()
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	296 #: main.py:1
6 1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	297 #, fuzzy, python-format
1801bc2b60ca Add basic PO file parsing, and change the PO writing procedure to also take flags (such as "python-format" or "fuzzy"). cmlenz parents: 5 diff changeset	298 msgid "foo %(name)s"
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	299 msgstr ""
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	300 <BLANKLINE>
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	301 #: main.py:3
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	302 msgid "bar"
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	303 msgid_plural "baz"
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	304 msgstr[0] ""
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	305 msgstr[1] ""
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	306 <BLANKLINE>
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	307 <BLANKLINE>
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	308
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	309 :param fileobj: the file-like object to write to
67 5496b9127a07 Enhance catalog to also manage the MIME headers. cmlenz parents: 64 diff changeset	310 :param catalog: the `Catalog` instance
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	311 :param width: the maximum line width for the generated output; use `None`,
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	312 0, or a negative number to completely disable line wrapping
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	313 :param no_location: do not emit a location comment for every message
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	314 :param omit_header: do not include the ``msgid ""`` entry at the top of the
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	315 output
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	316 """
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	317 def _normalize(key):
102 eb0d9591d555 Project name and version, and the charset are available via the `Catalog` object, and do not need to be passed to `write_pot()`. cmlenz parents: 97 diff changeset	318 return normalize(key, width=width).encode(catalog.charset,
eb0d9591d555 Project name and version, and the charset are available via the `Catalog` object, and do not need to be passed to `write_pot()`. cmlenz parents: 97 diff changeset	319 'backslashreplace')
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	320
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	321 def _write(text):
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	322 if isinstance(text, unicode):
102 eb0d9591d555 Project name and version, and the charset are available via the `Catalog` object, and do not need to be passed to `write_pot()`. cmlenz parents: 97 diff changeset	323 text = text.encode(catalog.charset)
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	324 fileobj.write(text)
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	325
104 22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	326 messages = list(catalog)
71 ea4cb904df8f Implemented message sorting, see #7. palgarvio parents: 68 diff changeset	327 if sort_output:
ea4cb904df8f Implemented message sorting, see #7. palgarvio parents: 68 diff changeset	328 messages.sort(lambda x,y: cmp(x.id, y.id))
ea4cb904df8f Implemented message sorting, see #7. palgarvio parents: 68 diff changeset	329 elif sort_by_file:
ea4cb904df8f Implemented message sorting, see #7. palgarvio parents: 68 diff changeset	330 messages.sort(lambda x,y: cmp(x.locations, y.locations))
68 7e64668126d9 Add back POT header broken in previous check-in. cmlenz parents: 67 diff changeset	331
71 ea4cb904df8f Implemented message sorting, see #7. palgarvio parents: 68 diff changeset	332 for message in messages:
67 5496b9127a07 Enhance catalog to also manage the MIME headers. cmlenz parents: 64 diff changeset	333 if not message.id: # This is the header "message"
5496b9127a07 Enhance catalog to also manage the MIME headers. cmlenz parents: 64 diff changeset	334 if omit_header:
5496b9127a07 Enhance catalog to also manage the MIME headers. cmlenz parents: 64 diff changeset	335 continue
104 22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	336 comment_header = catalog.header_comment
103 7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14. cmlenz parents: 102 diff changeset	337 if width and width > 0:
7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14. cmlenz parents: 102 diff changeset	338 lines = []
104 22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	339 for line in comment_header.splitlines():
103 7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14. cmlenz parents: 102 diff changeset	340 lines += wrap(line, width=width, subsequent_indent='# ',
7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14. cmlenz parents: 102 diff changeset	341 break_long_words=False)
104 22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	342 comment_header = u'\n'.join(lines) + u'\n'
22f222e23b86 Merged `write_pot` and `write_po` functions by moving more functionality to the `Catalog` class. This is certainly not perfect yet, but moves us in the right direction. cmlenz parents: 103 diff changeset	343 _write(comment_header)
102 eb0d9591d555 Project name and version, and the charset are available via the `Catalog` object, and do not need to be passed to `write_pot()`. cmlenz parents: 97 diff changeset	344
105 f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	345 if message.user_comments:
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	346 for comment in message.user_comments:
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	347 for line in wrap(comment, width, break_long_words=False):
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	348 _write('# %s\n' % line.strip())
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	349
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	350 if message.auto_comments:
f744dd56573d `Message`, `read_po` and `write_po` now all handle user/auto comments correctly. palgarvio parents: 104 diff changeset	351 for comment in message.auto_comments:
103 7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14. cmlenz parents: 102 diff changeset	352 for line in wrap(comment, width, break_long_words=False):
80 9c84b9fa5d30 Added support for translator comments at the API and frontends levels.(See #12, item 1). Updated docs and tests accordingly. palgarvio parents: 79 diff changeset	353 _write('#. %s\n' % line.strip())
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	354
f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	355 if not no_location:
56 27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	356 locs = u' '.join([u'%s:%d' % item for item in message.locations])
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	357 if width and width > 0:
103 7cdf89eb9007 Implement wrapping of header comments in PO(T) output. Related to #14. cmlenz parents: 102 diff changeset	358 locs = wrap(locs, width, break_long_words=False)
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	359 for line in locs:
4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	360 _write('#: %s\n' % line.strip())
56 27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	361 if message.flags:
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	362 _write('#%s\n' % ', '.join([''] + list(message.flags)))
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	363
56 27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	364 if isinstance(message.id, (list, tuple)):
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	365 _write('msgid %s\n' % _normalize(message.id[0]))
27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	366 _write('msgid_plural %s\n' % _normalize(message.id[1]))
68 7e64668126d9 Add back POT header broken in previous check-in. cmlenz parents: 67 diff changeset	367 for i, string in enumerate(message.string):
7e64668126d9 Add back POT header broken in previous check-in. cmlenz parents: 67 diff changeset	368 _write('msgstr[%d] %s\n' % (i, _normalize(message.string[i])))
1 f71ca60f2a4a Import of initial code base. cmlenz parents: diff changeset	369 else:
56 27fba894d3ca Add actual data structures for handling message catalogs, so that more code can be reused here between the frontends. cmlenz parents: 55 diff changeset	370 _write('msgid %s\n' % _normalize(message.id))
68 7e64668126d9 Add back POT header broken in previous check-in. cmlenz parents: 67 diff changeset	371 _write('msgstr %s\n' % _normalize(message.string or ''))
24 4fad20ab7cca Reimplement line wrapping for PO writing (as the `textwrap` module is too destructive with white space) and move it to the `normalize` function (which was already doing some handling of line breaks). cmlenz parents: 23 diff changeset	372 _write('\n')

Mercurial > babel > old > babel-test

annotate babel/messages/pofile.py @ 120:733cca7ff6a5