# HG changeset patch # User cmlenz # Date 1201877762 0 # Node ID 274506f102ed6f40284f9f95f206f082dded21ca # Parent fc1fcf6b002c335c4239422e2126565d4981e12c Ported [346] and [347] to 0.9.x branch. diff --git a/ChangeLog b/ChangeLog --- a/ChangeLog +++ b/ChangeLog @@ -3,7 +3,11 @@ (?, from branches/stable/0.9.x) * Fixed catalogs' charset values not being recognized (ticket #66). - + * Fixed fuzzy matching when updating message catalogs (ticket #82). + * Fixed bug in catalog updating, that in some cases pulled in + translations from different catalogs based on the same template. + + Version 0.9.1 http://svn.edgewall.org/repos/babel/tags/0.9.1/ (Sep 7 2007, from branches/stable/0.9.x) diff --git a/babel/messages/catalog.py b/babel/messages/catalog.py --- a/babel/messages/catalog.py +++ b/babel/messages/catalog.py @@ -92,6 +92,11 @@ return cmp(self.id, obj.id[0]) return cmp(self.id, obj.id) + def clone(self): + return Message(self.id, self.string, self.locations, self.flags, + self.auto_comments, self.user_comments, + self.previous_id, self.lineno) + def fuzzy(self): return 'fuzzy' in self.flags fuzzy = property(fuzzy, doc="""\ @@ -231,11 +236,11 @@ >>> catalog = Catalog(project='Foobar', version='1.0', ... copyright_holder='Foo Company') - >>> print catalog.header_comment + >>> print catalog.header_comment #doctest: +ELLIPSIS # Translations template for Foobar. - # Copyright (C) 2007 Foo Company + # Copyright (C) ... Foo Company # This file is distributed under the same license as the Foobar project. - # FIRST AUTHOR , 2007. + # FIRST AUTHOR , .... # The header can also be set from a string. Any known upper-case variables @@ -611,17 +616,31 @@ :param no_fuzzy_matching: whether to use fuzzy matching of message IDs """ messages = self._messages + remaining = messages.copy() self._messages = odict() + # Prepare for fuzzy matching + fuzzy_candidates = [] + if not no_fuzzy_matching: + fuzzy_candidates = [ + self._key_for(msgid) for msgid in messages + if msgid and messages[msgid].string + ] + fuzzy_matches = set() + def _merge(message, oldkey, newkey): + message = message.clone() fuzzy = False - oldmsg = messages.pop(oldkey) if oldkey != newkey: fuzzy = True + fuzzy_matches.add(oldkey) + oldmsg = messages.get(oldkey) if isinstance(oldmsg.id, basestring): message.previous_id = [oldmsg.id] else: message.previous_id = list(oldmsg.id) + else: + oldmsg = remaining.pop(oldkey) message.string = oldmsg.string if isinstance(message.id, (list, tuple)): if not isinstance(message.string, (list, tuple)): @@ -649,14 +668,17 @@ if no_fuzzy_matching is False: # do some fuzzy matching with difflib matches = get_close_matches(key.lower().strip(), - [self._key_for(msgid) for msgid in messages], 1) + fuzzy_candidates, 1) if matches: _merge(message, matches[0], key) continue self[message.id] = message - self.obsolete = messages + self.obsolete = odict() + for msgid in remaining: + if no_fuzzy_matching or msgid not in fuzzy_matches: + self.obsolete[msgid] = remaining[msgid] def _key_for(self, id): """The key for a message is just the singular ID even for pluralizable diff --git a/babel/messages/tests/catalog.py b/babel/messages/tests/catalog.py --- a/babel/messages/tests/catalog.py +++ b/babel/messages/tests/catalog.py @@ -131,6 +131,40 @@ self.assertEqual('Voh', cat['foo'].string) self.assertEqual(True, cat['foo'].fuzzy) + def test_update_fuzzy_matching_no_msgstr(self): + cat = catalog.Catalog() + cat.add('fo', '') + tmpl = catalog.Catalog() + tmpl.add('fo') + tmpl.add('foo') + cat.update(tmpl) + assert 'fo' in cat + assert 'foo' in cat + + self.assertEqual('', cat['fo'].string) + self.assertEqual(False, cat['fo'].fuzzy) + self.assertEqual(None, cat['foo'].string) + self.assertEqual(False, cat['foo'].fuzzy) + + def test_update_fuzzy_matching_no_cascading(self): + cat = catalog.Catalog() + cat.add('fo', 'Voh') + cat.add('foo', 'Vohe') + tmpl = catalog.Catalog() + tmpl.add('fo') + tmpl.add('foo') + tmpl.add('fooo') + cat.update(tmpl) + assert 'fo' in cat + assert 'foo' in cat + + self.assertEqual('Voh', cat['fo'].string) + self.assertEqual(False, cat['fo'].fuzzy) + self.assertEqual('Vohe', cat['foo'].string) + self.assertEqual(False, cat['foo'].fuzzy) + self.assertEqual('Vohe', cat['fooo'].string) + self.assertEqual(True, cat['fooo'].fuzzy) + def test_update_without_fuzzy_matching(self): cat = catalog.Catalog() cat.add('fo', 'Voh') @@ -140,6 +174,17 @@ cat.update(tmpl, no_fuzzy_matching=True) self.assertEqual(2, len(cat.obsolete)) + def test_update_no_template_mutation(self): + tmpl = catalog.Catalog() + tmpl.add('foo') + cat1 = catalog.Catalog() + cat1.add('foo', 'Voh') + cat1.update(tmpl) + cat2 = catalog.Catalog() + cat2.update(tmpl) + + self.assertEqual(None, cat2['foo'].string) + self.assertEqual(False, cat2['foo'].fuzzy) def suite(): suite = unittest.TestSuite()