annotate babel/localedata.py @ 530:85e1beadacb0

Update the copyright line.
author jruigrok
date Sat, 05 Mar 2011 15:22:28 +0000
parents b1c8cc751458
children b33c36615fe9
rev   line source
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
1 # -*- coding: utf-8 -*-
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
2 #
530
85e1beadacb0 Update the copyright line.
jruigrok
parents: 440
diff changeset
3 # Copyright (C) 2007-2011 Edgewall Software
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
4 # All rights reserved.
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
5 #
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
6 # This software is licensed as described in the file COPYING, which
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
7 # you should have received as part of this distribution. The terms
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
8 # are also available at http://babel.edgewall.org/wiki/License.
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
9 #
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
10 # This software consists of voluntary contributions made by many
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
11 # individuals. For the exact contribution history, see the revision
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
12 # history and logs, available at http://babel.edgewall.org/log/.
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
13
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
14 """Low-level locale data access.
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
15
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
16 :note: The `Locale` class, which uses this module under the hood, provides a
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
17 more convenient interface for accessing the locale data.
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
18 """
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
19
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
20 import os
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
21 import pickle
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
22 try:
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
23 import threading
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
24 except ImportError:
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
25 import dummy_threading as threading
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
26 from UserDict import DictMixin
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
27
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
28 __all__ = ['exists', 'list', 'load']
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
29 __docformat__ = 'restructuredtext en'
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
30
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
31 _cache = {}
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
32 _cache_lock = threading.RLock()
41
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
33 _dirname = os.path.join(os.path.dirname(__file__), 'localedata')
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
34
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
35
41
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
36 def exists(name):
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
37 """Check whether locale data is available for the given locale.
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
38
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
39 :param name: the locale identifier string
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
40 :return: `True` if the locale data exists, `False` otherwise
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
41 :rtype: `bool`
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
42 """
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
43 if name in _cache:
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
44 return True
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
45 return os.path.exists(os.path.join(_dirname, '%s.dat' % name))
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
46
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
47
185
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
48 def list():
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
49 """Return a list of all locale identifiers for which locale data is
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
50 available.
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
51
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
52 :return: a list of locale identifiers (strings)
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
53 :rtype: `list`
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
54 :since: version 0.8.1
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
55 """
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
56 return [stem for stem, extension in [
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
57 os.path.splitext(filename) for filename in os.listdir(_dirname)
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
58 ] if extension == '.dat' and stem != 'root']
170cffc66554 Add a command-line option that prints out all available locales. Closes #24.
cmlenz
parents: 99
diff changeset
59
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
60
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
61 def load(name, merge_inherited=True):
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
62 """Load the locale data for the given locale.
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
63
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
64 The locale data is a dictionary that contains much of the data defined by
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
65 the Common Locale Data Repository (CLDR). This data is stored as a
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
66 collection of pickle files inside the ``babel`` package.
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
67
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
68 >>> d = load('en_US')
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
69 >>> d['languages']['sv']
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
70 u'Swedish'
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
71
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
72 Note that the results are cached, and subsequent requests for the same
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
73 locale return the same dictionary:
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
74
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
75 >>> d1 = load('en_US')
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
76 >>> d2 = load('en_US')
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
77 >>> d1 is d2
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
78 True
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
79
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
80 :param name: the locale identifier string (or "root")
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
81 :param merge_inherited: whether the inherited data should be merged into
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
82 the data of the requested locale
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
83 :return: the locale data
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
84 :rtype: `dict`
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
85 :raise `IOError`: if no locale data file is found for the given locale
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
86 identifer, or one of the locales it inherits from
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
87 """
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
88 _cache_lock.acquire()
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
89 try:
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
90 data = _cache.get(name)
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
91 if not data:
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
92 # Load inherited data
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
93 if name == 'root' or not merge_inherited:
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
94 data = {}
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
95 else:
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
96 parts = name.split('_')
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
97 if len(parts) == 1:
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
98 parent = 'root'
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
99 else:
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
100 parent = '_'.join(parts[:-1])
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
101 data = load(parent).copy()
41
e967fbafcda1 Load locale data lazily to avoid penalizing usage of `Locale` objects when no locale data is actually needed.
cmlenz
parents: 26
diff changeset
102 filename = os.path.join(_dirname, '%s.dat' % name)
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
103 fileobj = open(filename, 'rb')
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
104 try:
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
105 if name != 'root' and merge_inherited:
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
106 merge(data, pickle.load(fileobj))
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
107 else:
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
108 data = pickle.load(fileobj)
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
109 _cache[name] = data
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
110 finally:
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
111 fileobj.close()
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
112 return data
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
113 finally:
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
114 _cache_lock.release()
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
115
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
116
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
117 def merge(dict1, dict2):
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
118 """Merge the data from `dict2` into the `dict1` dictionary, making copies
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
119 of nested dictionaries.
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
120
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
121 >>> d = {1: 'foo', 3: 'baz'}
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
122 >>> merge(d, {1: 'Foo', 2: 'Bar'})
440
b1c8cc751458 remove sorted and don't assume dict ordering (Python 2.3 & Jython compat)
pjenvey
parents: 399
diff changeset
123 >>> items = d.items(); items.sort(); items
b1c8cc751458 remove sorted and don't assume dict ordering (Python 2.3 & Jython compat)
pjenvey
parents: 399
diff changeset
124 [(1, 'Foo'), (2, 'Bar'), (3, 'baz')]
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
125
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
126 :param dict1: the dictionary to merge into
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
127 :param dict2: the dictionary containing the data that should be merged
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
128 """
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
129 for key, val2 in dict2.items():
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
130 if val2 is not None:
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
131 val1 = dict1.get(key)
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
132 if isinstance(val2, dict):
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
133 if val1 is None:
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
134 val1 = {}
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
135 if isinstance(val1, Alias):
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
136 val1 = (val1, val2)
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
137 elif isinstance(val1, tuple):
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
138 alias, others = val1
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
139 others = others.copy()
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
140 merge(others, val2)
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
141 val1 = (alias, others)
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
142 else:
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
143 val1 = val1.copy()
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
144 merge(val1, val2)
26
710090104678 * Reduce size of locale data pickles by only storing the data provided by each locale itself, and merging inherited data at runtime.
cmlenz
parents:
diff changeset
145 else:
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
146 val1 = val2
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
147 dict1[key] = val1
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
148
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
149
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
150 class Alias(object):
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
151 """Representation of an alias in the locale data.
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
152
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
153 An alias is a value that refers to some other part of the locale data,
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
154 as specified by the `keys`.
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
155 """
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
156
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
157 def __init__(self, keys):
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
158 self.keys = tuple(keys)
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
159
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
160 def __repr__(self):
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
161 return '<%s %r>' % (type(self).__name__, self.keys)
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
162
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
163 def resolve(self, data):
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
164 """Resolve the alias based on the given data.
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
165
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
166 This is done recursively, so if one alias resolves to a second alias,
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
167 that second alias will also be resolved.
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
168
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
169 :param data: the locale data
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
170 :type data: `dict`
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
171 """
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
172 base = data
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
173 for key in self.keys:
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
174 data = data[key]
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
175 if isinstance(data, Alias):
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
176 data = data.resolve(base)
399
3510257d57d3 Fix for memory leak reported in #128. Thanks to Manlio Perillo for reporting the problem.
cmlenz
parents: 375
diff changeset
177 elif isinstance(data, tuple):
3510257d57d3 Fix for memory leak reported in #128. Thanks to Manlio Perillo for reporting the problem.
cmlenz
parents: 375
diff changeset
178 alias, others = data
3510257d57d3 Fix for memory leak reported in #128. Thanks to Manlio Perillo for reporting the problem.
cmlenz
parents: 375
diff changeset
179 data = alias.resolve(base)
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
180 return data
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
181
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
182
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
183 class LocaleDataDict(DictMixin, dict):
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
184 """Dictionary wrapper that automatically resolves aliases to the actual
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
185 values.
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
186 """
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
187
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
188 def __init__(self, data, base=None):
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
189 dict.__init__(self, data)
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
190 if base is None:
399
3510257d57d3 Fix for memory leak reported in #128. Thanks to Manlio Perillo for reporting the problem.
cmlenz
parents: 375
diff changeset
191 base = data
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
192 self.base = base
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
193
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
194 def __getitem__(self, key):
399
3510257d57d3 Fix for memory leak reported in #128. Thanks to Manlio Perillo for reporting the problem.
cmlenz
parents: 375
diff changeset
195 orig = val = dict.__getitem__(self, key)
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
196 if isinstance(val, Alias): # resolve an alias
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
197 val = val.resolve(self.base)
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
198 if isinstance(val, tuple): # Merge a partial dict with an alias
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
199 alias, others = val
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
200 val = alias.resolve(self.base).copy()
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
201 merge(val, others)
399
3510257d57d3 Fix for memory leak reported in #128. Thanks to Manlio Perillo for reporting the problem.
cmlenz
parents: 375
diff changeset
202 if type(val) is dict: # Return a nested alias-resolving dict
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
203 val = LocaleDataDict(val, base=self.base)
399
3510257d57d3 Fix for memory leak reported in #128. Thanks to Manlio Perillo for reporting the problem.
cmlenz
parents: 375
diff changeset
204 if val is not orig:
3510257d57d3 Fix for memory leak reported in #128. Thanks to Manlio Perillo for reporting the problem.
cmlenz
parents: 375
diff changeset
205 self[key] = val
375
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
206 return val
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
207
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
208 def copy(self):
369300a7ebd3 Implement support for aliases in the CLDR data. Closes #68. Also, update to CLDR 1.6, and a much improved `dump_data` script.
cmlenz
parents: 233
diff changeset
209 return LocaleDataDict(dict.copy(self), base=self.base)
Copyright (C) 2012-2017 Edgewall Software