Mercurial > genshi > mirror

annotate genshi/template/interpolation.py @ 425:073640758a42 trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Try to use proper reStructuredText for docstrings throughout.
author cmlenz
date Thu, 22 Mar 2007 12:45:18 +0000
parents 5d08a744636e
children 97544725bb7f
rev   line source
407
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
1 # -*- coding: utf-8 -*-
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
2 #
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
3 # Copyright (C) 2007 Edgewall Software
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
4 # All rights reserved.
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
5 #
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
6 # This software is licensed as described in the file COPYING, which
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
7 # you should have received as part of this distribution. The terms
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
8 # are also available at http://genshi.edgewall.org/wiki/License.
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
9 #
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
10 # This software consists of voluntary contributions made by many
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
11 # individuals. For the exact contribution history, see the revision
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
12 # history and logs, available at http://genshi.edgewall.org/log/.
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
13
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
14 """String interpolation routines, i.e. the splitting up a given text into some
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
15 parts that are literal strings, and others that are Python expressions.
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
16 """
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
17
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
18 from itertools import chain
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
19 import os
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
20 from tokenize import tokenprog
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
21
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
22 from genshi.core import TEXT
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
23 from genshi.template.base import TemplateSyntaxError, EXPR
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
24 from genshi.template.eval import Expression
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
25
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
26 __all__ = ['interpolate']
425
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
27 __docformat__ = 'restructuredtext en'
407
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
28
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
29 NAMESTART = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_'
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
30 NAMECHARS = NAMESTART + '.0123456789'
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
31 PREFIX = '$'
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
32
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
33 def interpolate(text, basedir=None, filename=None, lineno=-1, offset=0):
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
34 """Parse the given string and extract expressions.
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
35
425
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
36 This function is a generator that yields `TEXT` events for literal strings,
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
37 and `EXPR` events for expressions, depending on the results of parsing the
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
38 string.
407
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
39
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
40 >>> for kind, data, pos in interpolate("$foo bar"):
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
41 ... print kind, `data`
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
42 EXPR Expression('foo')
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
43 TEXT u' bar'
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
44
425
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
45 :param text: the text to parse
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
46 :param basedir: base directory of the file in which the text was found
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
47 (optional)
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
48 :param filename: basename of the file in which the text was found (optional)
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
49 :param lineno: the line number at which the text was found (optional)
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
50 :param offset: the column number at which the text starts in the source
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
51 (optional)
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
52 :return: a list of `TEXT` and `EXPR` events
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
53 :raise TemplateSyntaxError: when a syntax error in an expression is
073640758a42 Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
54 encountered
407
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
55 """
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
56 filepath = filename
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
57 if filepath and basedir:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
58 filepath = os.path.join(basedir, filepath)
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
59 pos = [filepath, lineno, offset]
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
60
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
61 textbuf = []
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
62 textpos = None
422
5d08a744636e More work to include absolute file paths in exceptions.
cmlenz
parents: 407
diff changeset
63 for is_expr, chunk in chain(lex(text, pos, filepath), [(True, '')]):
407
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
64 if is_expr:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
65 if textbuf:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
66 yield TEXT, u''.join(textbuf), textpos
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
67 del textbuf[:]
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
68 textpos = None
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
69 if chunk:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
70 try:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
71 expr = Expression(chunk.strip(), pos[0], pos[1])
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
72 yield EXPR, expr, tuple(pos)
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
73 except SyntaxError, err:
422
5d08a744636e More work to include absolute file paths in exceptions.
cmlenz
parents: 407
diff changeset
74 raise TemplateSyntaxError(err, filepath, pos[1],
407
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
75 pos[2] + (err.offset or 0))
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
76 else:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
77 textbuf.append(chunk)
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
78 if textpos is None:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
79 textpos = tuple(pos)
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
80
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
81 if '\n' in chunk:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
82 lines = chunk.splitlines()
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
83 pos[1] += len(lines) - 1
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
84 pos[2] += len(lines[-1])
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
85 else:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
86 pos[2] += len(chunk)
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
87
422
5d08a744636e More work to include absolute file paths in exceptions.
cmlenz
parents: 407
diff changeset
88 def lex(text, textpos, filepath):
407
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
89 offset = pos = 0
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
90 end = len(text)
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
91 escaped = False
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
92
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
93 while 1:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
94 if escaped:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
95 offset = text.find(PREFIX, offset + 2)
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
96 escaped = False
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
97 else:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
98 offset = text.find(PREFIX, pos)
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
99 if offset < 0 or offset == end - 1:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
100 break
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
101 next = text[offset + 1]
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
102
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
103 if next == '{':
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
104 if offset > pos:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
105 yield False, text[pos:offset]
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
106 pos = offset + 2
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
107 level = 1
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
108 while level:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
109 match = tokenprog.match(text, pos)
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
110 if match is None:
422
5d08a744636e More work to include absolute file paths in exceptions.
cmlenz
parents: 407
diff changeset
111 raise TemplateSyntaxError('invalid syntax', filepath,
5d08a744636e More work to include absolute file paths in exceptions.
cmlenz
parents: 407
diff changeset
112 *textpos[1:])
407
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
113 pos = match.end()
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
114 tstart, tend = match.regs[3]
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
115 token = text[tstart:tend]
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
116 if token == '{':
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
117 level += 1
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
118 elif token == '}':
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
119 level -= 1
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
120 yield True, text[offset + 2:pos - 1]
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
121
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
122 elif next in NAMESTART:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
123 if offset > pos:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
124 yield False, text[pos:offset]
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
125 pos = offset
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
126 pos += 1
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
127 while pos < end:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
128 char = text[pos]
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
129 if char not in NAMECHARS:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
130 break
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
131 pos += 1
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
132 yield True, text[offset + 1:pos].strip()
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
133
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
134 elif not escaped and next == PREFIX:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
135 escaped = True
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
136 pos = offset + 1
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
137
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
138 else:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
139 yield False, text[pos:offset + 1]
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
140 pos = offset + 1
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
141
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
142 if pos < end:
f37d8e6acdf2 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
143 yield False, text[pos:]
Copyright (C) 2012-2017 Edgewall Software