Mercurial > genshi > genshi-test

annotate genshi/template/interpolation.py @ 647:5af131b37ab4 experimental-sandboxed

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
restricted is the new secure
author aronacher
date Wed, 26 Sep 2007 17:56:41 +0000
parents 9ada030ad986
children
rev   line source
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
1 # -*- coding: utf-8 -*-
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
2 #
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
3 # Copyright (C) 2007 Edgewall Software
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
4 # All rights reserved.
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
5 #
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
6 # This software is licensed as described in the file COPYING, which
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
7 # you should have received as part of this distribution. The terms
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
8 # are also available at http://genshi.edgewall.org/wiki/License.
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
9 #
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
10 # This software consists of voluntary contributions made by many
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
11 # individuals. For the exact contribution history, see the revision
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
12 # history and logs, available at http://genshi.edgewall.org/log/.
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
13
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
14 """String interpolation routines, i.e. the splitting up a given text into some
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
15 parts that are literal strings, and others that are Python expressions.
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
16 """
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
17
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
18 from itertools import chain
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
19 import os
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
20 from tokenize import tokenprog
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
21
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
22 from genshi.core import TEXT
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
23 from genshi.template.base import TemplateSyntaxError, EXPR
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
24 from genshi.template.eval import Expression
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
25
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
26 __all__ = ['interpolate']
425
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
27 __docformat__ = 'restructuredtext en'
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
28
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
29 NAMESTART = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_'
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
30 NAMECHARS = NAMESTART + '.0123456789'
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
31 PREFIX = '$'
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
32
442
ff7c72b52fb2 Back out [510] and instead implement configurable error handling modes. The default is the old 0.3.x behaviour, but more strict error handling is available as an option.
cmlenz
parents: 425
diff changeset
33 def interpolate(text, basedir=None, filename=None, lineno=-1, offset=0,
647
5af131b37ab4 restricted is the new secure
aronacher
parents: 606
diff changeset
34 lookup='strict', restricted=False):
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
35 """Parse the given string and extract expressions.
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
36
425
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
37 This function is a generator that yields `TEXT` events for literal strings,
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
38 and `EXPR` events for expressions, depending on the results of parsing the
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
39 string.
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
40
442
ff7c72b52fb2 Back out [510] and instead implement configurable error handling modes. The default is the old 0.3.x behaviour, but more strict error handling is available as an option.
cmlenz
parents: 425
diff changeset
41 >>> for kind, data, pos in interpolate("hey ${foo}bar"):
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
42 ... print kind, `data`
442
ff7c72b52fb2 Back out [510] and instead implement configurable error handling modes. The default is the old 0.3.x behaviour, but more strict error handling is available as an option.
cmlenz
parents: 425
diff changeset
43 TEXT u'hey '
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
44 EXPR Expression('foo')
442
ff7c72b52fb2 Back out [510] and instead implement configurable error handling modes. The default is the old 0.3.x behaviour, but more strict error handling is available as an option.
cmlenz
parents: 425
diff changeset
45 TEXT u'bar'
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
46
425
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
47 :param text: the text to parse
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
48 :param basedir: base directory of the file in which the text was found
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
49 (optional)
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
50 :param filename: basename of the file in which the text was found (optional)
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
51 :param lineno: the line number at which the text was found (optional)
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
52 :param offset: the column number at which the text starts in the source
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
53 (optional)
442
ff7c72b52fb2 Back out [510] and instead implement configurable error handling modes. The default is the old 0.3.x behaviour, but more strict error handling is available as an option.
cmlenz
parents: 425
diff changeset
54 :param lookup: the variable lookup mechanism; either "lenient" (the
ff7c72b52fb2 Back out [510] and instead implement configurable error handling modes. The default is the old 0.3.x behaviour, but more strict error handling is available as an option.
cmlenz
parents: 425
diff changeset
55 default), "strict", or a custom lookup class
425
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
56 :return: a list of `TEXT` and `EXPR` events
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
57 :raise TemplateSyntaxError: when a syntax error in an expression is
5b248708bbed Try to use proper reStructuredText for docstrings throughout.
cmlenz
parents: 422
diff changeset
58 encountered
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
59 """
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
60 filepath = filename
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
61 if filepath and basedir:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
62 filepath = os.path.join(basedir, filepath)
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
63 pos = [filepath, lineno, offset]
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
64
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
65 textbuf = []
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
66 textpos = None
422
95089b6e37ca More work to include absolute file paths in exceptions.
cmlenz
parents: 407
diff changeset
67 for is_expr, chunk in chain(lex(text, pos, filepath), [(True, '')]):
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
68 if is_expr:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
69 if textbuf:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
70 yield TEXT, u''.join(textbuf), textpos
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
71 del textbuf[:]
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
72 textpos = None
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
73 if chunk:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
74 try:
442
ff7c72b52fb2 Back out [510] and instead implement configurable error handling modes. The default is the old 0.3.x behaviour, but more strict error handling is available as an option.
cmlenz
parents: 425
diff changeset
75 expr = Expression(chunk.strip(), pos[0], pos[1],
647
5af131b37ab4 restricted is the new secure
aronacher
parents: 606
diff changeset
76 lookup=lookup, restricted=restricted)
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
77 yield EXPR, expr, tuple(pos)
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
78 except SyntaxError, err:
422
95089b6e37ca More work to include absolute file paths in exceptions.
cmlenz
parents: 407
diff changeset
79 raise TemplateSyntaxError(err, filepath, pos[1],
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
80 pos[2] + (err.offset or 0))
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
81 else:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
82 textbuf.append(chunk)
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
83 if textpos is None:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
84 textpos = tuple(pos)
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
85
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
86 if '\n' in chunk:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
87 lines = chunk.splitlines()
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
88 pos[1] += len(lines) - 1
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
89 pos[2] += len(lines[-1])
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
90 else:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
91 pos[2] += len(chunk)
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
92
422
95089b6e37ca More work to include absolute file paths in exceptions.
cmlenz
parents: 407
diff changeset
93 def lex(text, textpos, filepath):
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
94 offset = pos = 0
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
95 end = len(text)
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
96 escaped = False
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
97
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
98 while 1:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
99 if escaped:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
100 offset = text.find(PREFIX, offset + 2)
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
101 escaped = False
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
102 else:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
103 offset = text.find(PREFIX, pos)
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
104 if offset < 0 or offset == end - 1:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
105 break
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
106 next = text[offset + 1]
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
107
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
108 if next == '{':
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
109 if offset > pos:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
110 yield False, text[pos:offset]
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
111 pos = offset + 2
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
112 level = 1
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
113 while level:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
114 match = tokenprog.match(text, pos)
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
115 if match is None:
422
95089b6e37ca More work to include absolute file paths in exceptions.
cmlenz
parents: 407
diff changeset
116 raise TemplateSyntaxError('invalid syntax', filepath,
95089b6e37ca More work to include absolute file paths in exceptions.
cmlenz
parents: 407
diff changeset
117 *textpos[1:])
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
118 pos = match.end()
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
119 tstart, tend = match.regs[3]
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
120 token = text[tstart:tend]
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
121 if token == '{':
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
122 level += 1
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
123 elif token == '}':
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
124 level -= 1
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
125 yield True, text[offset + 2:pos - 1]
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
126
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
127 elif next in NAMESTART:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
128 if offset > pos:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
129 yield False, text[pos:offset]
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
130 pos = offset
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
131 pos += 1
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
132 while pos < end:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
133 char = text[pos]
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
134 if char not in NAMECHARS:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
135 break
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
136 pos += 1
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
137 yield True, text[offset + 1:pos].strip()
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
138
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
139 elif not escaped and next == PREFIX:
526
bd13c96cbfe4 Fix interpolation of short-form expressions that include literal text before the expression. Thanks to Alec for reporting the issue.
cmlenz
parents: 442
diff changeset
140 if offset > pos:
bd13c96cbfe4 Fix interpolation of short-form expressions that include literal text before the expression. Thanks to Alec for reporting the issue.
cmlenz
parents: 442
diff changeset
141 yield False, text[pos:offset]
407
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
142 escaped = True
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
143 pos = offset + 1
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
144
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
145 else:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
146 yield False, text[pos:offset + 1]
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
147 pos = offset + 1
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
148
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
149 if pos < end:
ea71a51e0258 Move string interpolation code into separate module (`genshi.template.interpolation`).
cmlenz
parents:
diff changeset
150 yield False, text[pos:]
Copyright (C) 2012-2017 Edgewall Software